evers4

I’d like to share my design, but I’m not sure how to send it to you due to the size. I’ve migrated the project to UDP, though I’m having trouble receiving all the data from the FPGA. The FPGA is periodically sending 64 MB of data to a python script listening form a remote pc. What happening is, a function on the FPGA loops calling the Xilinx function udp_sendto(), transferring a packet at a time, until 64MB have been processed. After this I wait 1 second and do this again. After 4-11 times of running my function I get an error in putty over the UART “pack dropped, no space”. It looks like, there is a transmit buffer somewhere that is being full. I don’t know how to flush it. Actually, I’m not sure why its not being flushed in my function, because after every call to udp_sendto() I free the pbuf passed to it using pbuf_free(). I’ve attached the relevant parts of the Vitis code & the entire python script. If we can find a way to transfer the entire Vitis/Vivado project ill do that. The guts of the code is in main.c & echo.c, and the project is a modified version of the Xilinx echo server example project.scratch_pad_script.py Vitis Project Code.7z

Hi artvvb, I've managed to resolve the issue with the factored out signal. As well as another problem involving the ILA. I found that I couldn't connect to the ILA debug core when programming the FPGA in Vivado, only when I ran the Vitis Project (programming the FPGA with both the bitstream and app) could I connect. I suspect that this is related to having the Zynq in the design which provides the Fabric clock driving the core. Now, I'm having success with the entire data path. From fabric to remote PC. However, I'm seeing errors in the data received. The error rate is low and is observed when sending large quantities of it (I'm sending 64MB at a time). What I'm finding is the combined summation of the len argument in sent_callback is greater than what I'd expect. I'm suspecting that this could be due to TCP retransmissions. As it turns out the callback indicates an acknowledgement from the receiver side of data to be sent, not that it was successfully transmitted. I'm still sending a counting value 0->UINT32_MAX_VALUE, and verifying that each value received is value[i+1]=value[i] (Except on overflow). That data is mostly correct, but there are some segments with errors. I'm suspecting data reordering, retransmission, or transmission errors. I'd like to try using the UDP protocol and compare the results. Is this straight forward? How would I go about modifying the Vitis project & script?

Hi and happy new year, thanks for following up on the example and updating the ethernet example. I don't think the updates will directly effect my current project since it has diverged somewhat since we started working together. The general issue I’d like to work out with you is debugging the data path from the Firmware to remote PC. I’m receiving what looks like mostly valid data, but with occasional hiccups. I’m still seeing a data rate error of about 0.4%. Among the few issues I’m still trying to resolve pertains towards the firmware that generates the data forwarded out the Zybo board’s ethernet port. For some reason I’m seeing one of my debug signals (m00_axi_bresp[1:0]) being factored out of my synthesized design. I’d like to debug this particular signal using the embedded logic analyzer. Its unclear to me why this is happening, since this block (AxiDeviceMasterSlave) is a barebones IP generated using the Vivado IP Package Manger. An AXI-Lite example design with a master and slave port. I did check the path of this signal, it begins from a Xilinx IP called AXI Memory Interconnect, and ends within the generated IP being captured by a register. Is there any reason why this signal could be factored out? And how can I tell the tools to keep this one?

HI again artvvb, let me update you on the status of the project: I'm able to receive large quantities of data from the FPGA over Ethernet using the (tcp_write/tcp_mss). I've tested packet sizes up to 65500, with num_bytes up to 1 million. I'm seeing error rates of approximately 0.4%. For example, for 1,000,000 bytes requested I see 4374 errors. I see that the packet size throttles as the ring buffer on the FPGA empties (packet_size=[65500, 65500, 5012, 1446, 1446, 1446, 1446, ...]). I tied out the tcp_err function, however it doesn't call the supplied callback when using client.close() in the script, nor when physically unplugging the Zybo Ethernet port. The problem I'm trying to solve is that when relaunching the script I have to reboot the FPGA. It would be great too if I could disconnect/reconnect the Zybo Ethernet port though. When I run use the code from the revised forum post I still receive errors when requesting over 1446 bytes. Which I just noticed matches the received packet size I'm observing as the ring buffer empties. Do you still see error when running the revised code?

Great news, the original goal has mostly been accomplished at this point, thank you! I'm receiving data over Ethernet from the custom IP in the firmware, and better still, so far it appears correct. In main() I'm sending data once a connection is made via the accept callback. My question now is how can I detect an active connection. The accept callback increments the connection variable, but it doesn't decrement it. So I'm wondering how to reliably detect that an active connection exists. The way the C code works is it fills a ring buffer with data, from the IP in the firmware, and pushes that out to an active connection as long as new data exists. I'm currently looping on 3 conditions: an active connection, data available, & a state I call sending. I'm using the connection variable in the accept callback to determine if a connection is active. But, this value does seem to track disconnects. err_t accept_callback(void *arg, struct tcp_pcb *newpcb, err_t err) { static int connection = 1; xil_printf("Echo: c_pcb=%u\n\r",(u32)newpcb); /* set the receive callback for this connection */ tcp_recv(newpcb, recv_callback); /* set the sent callback for this connection */ tcp_sent(newpcb, sent_callback); // <- add this /* just use an integer number indicating the connection id as the callback argument */ tcp_arg(newpcb, (void*)(UINTPTR)connection); /* increment for subsequent accepted connections */ connection++; return ERR_OK; }

Regarding the fix to the Ethernet Example, I replaced the calls to tcp_sndbuf(tpcb) with tcp_mss(tpcb) following the updated blog post. However, I'm still seeing errors when requesting more than 1446. Do you still see this problem on your end, if not I'm wondering what could be different with my configuration.

I'm attempting now to modify the Python script to request an arbitrary amount of data/bytes from the FPGA. Since I'm not sending any commands to the FPGA, such as how many bytes to send I tried removing the call to client.send(). However, when I run this code the script hangs. Is it possible to have one way communication here by only calling client.recv()? import socket client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) client.connect(("192.168.1.10", 7)) # connect to the server; enter your server's IP and port here # number of bytes we're requesting from the server; change this as desired num_bytes = 5000 # arbitrary packet size, max number of bytes we'll receive at once packet_size = 256 #256 # note: little endian is important, requirement for Zynq-7000 to easily translate the sent number to an int without reordering #print(f"requesting {num_bytes.to_bytes(2, 'little')} bytes") #client.send(num_bytes.to_bytes(2, 'little')) # request {num_bytes} bytes in response # loop while calling recv to receive data from the client until the expected number of bytes has been successfully transferred received = 0 while received < num_bytes: data = client.recv(packet_size) for d in range(len(data)): print(f"data[{d}] ({data[d]})") received += len(data) if received == num_bytes: print("All data received!") else: print("Missing data!")

Ow wow! That did it, thanks for the easy fix! Side question, is it possible to read from DDR while within an ISR? Does it require any special consideration(s)?

Ah okay, is there anything I can provide you with to confirm? Does this fit this scenario where I'm able to perform a single write from the IP and read from the Zynq? I'm able to reread from this section of DDR, but the values don't continue to be updated by the IP. I checked the data written from the IP to DDR using the ILA as you had suggested earlier. I can confirm the write values out of the IP are correct.

Hi Artvvb, I’m now debugging the data transferred from this IP to the DDR memory of the Zybo board. The purpose of custom_int is to tell the Zynq new data is available. Prior to each trigger the IP writes 100 32bit words to DDR addresses 0-99 (This data is a count value 0 to 4 billion). When I read this data in my C code from the Zynq the first write of 100 words works. However, I’m not seeing this data updated in following writes. The IP I’m using is a modified version of that generated by Xilinx’s IP Packaging Tool, where I specified a generic IP with a single master & single slave AXI-Lite port. I’ve attached the updated firmware for the IP, can you spot any issues here? Is there anything else I can provide you with to help in debugging this issue? I’ve excluded the Slave AXI-Lite HDL, since its currently unused. AxiDeviceMasterSlave_v1_0.v AxiDeviceMasterSlave_v1_0_M00_AXI.v

I connected up a slow 200MHz scope to the custom_int signal in the block design. What I see is distorted pulse that does otherwise appear correct. After this, I replaced the single cycle pulse by a slow 4Hz clock, the ISR appears to be working as expected now. I expected some distortion on the probe, but I don't think the interface between the Zynq/IP should be effected by this. I'm cautiously optimistic that the problem has been resolved. I'm now attempting to merge this code with the Ethernet Example you provided. I'll follow up with you after making some progress. Thanks!

Great information, also the link you provided is very helpful. Initially, I had though both the Fabric and Zynq were operating synchronously at 100MHz. I see that this is not the case, the Zynq is actually clocked at 666MHz, while the custom IP in the fabric (clocked off of the Zynq's FCLK) is running at 100MHz. To complicate matters the documentation on the GIC states that it operates at half the CPU/Zynq frequency. I'm not sure being asynchronous is an issue. I've configured a one cycle trigger here, perhaps that could be an issue? I'm about to test replacing this pulse a 2Hz clock to see if that could have an effect. As for XScuGic_SetPriorityTriggerType, based upon this documentation it looks like I'm working with an SPI & the trigger type should be set to 3 for a rising edge trigger.

The output I see is shown below after waiting 5 seconds following the line: "--- Entering main() --". Every five seconds results in another 3 lines of "--- Entering CustomIntrHandler() ---" --- Entering main() --- --- Entering CustomIntrHandler() --- --- Entering CustomIntrHandler() --- --- Entering CustomIntrHandler() ---

My mistake, I didn't include the top level HDL. Here is that file. That's helpful information though, I thought that the value 0x3 configured the interrupt to trigger on a rising edge. I don't fully follow the documentation written for this function (attached below). What do the acronyms SFI, PPI, SPI stand for? This indictates that 1 is active high & 3 is rising edge. It doesn't specify what values 0 & 2 correspond to. It appears that each bit pair of the 8bit uTrigger corresponds to 1 of 4 interrupts, and the interrupt ID assigned to my custom IP corresponds to one of these entries (I think....). My goal is to set this to a rising edge trigger and not to disturb the interrupts already used in the Ethernet Example you provided (Since I'm merging the 2 projects). One other thing I'd like to indicate is that I left the priority as it was set in the Xilinx DMA example (since I don't know what would be an appropriate value). /****************************************************************************/ /** * Sets the interrupt priority and trigger type for the specificd IRQ source. * * @param InstancePtr is a pointer to the instance to be worked on. * @param Int_Id is the IRQ source number to modify * @param Priority is the new priority for the IRQ source. 0 is highest * priority, 0xF8(248) is lowest. There are 32 priority levels * supported with a step of 8. Hence the supported priorities are * 0, 8, 16, 32, 40 ..., 248. * @param Trigger is the new trigger type for the IRQ source. * Each bit pair describes the configuration for an INT_ID. * SFI Read Only b10 always * PPI Read Only depending on how the PPIs are configured. * b01 Active HIGH level sensitive * b11 Rising edge sensitive * SPI LSB is read only. * b01 Active HIGH level sensitive * b11 Rising edge sensitive/ * * @return None. * * @note None. * *****************************************************************************/ void XScuGic_SetPriorityTriggerType(XScuGic *InstancePtr, u32 Int_Id, u8 Priority, u8 Trigger) AxiDeviceMasterSlave_v1_0.v AxiDeviceMasterSlave_v1_0_M00_AXI.v

That's great, thanks for resolving the issue with the example. Regarding AXI, I've continued to experiment with the AXI Lite Mater/Slave protocols & had success. I generated a custom IP with one master & one slave again using the Vivado IP Packing tool. I've modified it to write to addresses 0-99 of the Zybo's DDR using the AXI Master Port (I simply wrote the address number to the address, so values 0-99). I was then able to read these values back in a loop using the function Xil_in32. for(Index = 0; Index < NUMBER_OF_WORDS; Index ++) { xil_printf("Data Received %d\r\n", Xil_In32(0x00000000 + (Index*4))); } After this, I added a custom output port to this IP to send an interrupt to the Zynq each time it finishes writing to the DDR. So that the Software knows that new data is available to be buffered & then sent over ethernet. However, this is where I'm currently stuck. While I was able to set up an ISR to intercept the data, it was firing repeatedly. I've since modified the IP to send an interrupt ever five seconds using a counter. This helped, but now I see the interrupt firing 2 to 3 times every five seconds. Do you see what could be the problem here? I've attached the debug version of the Vitis project code, which was implemented using the HelloWorld example (only modifying the hello.c file with the code shown). I based the interrupt configuration off of Xilinx's example code (xaxidma_example_simple_intr.c) found at the location below. I've also attached the original C code & firmware for the Custom IP as well. C:\Vitis\2023.1\data\embeddedsw\XilinxProcessorIPLib\drivers\axidma_v9_16\examples #include <stdio.h> #include "platform.h" #include "xil_printf.h" #include "xparameters.h" #include "xil_io.h" #include "sleep.h" /////////////////////////////// // Interrupt Declaration /////////////////////////////// #include "xscugic.h" #define CUSTOM_INTR_ID 61U #define INTC_DEVICE_ID XPAR_SCUGIC_SINGLE_DEVICE_ID static XScuGic Intc; /* Instance of the Interrupt Controller */ #define INTC_HANDLER XScuGic_InterruptHandler static int SetupIntrSystem(XScuGic * IntcInstancePtr, u16 CustomIntrId); static void DisableIntrSystem(XScuGic * IntcInstancePtr, u16 CustomIntrId); int main() { int Status; xil_printf("\r\n--- Entering main() --- \r\n"); /* Set up Interrupt system */ Status = SetupIntrSystem(&Intc, CUSTOM_INTR_ID); if (Status != XST_SUCCESS) { xil_printf("Failed intr setup\r\n"); return XST_FAILURE; } while(1){} DisableIntrSystem(&Intc, CUSTOM_INTR_ID); xil_printf("--- Exiting main() --- \r\n"); return 0; } //////////////////////////////////////////////////////////////////////////////////// // This is the Custom AXI Device Interrupt handler function. //////////////////////////////////////////////////////////////////////////////////// static void CustomIntrHandler(void *Callback) { xil_printf("--- Entering CustomIntrHandler() --- \r\n"); } //////////////////////////////////////////////////////////////////////////////////// // This function setups the interrupt system so interrupts can occur for the // Custom AXI IP, it assumes XScuGic component exists in the hardware system. //////////////////////////////////////////////////////////////////////////////////// static int SetupIntrSystem(XScuGic * IntcInstancePtr, u16 CustomIntrId) { int Status; XScuGic_Config *IntcConfig; /* * Initialize the interrupt controller driver so that it is ready to * use. */ IntcConfig = XScuGic_LookupConfig(INTC_DEVICE_ID); if (NULL == IntcConfig) { xil_printf("Failed in IntcConfig\r\n"); return XST_FAILURE; } Status = XScuGic_CfgInitialize(IntcInstancePtr, IntcConfig, IntcConfig->CpuBaseAddress); if (Status != XST_SUCCESS) { xil_printf("Failed in XScuGic_CfgInitialize\r\n"); return XST_FAILURE; } //XScuGic_SetPriorityTriggerType(IntcInstancePtr, CustomIntrId, 0xA0, 0x3); XScuGic_SetPriorityTriggerType(IntcInstancePtr, CustomIntrId, 0xA0, 0x3); /* * Connect the device driver handler that will be called when an * interrupt for the device occurs, the handler defined above performs * the specific interrupt processing for the device. */ void* argument; Status = XScuGic_Connect(IntcInstancePtr, CustomIntrId, (Xil_InterruptHandler)CustomIntrHandler, argument); if (Status != XST_SUCCESS) { xil_printf("Failed in XScuGic_Connect\r\n"); return Status; } XScuGic_Enable(IntcInstancePtr, CustomIntrId); /* Enable interrupts from the hardware */ Xil_ExceptionInit(); Xil_ExceptionRegisterHandler(XIL_EXCEPTION_ID_INT, (Xil_ExceptionHandler)INTC_HANDLER, (void *)IntcInstancePtr); Xil_ExceptionEnable(); return XST_SUCCESS; } //////////////////////////////////////////////////////////////////////////////////// // This function disables the interrupts for DMA engine. //////////////////////////////////////////////////////////////////////////////////// static void DisableIntrSystem(XScuGic * IntcInstancePtr, u16 CustomIntrId) { XScuGic_Disconnect(IntcInstancePtr, CustomIntrId); } AxiDeviceMasterSlave_v1_0_M00_AXI.v helloworld.c