Implimenting Etherent on the Zybo Z7 development board

[evers4]

I'm working with a Zybo Z7 development board and trying to implement Ethernet. I'm new to Ethernet and have been looking for an example project that utilizes it.

The example design for the (AXI 1G/2.5G Ethernet Subsystem) is quite bare bones and I don't see how to configure it for the Zybo. Addtionally, I'm looking connect the Ethernet Subsystem to an existing project that generates bytes. I'm attempting to have those bytes streamed to a PC. My plan is to utiize a python script to capture this data.

 

Thank you so much!

Chris

[artvvb]

Hi Chris,

The Ethernet PHY on the Zybo Z7 is connected to Zynq PS instead of the PL (the subsystem IP would potentially be used if it was on the PL side). This means that there are very few steps required to get a Vivado project that will support Ethernet - the board preset applied during block automation already enables Ethernet.

This tutorial for running Xilinx's LWIP TCP echo server example was recently updated: https://digilent.com/reference/programmable-logic/guides/zynq-servers, and this blog post extends off of it to do pretty much exactly what you are talking about with streaming bytes back up to a host via a Python script: https://digilent.com/blog/getting-out-of-the-echo-chamber-transferring-data-over-ethernet-using-zynq/.

Thanks,

Arthur

[evers4]

18 hours ago, artvvb said:

Hi Chris,

The Ethernet PHY on the Zybo Z7 is connected to Zynq PS instead of the PL (the subsystem IP would potentially be used if it was on the PL side). This means that there are very few steps required to get a Vivado project that will support Ethernet - the board preset applied during block automation already enables Ethernet.

This tutorial for running Xilinx's LWIP TCP echo server example was recently updated: https://digilent.com/reference/programmable-logic/guides/zynq-servers, and this blog post extends off of it to do pretty much exactly what you are talking about with streaming bytes back up to a host via a Python script: https://digilent.com/blog/getting-out-of-the-echo-chamber-transferring-data-over-ethernet-using-zynq/.

Thanks,

Arthur

Excellent,

Thank you Arthur. I'll read through the material you provided and follow up. To clarify, its not a requirement that I use the 1/2.5 Eth Subsystem I mentioned. It was simply the only example I had to work with at the time.

The design also has another requirement, that I can transmit and receive data using a protocol known as SpaceWire. For this ill need to support transferring/receiving data at a rate of 100MHz. I'm curious if the High Speed PMOD ports/connectors can support this.

Thank you so much!

Chris

[evers4]

Hi again Arthur,

the example echo server & follow project were very helpful. Now a couple of questions.

1. What data rate is used/supported by the Ethernet connection created here? While the UART serial connection is at 115,200 bits/sec displaying information regarding the server, is the Ethernet connection capable of supporting 100/1000Mbs speeds?
2. Just to confirm, this a TCP/IP connection with the server and not UDP?
3. Is Telnet required here, or does this design support SSH?

This gets me most of the way there. What I need to do for this project is transfer data from the fabric to a PC. I think this means transferring (in my case streaming) bytes to the Zynq so that it can be forwarded through the Ethernet connection. Can you advise on how to accomplish this? I believe I once read that a Fabric/PL based design can be encapsulated as a device which the Zynq can read/write from/to.

 

[artvvb]

On 11/22/2023 at 11:22 AM, evers4 said:

 

The design also has another requirement, that I can transmit and receive data using a protocol known as SpaceWire. For this ill need to support transferring/receiving data at a rate of 100MHz. I'm curious if the High Speed PMOD ports/connectors can support this.

I'm not familiar with SpaceWire. The Pmod ports won't successfully recover a 100 MHz signal. The top speed hasn't been characterized, but is likely less than 25 MHz and north of 10 MHz.

On 11/22/2023 at 12:31 PM, evers4 said:

 

1. What data rate is used/supported by the Ethernet connection created here? While the UART serial connection is at 115,200 bits/sec displaying information regarding the server, is the Ethernet connection capable of supporting 100/1000Mbs speeds?
2. Just to confirm, this a TCP/IP connection with the server and not UDP?
3. Is Telnet required here, or does this design support SSH?

1. It will negotiate for 100/1000 Mbps links, but I am unsure if it can sustain data rates close to this.

2. Yes, TCP/IP, though the library should also support UDP.

3. I believe telnet only.

There's some more info in XAPP1026, here: https://docs.xilinx.com/v/u/en-US/xapp1026. Also in the template application project's README. Other examples that use the lwIP library may also be helpful (though Digilent has not tested them):

On 11/22/2023 at 12:31 PM, evers4 said:

This gets me most of the way there. What I need to do for this project is transfer data from the fabric to a PC. I think this means transferring (in my case streaming) bytes to the Zynq so that it can be forwarded through the Ethernet connection. Can you advise on how to accomplish this? I believe I once read that a Fabric/PL based design can be encapsulated as a device which the Zynq can read/write from/to.

If the data rate is over 100 Mbps, DMA is probably required. The DMA Audio demo for the Zybo Z7 might be a starting point. It takes data coming in from the audio codec, puts it into an AXI stream, and then uses an AXI DMA controller to send data from that AXI stream to PS-accessible DDR memory - there're all sorts of finicky details about how this transfer works under the hood. There are also plenty of other approaches for PS/PL data transfer, depending on the specific data rate needed, and on how "bursty" the data is.

Thanks,

Arthur

[evers4]

On 11/27/2023 at 3:09 PM, artvvb said:

If the data rate is over 100 Mbps, DMA is probably required. The DMA Audio demo for the Zybo Z7 might be a starting point. It takes data coming in from the audio codec, puts it into an AXI stream, and then uses an AXI DMA controller to send data from that AXI stream to PS-accessible DDR memory - there're all sorts of finicky details about how this transfer works under the hood. There are also plenty of other approaches for PS/PL data transfer, depending on the specific data rate needed, and on how "bursty" the data is.

Great!

I'll take a look at the DMA audio demo. I'm looking to transfer streaming data at rates around 2-3Mbit/sec. If that's viable then the demo should be helpful, and a great start. I'll follow up with you after reviewing it.

Note: I was able to successfully run both the Echo & Getting out of the Echo Chamber examples. Though after adjusting the second example to transfer more than 1446 bytes the script reports errors in the data received. Its odd in that it occurs when the number of requested bytes is 1447 or greater. Do you see this as well?

[evers4]

Hi Arthur,
the example works as a starting point. As it stands the example abstracts away the hardware & software that transfers the audio data to/from memory.

Hardware Abstracted
It has abstracted the Audio Firmware into (axi_i2s_audio_v2_0). What I'm interested in is how this device talks AXI. How I can take data I've registered in the Firmware and forward this to the DMA Controller (AXI Direct Memory Access).

Software Abstracted
In the Vitis project there is the (demo.c) file. It shows how to record/play audio using functions contained in the audio driver (audio.c). Digging into (audio.c) I looked at the function (fnAudioRecord) which writes to memory using another function (Xil_Out32). I suspect there is another function (Xil_In32) for reading. As far a software is concerned would I be right in assuming that my custom firmware will read/write using the AXI protocol to addresses I choose, while my implementation of (demo.c) would tell the Zynq to read/write from them?

[artvvb]

On 11/29/2023 at 3:53 PM, evers4 said:

I'm looking to transfer streaming data at rates around 2-3Mbit/sec.

2-3 Mb/s is very doable with DMA. I've seen 125x4 MB/s streaming to DDR.

On 11/29/2023 at 3:53 PM, evers4 said:

 

Note: I was able to successfully run both the Echo & Getting out of the Echo Chamber examples. Though after adjusting the second example to transfer more than 1446 bytes the script reports errors in the data received. Its odd in that it occurs when the number of requested bytes is 1447 or greater. Do you see this as well?

I haven't seen this before but will attempt to reproduce. If possible, could you post your modified code?

On 11/29/2023 at 4:44 PM, evers4 said:

Hardware Abstracted
It has abstracted the Audio Firmware into (axi_i2s_audio_v2_0). What I'm interested in is how this device talks AXI. How I can take data I've registered in the Firmware and forward this to the DMA Controller (AXI Direct Memory Access).

Is data moving from DDR to the FPGA or vice-versa? They work pretty similarly with the DMA, but there are different catches to designing the hardware for each direction.

On 11/29/2023 at 4:44 PM, evers4 said:

Software Abstracted
In the Vitis project there is the (demo.c) file. It shows how to record/play audio using functions contained in the audio driver (audio.c). Digging into (audio.c) I looked at the function (fnAudioRecord) which writes to memory using another function (Xil_Out32). I suspect there is another function (Xil_In32) for reading. As far a software is concerned would I be right in assuming that my custom firmware will read/write using the AXI protocol to addresses I choose, while my implementation of (demo.c) would tell the Zynq to read/write from them?

Xil_In32 and Xil_Out32 are "just" macros to replace single-word memory accesses. Meaning something like "Xil_Out32(BASEADDR, value);" is equivalent to "*((u32*)BASEADDR) = value;". When the device tries to read from or write to a memory address assigned to a peripheral, the processor translates it into an AXI transaction, which is routed through the PS and PL interconnects to the peripheral's AXI controller. It's worth noting that the bulk transfer using the DMA is handled by the DMA controller: you configure what the transfer will be by setting values in its config registers, start it, and wait for the DMA controller to finish performing AXI transactions to write the data it's receiving to other addresses (in DDR). The demo should be using the XAxiDma_SimpleTransfer function to do most of the heavy lifting, though the simple transfer function is not that complicated under the hood - if I recall correctly, it's effectively three or four register reads & writes.

https://github.com/Digilent/Zybo-Z7-SW/blob/27cf9ab8a5dc18ca0bf9e4e551551a6c509055af/src/Zybo-Z7-10-DMA/src/audio/audio.c#L374

The meat is in the audio.c source file. fnAudioRecord sets up a DMA transfer using XAxiDma_SimpleTransfer, then sets several control registers in the audio controller - first a sample count, so that the controller knows when to assert the "tlast" signal on the AXI stream, indicating to the DMA that the transfer has finished. Second, clearing and then setting the bit index 1 of the transfer control register, which is tied to a receive enable signal. Third, enabling the stream to the DMA.

VHDL sources for the AXI controller and stream controller can be found in the IP's sources: https://github.com/Digilent/Zybo-Z7-HW/tree/3d1cfeb56b83dc7f9314aa4e2f0456d9bfc4bf68/repo/local/ip/d_axi_i2s_audio_v2_0/src, particularly d_axi_i2s_audio_v2_0_AXI_L.vhd and i2s_stream.vhd. You can access IP sources in the project by right-clicking on the IP in the block design and selecting "Edit in IP packager".

They're a lot to read through, and there are easier ways of implementing AXI communication - I'd point to CSR generator tools capable of creating AXI controllers. Cheby seems interesting, since it can create AXI4-lite interfaces in VHDL source files that can be added directly to a block design through "Add module": https://gitlab.cern.ch/be-cem-edl/common/cheby/-/tree/master/doc?ref_type=heads. Digilent doesn't have a specific tutorial for this kind of thing right now.

Apologies, this was a bit of an info-dump.

Thanks,

Arthur

[evers4]

Hi Arthur,

I've had some success with the AXI Streaming protocol. I was able to program the Zynq to read data from a custom IP in the fabric with an AXI Stream Master Port. I generated this IP using the IP Package Manger Tool, selecting a single AXI Stream Master Port. As for the Vitis Project code, I used a modified version of the Simple DMA with Interrupts example provided by Xilinx (xaxidma_example_simple_intr.c).

I 'm able to set up a DMA transfer using (XAxiDma_SimpleTransfer), and wait on a receive interrupt from the DMA. The firmware/HDL is configured to output/stream 8 32-bit words per read transaction, with incrementing values starting from 1.

Each time I request a DMA transfer using (XAxiDma_SimpleTransfer) with an 32-Byte buffer I receive/see the incrementing values. With each request I see the values continuing to increment from the previous transfer.

However, I'm seeing a problem with the first and only the first transfer. The first 4 values are incrementing, but starting at 5, the remaining 4 values appear to be garbage. After this all values appear as expected for proceeding DMA transfer requests. Do you know what might be causing this?

Note: I had better luck using AXI Lite. I was able to build an AXI Lite Slave Device using IP Package Manager and read values using the Xil_In32 Macro. However, AXI Lite may not be the best solution since it appears you can only read from single word registers and are limited to 512 of them.

Attached Files:

helloworld.c This is the modified Xilinx example (xaxidma_example_simple_intr.c)

AxisDeviceV1_v1_0_M00_AXIS.v This is the modified IP Package Manager auto-generated HDL that implements the AXIS Slave Port.

 

helloworld.c AxisDeviceV1_v1_0_M00_AXIS.v

[artvvb]

23 minutes ago, evers4 said:

 

However, I'm seeing a problem with the first and only the first transfer. The first 4 values are incrementing, but starting at 5, the remaining 4 values appear to be garbage. After this all values appear as expected for proceeding DMA transfer requests. Do you know what might be causing this?

We've seen before that the DMA's ready signal comes up pretty quickly after the IP comes out of reset rather than waiting until a transfer has been initiated, which would allow extra samples into the DMA's internal FIFO if the upstream IP is set to assert tvalid right out of reset. If this is happening, I'd guess that a software reset of the DMA is clearing the FIFOs and discarding the first couple of values that the IP passes in. When the IP then sends the first block of data to the DMA after the PS starts the DMA transfer, tlast is sent early, terminating the DMA transfer a couple of words earlier than expected, leaving the last few pieces of data in the DDR buffer unchanged.

A way to fix this would be to add an enable bit to the upstream AXI4-stream IP, controlled by an AXI4-lite module so that the software can tell the stream IP when to start. The IP could then be started after the first DMA transfer has already been initiated - after the first SimpleTransfer call. It looks like for your IP, it would require modifying the state machine in the verilog source to wait to exit IDLE state until a new enable input port is high.

I might be misremembering some details, but that's my best guess.

30 minutes ago, evers4 said:

Note: I had better luck using AXI Lite. I was able to build an AXI Lite Slave Device using IP Package Manager and read values using the Xil_In32 Macro. However, AXI Lite may not be the best solution since it appears you can only read from single word registers and are limited to 512 of them.

Yeah, AXI lite is good for control and status registers that you hit infrequently or poll, but can't do fast burst transfers. It's great to hear you're having success with this, it's an area with a steep learning curve. You could also add logic so that whenever a single address is read, the data in the register associated with that address is updated, potentially from a FIFO. This might consist of, for example, driving a FIFO read enable pin with "tvalid & tready" from the AXI4-lite read data channel.

Thanks,

Arthur

[artvvb]

On 11/29/2023 at 3:53 PM, evers4 said:

Note: I was able to successfully run both the Echo & Getting out of the Echo Chamber examples. Though after adjusting the second example to transfer more than 1446 bytes the script reports errors in the data received. Its odd in that it occurs when the number of requested bytes is 1447 or greater. Do you see this as well?

I reproduced this and I think fixed it. Below 1446 bytes, the system only actually sends a single burst, indicated by the sent_callback only being entered once per client request. Above it, multiple callbacks would occur, but some data would be lost. What seems to be happening is that submitting more data to tcp_write than fits in a single segment (typically 1500 bytes, slightly lower for a reason I'm not sure of here) maybe causes the additional data to be lost or ignored. What worked for me to fix it was to replace the tcp_sndbuf(tpcb) calls in the sent and recv callbacks with tcp_mss(tpcb), so that no more data than fits into a single segment is included in a single tcp_write call. "MSS" in the function name stands for max segment size. I've revised the code in the blog post.

Thanks!

Arthur

[evers4]

That's great,

thanks for resolving the issue with the example. Regarding AXI, I've continued to experiment with the AXI Lite Mater/Slave protocols & had success. I generated a custom IP with one master & one slave again using the Vivado IP Packing tool. I've modified it to write to addresses 0-99 of the Zybo's DDR using the AXI Master Port (I simply wrote the address number to the address, so values 0-99). I was then able to read these values back in a loop using the function Xil_in32.

        for(Index = 0; Index < NUMBER_OF_WORDS; Index ++)
        {
            xil_printf("Data Received %d\r\n", Xil_In32(0x00000000 + (Index*4)));
        }

After this,

I added a custom output port to this IP to send an interrupt to the Zynq each time it finishes writing to the DDR. So that the Software knows that new data is available to be buffered & then sent over ethernet. However, this is where I'm currently stuck. While I was able to set up an ISR to intercept the data, it was firing repeatedly. I've since modified the IP to send an interrupt ever five seconds using a counter. This helped, but now I see the interrupt firing 2 to 3 times every five seconds. Do you see what could be the problem here?

I've attached the debug version of the Vitis project code, which was implemented using the HelloWorld example (only modifying the hello.c file with the code shown). I based the interrupt configuration off of Xilinx's example code (xaxidma_example_simple_intr.c) found at the location below. I've also attached the original C code & firmware for the Custom IP as well.

C:\Vitis\2023.1\data\embeddedsw\XilinxProcessorIPLib\drivers\axidma_v9_16\examples

 

 

#include <stdio.h>
#include "platform.h"
#include "xil_printf.h"

#include "xparameters.h"
#include "xil_io.h"
#include "sleep.h"

///////////////////////////////
// Interrupt Declaration
///////////////////////////////
#include "xscugic.h"

#define CUSTOM_INTR_ID    61U
#define INTC_DEVICE_ID    XPAR_SCUGIC_SINGLE_DEVICE_ID

static XScuGic Intc;    /* Instance of the Interrupt Controller */
#define INTC_HANDLER    XScuGic_InterruptHandler

static int SetupIntrSystem(XScuGic * IntcInstancePtr, u16 CustomIntrId);
static void DisableIntrSystem(XScuGic * IntcInstancePtr, u16 CustomIntrId);

int main()
{
    int Status;
    xil_printf("\r\n--- Entering main() --- \r\n");

    /* Set up Interrupt system  */
    Status = SetupIntrSystem(&Intc, CUSTOM_INTR_ID);
    if (Status != XST_SUCCESS)
    {
        xil_printf("Failed intr setup\r\n");
        return XST_FAILURE;
    }
    while(1){}

    DisableIntrSystem(&Intc, CUSTOM_INTR_ID);
    xil_printf("--- Exiting main() --- \r\n");
    return 0;
}

////////////////////////////////////////////////////////////////////////////////////
// This is the Custom AXI Device Interrupt handler function.
////////////////////////////////////////////////////////////////////////////////////
static void CustomIntrHandler(void *Callback)
{
    xil_printf("--- Entering CustomIntrHandler() --- \r\n");
}

////////////////////////////////////////////////////////////////////////////////////
// This function setups the interrupt system so interrupts can occur for the
// Custom AXI IP, it assumes XScuGic component exists in the hardware system.
////////////////////////////////////////////////////////////////////////////////////
static int SetupIntrSystem(XScuGic * IntcInstancePtr, u16 CustomIntrId)
{
    int Status;
    XScuGic_Config *IntcConfig;

    /*
     * Initialize the interrupt controller driver so that it is ready to
     * use.
     */
    IntcConfig = XScuGic_LookupConfig(INTC_DEVICE_ID);
    if (NULL == IntcConfig)
    {
        xil_printf("Failed in IntcConfig\r\n");
        return XST_FAILURE;
    }
    Status = XScuGic_CfgInitialize(IntcInstancePtr, IntcConfig, IntcConfig->CpuBaseAddress);
    if (Status != XST_SUCCESS)
    {
        xil_printf("Failed in XScuGic_CfgInitialize\r\n");
        return XST_FAILURE;
    }
    //XScuGic_SetPriorityTriggerType(IntcInstancePtr, CustomIntrId, 0xA0, 0x3);
    XScuGic_SetPriorityTriggerType(IntcInstancePtr, CustomIntrId, 0xA0, 0x3);

    /*
     * Connect the device driver handler that will be called when an
     * interrupt for the device occurs, the handler defined above performs
     * the specific interrupt processing for the device.
     */
    void* argument;
    Status = XScuGic_Connect(IntcInstancePtr, CustomIntrId, (Xil_InterruptHandler)CustomIntrHandler, argument);
    if (Status != XST_SUCCESS)
    {
        xil_printf("Failed in XScuGic_Connect\r\n");
        return Status;
    }
    XScuGic_Enable(IntcInstancePtr, CustomIntrId);

    /* Enable interrupts from the hardware */
    Xil_ExceptionInit();
    Xil_ExceptionRegisterHandler(XIL_EXCEPTION_ID_INT, (Xil_ExceptionHandler)INTC_HANDLER, (void *)IntcInstancePtr);
    Xil_ExceptionEnable();

    return XST_SUCCESS;
}

////////////////////////////////////////////////////////////////////////////////////
// This function disables the interrupts for DMA engine.
////////////////////////////////////////////////////////////////////////////////////
static void DisableIntrSystem(XScuGic * IntcInstancePtr, u16 CustomIntrId)
{
    XScuGic_Disconnect(IntcInstancePtr, CustomIntrId);
}

AxiDeviceMasterSlave_v1_0_M00_AXI.v

helloworld.c

Edited December 13, 2023 by evers4

[artvvb]

XScuGic_SetPriorityTriggerType's last parameter sets the trigger type - active-high level sensitive. Given this, I would expect that the interrupt handler would access the peripheral that triggered the interrupt to clear the interrupt bit, which would remain high until this access happens. I don't see the custom_int port in the verilog source, so can't confirm when it is cleared, but the behavior sounds like the interrupt goes high and stays high.

[evers4]

My mistake, I didn't include the top level HDL. Here is that file.

That's helpful information though, I thought that the value 0x3 configured the interrupt to trigger on a rising edge. I don't fully follow the documentation written for this function (attached below). What do the acronyms SFI, PPI, SPI stand for? This indictates that 1 is active high & 3 is rising edge. It doesn't specify what values 0 & 2 correspond to. It appears that each bit pair of the 8bit uTrigger corresponds to 1 of 4 interrupts, and the interrupt ID assigned to my custom IP corresponds to one of these entries (I think....).

My goal is to set this to a rising edge trigger and not to disturb the interrupts already used in the Ethernet Example you provided (Since I'm merging the 2 projects). One other thing I'd like to indicate is that I left the priority as it was set in the Xilinx DMA example (since I don't know what would be an appropriate value).

 

/****************************************************************************/
/**
* Sets the interrupt priority and trigger type for the specificd IRQ source.
*
* @param    InstancePtr is a pointer to the instance to be worked on.
* @param    Int_Id is the IRQ source number to modify
* @param    Priority is the new priority for the IRQ source. 0 is highest
*           priority, 0xF8(248) is lowest. There are 32 priority levels
*           supported with a step of 8. Hence the supported priorities are
*           0, 8, 16, 32, 40 ..., 248.
* @param    Trigger is the new trigger type for the IRQ source.
* Each bit pair describes the configuration for an INT_ID.
* SFI    Read Only    b10 always
* PPI    Read Only    depending on how the PPIs are configured.
*                    b01    Active HIGH level sensitive
*                    b11 Rising edge sensitive
* SPI                LSB is read only.
*                    b01    Active HIGH level sensitive
*                    b11 Rising edge sensitive/
*
* @return    None.
*
* @note        None.
*
*****************************************************************************/
void XScuGic_SetPriorityTriggerType(XScuGic *InstancePtr, u32 Int_Id,
                    u8 Priority, u8 Trigger)

AxiDeviceMasterSlave_v1_0.v AxiDeviceMasterSlave_v1_0_M00_AXI.v

[artvvb]

Ah, yeah, my bad, looks like it's rising edge sensitive. I'll need to see if I can reproduce.

The acronyms are defined in the Zynq TRM: https://docs.xilinx.com/r/en-US/ug585-zynq-7000-SoC-TRM/Private-Shared-and-Software-Interrupts

[evers4]

The output I see is shown below after waiting 5 seconds following the line:

"--- Entering main() --".

Every five seconds results in another 3 lines of

"--- Entering CustomIntrHandler() ---"

 

 

 

--- Entering main() ---
--- Entering CustomIntrHandler() ---
--- Entering CustomIntrHandler() ---
--- Entering CustomIntrHandler() ---

[artvvb]

Since the interrupt output is connected to a port, are you checking it with a logic analyzer? You could also wire up various signals to an ILA and view them via the hardware manager. To add an ILA to the design, you'd right click on any nets you want to test, select debug (screenshot below), then run connection automation to connect them all to ILAs. Then, when the bitstream is loaded onto the board (maybe while debugging in Vitis), you can open the Vivado hardware manager, and once the board is connected, some internal logic analyzers should pop up. This would let you take a look at the waveforms for the interrupt signal and any AXI traffic into or out of your module (I think by default the window size is 1000 clocks, centered on a trigger). Apologies if this is vague, we don't currently have any step-by-step content on the topic.

[artvvb]

55 minutes ago, evers4 said:

What do the acronyms SFI, PPI, SPI stand for?

IRQ_F2P interrupts are all "SPI", shared peripheral interrupts, from p45 of the Zynq TRM:

[evers4]

Great information,

also the link you provided is very helpful. Initially, I had though both the Fabric and Zynq were operating synchronously at 100MHz. I see that this is not the case, the Zynq is actually clocked at 666MHz, while the custom IP in the fabric (clocked off of the Zynq's FCLK) is running at 100MHz. To complicate matters the documentation on the GIC states that it operates at half the CPU/Zynq frequency.

I'm not sure being asynchronous is an issue. I've configured a one cycle trigger here, perhaps that could be an issue? I'm about to test replacing this pulse a 2Hz clock to see if that could have an effect.

As for XScuGic_SetPriorityTriggerType, based upon this documentation it looks like I'm working with an SPI & the trigger type should be set to 3 for a rising edge trigger.

 

[evers4]

I connected up a slow 200MHz scope to the custom_int signal in the block design. What I see is distorted pulse that does otherwise appear correct.

After this, I replaced the single cycle pulse by a slow 4Hz clock, the ISR appears to be working as expected now. I expected some distortion on the probe, but I don't think the interface between the Zynq/IP should be effected by this. I'm cautiously optimistic that the problem has been resolved.

I'm now attempting to merge this code with the Ethernet Example you provided. I'll follow up with you after making some progress. Thanks!

[evers4]

Hi Artvvb,
I’m now debugging the data transferred from this IP to the DDR memory of the Zybo board. The purpose of custom_int is to tell the Zynq new data is available. Prior to each trigger the IP writes 100 32bit words to DDR addresses 0-99 (This data is a count value 0 to 4 billion). When I read this data in my C code from the Zynq the first write of 100 words works. However, I’m not seeing this data updated in following writes.

The IP I’m using is a modified version of that generated by Xilinx’s IP Packaging Tool, where I specified a generic IP with a single master & single slave AXI-Lite port.

I’ve attached the updated firmware for the IP, can you spot any issues here? Is there anything else I can provide you with to help in debugging this issue? I’ve excluded the Slave AXI-Lite HDL, since its currently unused.

AxiDeviceMasterSlave_v1_0.v AxiDeviceMasterSlave_v1_0_M00_AXI.v

[artvvb]

Without having read the verilog sources yet, it sounds like you might not be invalidating the cache across the range of addresses that you're transferring data into before accessing them.

[evers4]

Ah okay, is there anything I can provide you with to confirm? Does this fit this scenario where I'm able to perform a single write from the IP and read from the Zynq? I'm able to reread from this section of DDR, but the values don't continue to be updated by the IP.

I checked the data written from the IP to DDR using the ILA as you had suggested earlier. I can confirm the write values out of the IP are correct.

 

Edited December 14, 2023 by evers4

[artvvb]

Quote

Ah okay, is there anything I can provide you with to confirm? Does this fit this scenario where I'm able to perform a single write from the IP and read from the Zynq? I'm able to reread from this section of DDR, but the values don't continue to be updated by the IP.

C sources should include xil_cache.h and call Xil_DCacheInvalidateRange before accessing the memory that the IP has written into. I think what's happening is that the data written on the first pass is being cached and the cached values are being read instead of new stuff in DDR on every subsequent pass. https://github.com/Digilent/Zybo-Z7-SW/blob/c21218c91e7d6dfd2018d35932a5f0d9d38eec9a/src/Zybo-Z7-20-DMA/src/demo.c#L262 is where the audio demo invalidates cache for received data. Your base address and byte count for the call will differ. You might call this in the handler or wherever in your C source you're reading that data.

[evers4]

Ow wow! That did it, thanks for the easy fix! Side question, is it possible to read from DDR while within an ISR? Does it require any special consideration(s)?

Edited December 14, 2023 by evers4

[artvvb]

19 minutes ago, evers4 said:

Side question, is it possible to read from DDR while within an ISR? Does it require any special consideration(s)?

This thread is pushing the bounds of my knowledge a bit, so grain of salt, but yeah, it's possible. DDR is "just memory addresses", you can access peripherals via interconnect while in an ISR as well. I would just be aware that you're potentially either blocking other interrupts from starting for as long as you stay in the ISR (by disabling interrupts), or you're potentially getting interrupted by other things coming through. This isn't super relevant until you're pushing the processor hard. Like if you have an interrupt that requires that the processor intervenes within a short time, while the processor happens to be busy copying a big array of data from one location to another with interrupts disabled.