Genesys ZU DDR on PL and PS side

[MaxPof]

Hi everyone,

I'm sorry, this post is basically the same that the one I've posted on Xilinx's forum.

I was waiting to be approved on this forum for days and I really needed to move forward... But this question is way better suited here.

 

So I would like to be able to access the DDR on both the PL and PS side.

I already did it for the zybo but I have a lot of trouble with the Genesys ZU 3EG.

 

Here’s where I stand:

-I've added the Genesys boards files in Vivado

-I've created the BD, added the Ultrascale Mpsoc bloc and enabled DDR

Q: How do I know that the default configuration for DDR is OK? Is it already configured in the Genesys boards files? Or should I change something.

-I've added the DDR4 MIG

Q: I have no clue how to configure that...

-Run Connection Automation

Q: Connection automation create a diff clock at 100MHz on the BD for the C0_SYS_CLK port of the MIG, which is not coherent with the default 70.1361MHz needed. Should I adapt the 100MHz?

Q: Should I define this diff clock that the MIG needs in my constraint file?

-Run synthesis: OK

-Run Implementation: return this error:

    [Mig 66-99] Memory Core Error - [memory/ddr/u_ddr4_0] Either port(s) c0_sys_clk_p, c0_sys_clk_n is/are not placed or un-supported clocking structure/circuit for memory ip instance. Please refer to clocking section of PG150 for supported clocking structures. 

    [Mig 66-99] Memory Core Error - [memory/ddr/u_ddr4_0] Port(s) c0_ddr4_ck_c[0],c0_ddr4_ck_t[0],c0_ddr4_adr[0],c0_ddr4_adr[1],c0_ddr4_adr[2],c0_ddr4_adr[3],c0_ddr4_adr[4],c0_ddr4_adr[5],c0_ddr4_adr[6],c0_ddr4_adr[7],c0_ddr4_adr[8],c0_ddr4_adr[9],c0_ddr4_adr[10],c0_ddr4_adr[11],c0_ddr4_adr[12],c0_ddr4_adr[13],c0_ddr4_adr[14],c0_ddr4_adr[15],c0_ddr4_adr[16],c0_ddr4_ba[0],c0_ddr4_ba[1],c0_ddr4_bg[0],c0_sys_clk_n,c0_sys_clk_p,c0_ddr4_bg[1],c0_ddr4_cs_n[0],c0_ddr4_cke[0],c0_ddr4_odt[0],c0_ddr4_act_n,c0_ddr4_reset_n,c0_ddr4_dqs_c[0],c0_ddr4_dqs_t[0],c0_ddr4_dm_dbi_n[0],c0_ddr4_dq[0],c0_ddr4_dq[1],c0_ddr4_dq[2],c0_ddr4_dq[3],c0_ddr4_dq[4],c0_ddr4_dq[5],c0_ddr4_dq[6],c0_ddr4_dq[7],c0_ddr4_dqs_c[2],c0_ddr4_dqs_t[2],c0_ddr4_dm_dbi_n[2],c0_ddr4_dq[16],c0_ddr4_dq[17],c0_ddr4_dq[18],c0_ddr4_dq[19],c0_ddr4_dq[20],c0_ddr4_dq[21],c0_ddr4_dq[22],c0_ddr4_dq[23],c0_ddr4_dqs_c[3],c0_ddr4_dqs_t[3],c0_ddr4_dm_dbi_n[3],c0_ddr4_dq[24],c0_ddr4_dq[25],c0_ddr4_dq[26],c0_ddr4_dq[27],c0_ddr4_dq[28],c0_ddr4_dq[29],c0_ddr4_dq[30],c0_ddr4_dq[31],c0_ddr4_dqs_c[4],c0_ddr4_dqs_t[4],c0_ddr4_dm_dbi_n[4],c0_ddr4_dq[32],c0_ddr4_dq[33],c0_ddr4_dq[34],c0_ddr4_dq[35],c0_ddr4_dq[36],c0_ddr4_dq[37],c0_ddr4_dq[38],c0_ddr4_dq[39],c0_ddr4_dqs_c[5],c0_ddr4_dqs_t[5],c0_ddr4_dm_dbi_n[5],c0_ddr4_dq[40],c0_ddr4_dq[41],c0_ddr4_dq[42],c0_ddr4_dq[43],c0_ddr4_dq[44],c0_ddr4_dq[45],c0_ddr4_dq[46],c0_ddr4_dq[47],c0_ddr4_dqs_c[6],c0_ddr4_dqs_t[6],c0_ddr4_dm_dbi_n[6],c0_ddr4_dq[48],c0_ddr4_dq[49],c0_ddr4_dq[50],c0_ddr4_dq[51],c0_ddr4_dq[52],c0_ddr4_dq[53],c0_ddr4_dq[54],c0_ddr4_dq[55],c0_ddr4_dqs_c[7],c0_ddr4_dqs_t[7],c0_ddr4_dm_dbi_n[7],c0_ddr4_dq[56],c0_ddr4_dq[57],c0_ddr4_dq[58],c0_ddr4_dq[59],c0_ddr4_dq[60],c0_ddr4_dq[61],c0_ddr4_dq[62],c0_ddr4_dq[63],c0_ddr4_dqs_c[1],c0_ddr4_dqs_t[1],c0_ddr4_dm_dbi_n[1],c0_ddr4_dq[8],c0_ddr4_dq[9],c0_ddr4_dq[10],c0_ddr4_dq[11],c0_ddr4_dq[12],c0_ddr4_dq[13],c0_ddr4_dq[14],c0_ddr4_dq[15] is/are not placed. Assign all ports to valid sites. 

The Digilent Reference Manual provide this link:

https://support.xilinx.com/s/article/75768?language=en_US

I tried to do what is said in "Enabling Dynamic DDR Configuration", but without any success.

 

I try to follow this: https://support.xilinx.com/s/question/0D52E00006hpgtFSAQ/memory-core-error?language=en_US

I've checked the DDR IO pins and they are not assigned.

But if I refer to the schematic:

https://digilent.com/reference/_media/reference/programmable-logic/genesys-zu/genesys_zu-3eg_revd1_sch_public.pdf

I can't select the corresponding pins (e.g. W28 for DDR4_ADDR0 is not available).

So I suppose the user don't have to assigne them. But I'm quite lost now...

I saw that there was a MIG file for Genesys 2 that does the pin assignement, but nothing for Genesys ZU

 

Has anyone ever got DDR working on Genesys ZU? (Not Genesy 2!)

 

Thanks

Edited February 23, 2022 by MaxPof

[zygot]

Your board doesn't have any DDR memory connected to PL pins, so the only way to access the PS external memory would be through the PS/PL AXI ports. It is no possible to have the PS external memory controller and a PL based memory controller control the same memory.

You say that you've done this for the ZYBO, but the same answer applies to that board as well. There aren't many ZYNQ boards having two separate DDR banks, one connected to the PS and the other connected to the PL. The ZCU106 is an exception, as both sides of the ZYNQ hardware have their own external memory.

BTW UltraScale doesn't use the Series 7 MiG IP for external memory controllers; this family has it's own external memory controller IP. Timing for DDR4 is very tight, limiting clocking options, unlike DDR3 controller IP.

Edited February 23, 2022 by zygot

[MaxPof]

Hi!

Quote

so the only way to access the PS external memory would be through the PS/PL AXI ports.

Yes, sorry I wasn't clear, this is what I do.

See here the method that I use for Zybo.

So yeah, my problem is not so much the PS/PL part, but just to be able to connect the DDR to my BD.

Quote

BTW UltraScale doesn't use the Series 7 MiG IP for external memory controllers; this family has it's own external memory controller IP. Timing for DDR4 is very tight, limiting clocking options, unlike DDR3 controller IP.

So I don't have to use the MIG on my bloc diagram? I attach a screenshot of my actual (cleaned) BD

 

[zygot]

I don't understand what you are trying to do. To enable the PS connected DDR4 just enable in in the ZYNQ UltraScale+ block. This allows the PS to access the external memory on your board directly. You can use the PS/PL AXI ports to allow a design in the PL to DMA data into or out of the PS memory if desired.

What is the DDR4 SDRAM block supposed to be connected to? It can't be the PS connected DDR4 memory because that's connected to the PS MIO pins. Even if you could mux the PS external memory controller pins through the EMIO to a PL design, why would you want to? The PS hard external memory controller is going to be higher performance and use no PL resources.

Your block diagram doesn't appear to be the toplevel module in your project so it's hard to see what you are trying to do without the source and associated constraints.

UltraScale(+) devices have their own version of the Series 7 and earlier MiG soft external memory controller IP for logic connected external memory. This IP provides different options for IPI instantiation relative to native HDL instantiation. If you have external memory physically connected to PL pins then you can instantiate a soft logic controller to use it and Xilinx will force you to use the AXI resources. I don't see a point in this arrangement so for the ZCU106, which does have PL connected DDR4 memory I create a basic IPI with nothing but a preset ZYNQ UltraScale+ block and use a wrapper to connect the block diagram generated sources to my toplevel source. For DDR4 soft IP controllers you can't assign arbitrary external clock pins to the controller due to the tight timing.

More importantly, UltraScale ZYNQ memory use is more complicated than the Series 7 ZYNQ so there might be good reasons to isolate any PL connected external memory from direct PS access. Do read the ZYNQ UltraScale+ TRM.

NOTE: You seem to be using the HLS tool which I don't have experience with ( you should make that clear from the beginning ). Even so, the device architecture and resources are the same regardless of the tools use to create a bitstream, so I think that my comments are relevant. A problem with the HLS tool is that it's suppose to hide the messy details of FPGA design, which causes it's own problems and confusion when things don't work out.

 

Edited February 24, 2022 by zygot

[MaxPof]

Quote

You can use the PS/PL AXI ports to allow a design in the PL to DMA data into or out of the PS memory if desired.

Through the HP0 port, right?

Alright perfect, that what I want to do!

Quote

What is the DDR4 SDRAM block supposed to be connected to? It can't be the PS connected DDR4 memory because that's connected to the PS MIO pins. Even if you could mux the PS external memory controller pins through the EMIO to a PL design, why would you want to?

Yeah I don't know, it's just very unclear how to access DDR with PS (let's leave the PL aside for the moment)

With Zynq, you have a DDR port which appears when  you enable DDR. That's not the case with Ultrascale. So I thought I should add the DDR block...

Quote

The PS hard external memory controller is going to be higher performance and use no PL resources.

Ok ok, so, same as Zynq, all is PS handled, so no need for the DDR bloc.

 

Quote

NOTE: You seem to be using the HLS tool which I don't have experience with ( you should make that clear from the beginning ).

Yes, sorry. My question was more about how to interface the DDR because I thought is was way more complicated. But I'm not very clear.

What I do on my zybo is:

Access DDR from PS ( from my application on the ARM) and access from HLS IP through AXI/HP0 interface.

 

So I think you resolve my problem if you say that I just have to enable the DDR in the Ultrascale config to make it works the same way.

I was just going on something much more complicated since the DDR port does not appear on the BD...

 

Thanks for your answer!

[Richm]

use axi DMA to transfer data to/from pl into ps ddr memory. 

Unless your board has separate ddr for both the pl and ps, you will need to use the ps ddr. 

I remember reading somewhere that the ps ddr connection is omitted in vivado for zynq US+ because it's not necessary for the block diagram

[zygot]

There are a lot of AXI options for connecting a design implemented in the PL to your ARM processors, depending on whether the ARM controls the bus or the PL design controls the bus.

Be aware that the UltraScale ZYNQ is a much more complicated beast than the Series 7 ZYNQ devices. Before moving ahead you need to read through the ZYNQ UltraScale TRM to gain an understanding of the differences. Unfortunately, the Xilinx tools are moving in the direction of abstracting user control of the logic resources and IP. When things go as planned this might be nice, but when things go badly its a big problem for the developer. Having the tools work out the details ( and hiding it behind encrypted source ) is like a two edged sword... or perhaps I should say like being a passenger in the back seat of a self-driving car; when the technology is working well you might have a nice experience. When the technology fails, you might want to be somewhere else in the moment.

I'm making the assumption that the HLS tools are more attractive to people wanting to add custom functionality to the PS without significant external interface logic design. For those wanting to implement high performance interfaces to external hardware the regular tools might be more appropriate. The UltraScale IO has changed significantly from the Series 7 devices and entails a lot of new skills except for the most trivial single-ended signals. Migrating from Series 7 to UltraScale using advanced IO resources is not a trivial endeavor. Even a simple RGMII implemented in HDL is challenging, based on my personal experience.

Edited February 25, 2022 by zygot