CMOD A7 15T: unable to start from Flash

[Guest]

Hi,

I'm trying to start my CMOD A7 from a design stored in the Micron Flash.

The design is working, I can start it by programming the FPGA directly without any issue. The DONE LED lights up, and the design does what it has to do.

Programming the flash (as an n45q32-3.3v-spi-x1_x2_x4 device) works without issue. I can also read back the design from flash, and it is identical to the BIN file used for programming, so the Flash seems ok.

However, when I power cycle the module after that, the design does not start from the flash. The DONE LED remains unlit, and the design does not run.

Any ideas would be appreciated.

 

[zygot]

Is the FLASH device on your CMOD-A7 the device that you think it is? Due to availability, there are a lot of variants out there and Digilent isn't very good at posting information about production run versions. Sadly, not all FLASH devices that seem to be "drop-in replacement" turn out to be so. 

A read through of the Series 7 Configuration Users' Manual might help focus your debug efforts.

Edited September 7, 2023 by zygot

[Kvass]

Oh man, I've ran into this issue many times over, and with multiple versions of vivado. One thing I found at some point is that if you have a microblaze program in your design, it likes to be the release version, not the debug one. Which version of Vivado/Vitis are you using?

 

https://forum.digilent.com/topic/23536-storing-program-in-flash-on-cmod-a7-in-2022/

https://forum.digilent.com/topic/26483-microblaze-program-not-starting-on-powerup-cmod-a7/

https://forum.digilent.com/topic/25802-cmod-a7-35t-spi-memory-programming

 

[Guest]

Hi @Kvass

My design has no Microblaze. It's just a very simple purely HDL-based design that periodically toggles some pins.

I'm using Vivado 2023.1.1.

[Guest]

Ok the same issue occurs with an CMOD A7-35T, also with a Micron flash chip. I'll try later with a 35T that has the other (Macronix) chip.

[Guest]

A CMOD A7-35T with a Macronix Flash chip (mx25l3273f) shows the same behavior -- the design simply won't start from flash.

@JColvin -- any idea what is going on here? Could you confirm on your side that flash programming a CMOD-A7 still works in Vivado 2023.1.1? I'm really at a loss here.

All these CMOD modules I tried have been flash-programmed without any issue in the past.

I will try some other parts FPGA development boards I have here, to figure out if there's a pattern.

Edited September 9, 2023 by reddish

[Guest]

Right, I tried it on a Nexys Video board with a 256 Mbit "Spansion" Flash memory chip.

 

That, at least, works as expected (the design can be flashed to the Spansion part and the FPGA will boot from it).

[Guest]

Likewise, I can program a Basys3 with a Spansion Flash chip just fine.

The preliminary conclusion is that the problem appears to be specific to the CMOD-A7 modules, and independent of their Flash chip.

[Guest]

I went back to the CMOD A7 15T that I first saw this issue with, and I am seeing some more interesting behavior.

First, I noticed, when I selected the FOGA in the Hardware Manager after programming, then right click then a "Boot from Configutation Memory Device", the device boots successfully from flash.

Then, I took the device off the USB of the computer, then attached it to a USB that is power-only (a powerbank). Then, also, the device boots from Flash.

Then I went back to my development PC, and inserted the CMOD-15T into USB, with Vivado not running. The device boots from Flash.

Restarted Vivado, without hardware manager not open. The device boots from Flash on USB-insertion.

Opened the Hardware Manager inside Vivado. The device still boots from Flash on USB-insertion.

Auto-connected to the device inside The HW manager. Unplugged the device, then re-inserted the device. The device will no longer boot from Flash.

Closed the target; pulled the device from USB then reinserted. The device boots from Flash.

In summary, what I'm seeing is that when a device is open in the Hardware Manager, and then pulled/reinserted from USB, booting will be inhibited. This behavior is different from what it used to be.

I now suspect this is due to a recent change in Vivado's Hardware Manager, but I'm not sure.

I don't see this behavior with the BASYS3 board.

It's quite puzzling. @JColvin, do you have any idea?
 

[zygot]

3 hours ago, reddish said:

I now suspect this is due to a recent change in Vivado's Hardware Manager, but I'm not sure.

You don't mention what version of Vivado you are using.

The implementation of the CMOD UART/JTAG interface is not the same as for the other boards. Moreover, the CMOD is unique in terms of size and PCB stack-up.

I do believe that the behavior of Vivado hardware manager ( with respect to JTAG activity ) has changed in recent years. I've seen evidence of this using even more robust designs like the Nexys Video. I use Vivado 2019.1 through 2023 on both Windows and Linux hosts consistently using a variety of FPGA boards from vairous vendors. It shouldn't be too hard for you to investigate you hypothesis and get a better answer than you are likely to obtain here.

I never use any version of Vivado Hardware Manager with any of my ( original version ) CMOD-A735T boards.

Edited September 9, 2023 by zygot

[JColvin]

Hi @reddish,

I don't have any immediate ideas as to what might be the situation. I've posted on a number of different threads over the years about how I've fought with Vivado / Vitis to get an SDK / processor based project to run off of flash (Kvass linked to some of tje,), but when I plugged in my two Cmod A7's into my host computer this morning (no instance of Vivado open), Tera Term is happily reporting that the SDK flash programs are being loaded.

Regrettably, I didn't have the foresight at the time to have the name of the Vitis project be printed out over terminal, so I don't know off-hand what project I used, though it looks like both the -35T and the -15T boards I have both have Micron memory rather than Macronix (-35T has some custom flash project, -15T is running the out of box project). I think one of the Cmod S7's I have might have the Macronix memory (and I think @artvvb got a Cmod A7 with a Macronix memory somewhat recently, though he is out of the office today), so I will do some testing to see what I can come up with with regards to straight HDL flash projects.

Though fair warning in advance that if this appears to be some sort of change due to in various Vivado versions, I'm going to be rapidly out of ideas as to what the resolution will be (aside from not having the device be snagged by the Vivado Hardware Manager) as Digilent (or at least certainly not me) does not have any special access Xilinx representatives.

Thanks,
JColvin

[Guest]

@JColvin

Thanks. It does seem that the behavior is different between the CMOD modules on the one hand and the BASYS / NEXYS VIDEO systems on the other hand. Seems like Vivado is inhibiting a flash-boot in some way on the CMODs.

Anyway, the important thing is that the boot-from-flash is actually working. The fact that Vivado behaves differently from the past is not that big of a deal now that I understand that it can be fixed by closing the Hardware Manager, or explicitly booting from flash, on startup.

[D@n]

@reddish,

Out of curiosity ... the CMod A7 has two QSPI SCK pins.  I've enjoyed this feature in the past, because it allows me to run the flash at 2x the speed because one pin as a DDR capability, and the other is used for configuration only.  My question therefore is, are you using both pins in your design at all?  If not, are you using the STARTUPE2 primitive in your design?  If so, are you overriding the DONE pin at all?

Not sure if any of these might help, but they are at least worth looking into.

Dan

[JColvin]

I think I found the problem and solution, but I'll walk through what I did. BLUF, you'll probably need to add the following line to your .xdc:

Quote

set_property CONFIG_MODE SPIx4 [current_design]

I also added the following two lines, but I don't believe these are strictly necessary:

Quote

set_property BITSTREAM.GENERAL.COMPRESS TRUE [current_design]
set_property BITSTREAM.CONFIG.CONFIGRATE 33 [current_design]

Vivado added these three lines for me by opening up the Implemented Design, then going to Project Manager Settings -> Bitstream -> Additional Bitstream configurations, making the changes in the various tabs, then saving the Implemented Design to update the .xdc, but I plan on simply typing them into the .xdc in the future.

Long version of how I got there is:

I set up 4 different boards to test out with various Vivado installs:

Basys 3 - Spansion flash, HDL, no UART (I'm treating this one as a pseudo-control variable)
Cmod A7 15 - Micron flash, processor, UART
Cmod S7 25 - Macronix flash, HDL, UART
Arty A7 100 - Spansion, HDL, UART

After I confirmed that their flash projects worked as is (i.e. operated as expected after disconnecting and then reconnecting power with no instance of Vivado or Adept or hwserver.exe opened), I opened up different instances of Vivado one at a time and individually tested to see if each board would start their flash program either with or without the Hardware Manager having specifically selected that board as a target.

My table of results that I got off of my machine (Windows 10, fwiw) is below:

Clearly, with the designs in their current state, the Cmod A7 and Arty A7 needed to be reset before they started working, but that evidently doesn't apply to the Basys 3 or the Cmod S7. The Basys 3, Cmod A7, and Cmod S7 were all running their original out-of-box designs and the Arty A7 was running a rebuilt version of its out-of-box demo.

Since the Cmod S7 and and Cmod A7 are very similar I decided I'd port the existing Cmod S7 out of box demo (https://digilent.com/reference/programmable-logic/cmod-s7/demos/oob) to the Cmod A7, upgrading the design to 2023.1 in the process.

As a sanity check before porting, I had Vivado upgrade the project to the newer version of the tools (Vivado had to do some directory restructure), ticked the box for a .bin to be created, added the Macronix memory configuration part to the listed Spartan 7 in the Hardware Manager, and suddenly found that the configuration stored in flash did not load unless it was not being targeted by Vivado. That was enough for me to remember the additional bitstream flash settings I mentioned at the beginning of this post, so I regenerated the bitstream with these new settings which did have the part successfully load.
(Or I guess, to be clear, when I initially programmed the memory configuration device, it did not boot from it that very first time. I had to select boot from memory configuration device, or disconnect the device entirely from power first before it then started to consistently boot from memory on POR)

With that success, I ported the design to the Cmod A7 15T (attached), made sure that the various bitstream / property settings were in place, and found that now the Cmod A7 also booted successfully, even when it was being targeted by the Vivado Hardware Manger.

So, basically any version of Vivado where the Xilinx Hardware Manager now automatically reconnects to the device (first occurring sometime in the 2016.X era), you'll need that extra CONFIG_MODE property set for the device to have SPI flash always be a configuration mode option. As for why auto-reconnecting to the device prevents booting up from flash when it can successfully boot in all other scenarios, I have no idea.

Thanks,
JColvin

CmodA715-S7-OOB-Port-23.1.xpr.zip

[Guest]

10 hours ago, D@n said:

@reddish,

My question therefore is, are you using both pins in your design at all?  If not, are you using the STARTUPE2 primitive in your design?  If so, are you overriding the DONE pin at all?

Hi @D@n

My design is not touching those pins. I'm testing with little more than a blinking LED.

Cheers, Sidney

 

[Guest]

Hi @JColvin,

Thanks for the investigative work, much appreciated; and good to have a workaround (even if we don't understand the specifics on how auto-reconnecting by the Hardware Manager interferes with the boot process).

On the topic of configuration-related properties: there are at least five of them (CONFIG_MODE, CONFIG_VOLTAGE, CFGBVS, BITSTREAM.CONFIG.CONFIGRATE, BITSTREAM.GENERAL.COMPRESS), and the XDC files provided by Digilent in the digilent-xdc repository on GitHub are not consistent in whether they provide values for them. Here's an overview (I attached the script to generate this below).
 

Arty-A7-100-Master.xdc         CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Arty-A7-35-Master.xdc          CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Arty-Master.xdc                CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Arty-Z7-10-Master.xdc          CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Arty-Z7-20-Master.xdc          CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Cmod-A7-Master.xdc             CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Cora-Z7-07S-Master.xdc         CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Cora-Z7-10-Master.xdc          CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Eclypse-Z7-Master.xdc          CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Genesys-2-Master.xdc           CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Nexys-4-DDR-Master.xdc         CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Nexys-4-Master.xdc             CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Nexys-A7-100T-Master.xdc       CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Nexys-A7-50T-Master.xdc        CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Sword-Master.xdc               CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Zedboard-Master.xdc            CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Zybo-Master.xdc                CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Zybo-Z7-Master.xdc             CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Genesys-ZU-3EG-D-Master.xdc    CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS Y
Genesys-ZU-3EG-Master.xdc      CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS Y
Genesys-ZU-5EV-D-Master.xdc    CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS Y
USB104-A7-100T-Master.xdc      CONFIG_MODE . CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS Y
Nexys-Video-Master.xdc         CONFIG_MODE . CONFIG_VOLTAGE Y CFGBVS Y BITSTREAM.CONFIG.CONFIGRATE . BITSTREAM.GENERAL.COMPRESS .
Cmod-S7-25-Master.xdc          CONFIG_MODE Y CONFIG_VOLTAGE . CFGBVS . BITSTREAM.CONFIG.CONFIGRATE Y BITSTREAM.GENERAL.COMPRESS Y
Arty-S7-25-Master.xdc          CONFIG_MODE Y CONFIG_VOLTAGE Y CFGBVS Y BITSTREAM.CONFIG.CONFIGRATE Y BITSTREAM.GENERAL.COMPRESS .
Arty-S7-50-Master.xdc          CONFIG_MODE Y CONFIG_VOLTAGE Y CFGBVS Y BITSTREAM.CONFIG.CONFIGRATE Y BITSTREAM.GENERAL.COMPRESS .
Basys-3-Master.xdc             CONFIG_MODE Y CONFIG_VOLTAGE Y CFGBVS Y BITSTREAM.CONFIG.CONFIGRATE Y BITSTREAM.GENERAL.COMPRESS Y

I think it would be useful to homogenize this, i.e., specify good default values for all five configuration-related parameters in all XDC files.



W.r.t. the CONFIG_VOLTAGE and CFGBVS properties: Vivado complains if they are not present (even if the defaults appear to work fine). Best to specify them.
 

check_digilent_xdc.py

[artvvb]

I'm back in the office and am going to be looking into this when I get a chance - the XDCs missing these settings should definitely have them added, just not sure which yet. Likely all spartan and artix boards. It's a change we likely should have made years ago. Bitstream compression and as high of a config rate as possible are also no-brainers.

To add on to board behavior, it should be the case that boards that have been programmed in a way that they don't boot from Flash on power on can still be booted from flash by pressing a prog button, assuming the board features one. Doesn't help when there is no prog button though, like on the Cmod A7.

Thanks,

Arthur

[Guest]

@artvvb

W.r.t. boards forcing a flash boot: there is also the "Boot from Configuration Memory Device" in Vivado's Hardware Manager (right-click the FPGA, it's listed in the context menu).