Eclypse DDR Streaming Project channel calibration question

[Xband]

I have this hardware and software running and giving output on the serial port.  When I run the calibration routine it seems to give reasonable data but when I run the S2MM code the output seems to be in hex.  My guess is that the cal file isn't referenced properly but its a guess and I can't seem to locate where a file such as this is read in the code, I see the scope_calibration.c file Below are the serial result of the  calibration_reader_system and s2mm_cyclic_transfer_test_system. 

Eventually I want to perform some math on channel acquisitions and spit out a result.  I've input a 1kHz +/- 1V sine wave but direct conversion from hex to decimal in Exce (sorry I hate it too) gives nonsense huge numbers column H & I converting B and C. 

Thanks for any insight. 

 

 

========= Zmod Port A : Zmod ADC 1410-105 Calibration Coefficients =========

    Factory Calibration:   December 24, 2021 at 02:55:34
    CHAN_1_LG_GAIN:        -0.000055
    CHAN_1_LG_OFFSET:      -0.076329
    CHAN_1_HG_GAIN:        0.011868
    CHAN_1_HG_OFFSET:      -0.002765
    CHAN_2_LG_GAIN:        -0.001325
    CHAN_2_LG_OFFSET:      0.130576
    CHAN_2_HG_GAIN:        0.010189
    CHAN_2_HG_OFFSET:      0.006026
    Ch1LgCoefMultStatic:   0x10CC9
    Ch1LgCoefAddStatic:    0x3FE71
    Ch1HgCoefMultStatic:   0x11951
    Ch1HgCoefAddStatic:    0x3FE97
    Ch2LgCoefMultStatic:   0x10C72
    Ch2LgCoefAddStatic:    0x002AD
    Ch2HgCoefMultStatic:   0x118D9
    Ch2HgCoefAddStatic:    0x00316

    User Calibration:      December 24, 2021 at 02:55:34
    CHAN_1_LG_GAIN:        -0.000055
    CHAN_1_LG_OFFSET:      -0.076329
    CHAN_1_HG_GAIN:        0.011868
    CHAN_1_HG_OFFSET:      -0.002765
    CHAN_2_LG_GAIN:        -0.001325
    CHAN_2_LG_OFFSET:      0.130576
    CHAN_2_HG_GAIN:        0.010189
    CHAN_2_HG_OFFSET:      0.006026
    Ch1LgCoefMultStatic:   0x10CC9
    Ch1LgCoefAddStatic:    0x3FE71
    Ch1HgCoefMultStatic:   0x11951
    Ch1HgCoefAddStatic:    0x3FE97
    Ch2LgCoefMultStatic:   0x10C72
    Ch2LgCoefAddStatic:    0x002AD
    Ch2HgCoefMultStatic:   0x118D9
    Ch2HgCoefAddStatic:    0x00316

Done initializing device drivers
TestMode: 0
Initialization done
ADC initialization done
Waiting for trigger...
Last beat found:
  BD base address: 00124168
  BD actual length: 0000000C
Buffer base address: 00124168
Buffer high address: 00128164
Length of buffer (words): 4096
Index of buffer head: 3
Trigger position: 0
Index of trigger position: 3
Detected trigger condition: 00000002
Transfer done
@00124174       FFE70007        3FF9    0001    0       3
@00124178       FFECFFFF        3FFB    3FFF    0       -3
@0012417C       FFF0FFEE        3FFC    3FFB    0       -15
@00124180       FFE7FFFF        3FF9    3FFF    0       -3
@00124184       FFDE0003        3FF7    0000    -1      0
@00124188       FFDEFFEA        3FF7    3FFA    -1      -18
@0012418C       FFD60003        3FF5    0000    -1      0
@00124190       FFDE0003        3FF7    0000    -1      0
@00124194       FFDAFFEA        3FF6    3FFA    -1      -18
@00124198       FFEC0003        3FFB    0000    0       0
@0012419C       FFECFFFF        3FFB    3FFF    0       -3
@001241A0       FFDA0003        3FF6    0000    -1      0
@001241A4       FFE3FFFB        3FF8    3FFE    0       -6
@001241A8       FFECFFF6        3FFB    3FFD    0       -9
@001241AC       036C0003        00DB    0000    26      0
@001241B0       0BD20035        02F4    000D    92      39
@001241B4       0A9100D5        02A4    0035    82      161
@001241B8       04E60157        0139    0055    38      259
@001241BC       049C0191        0127    0064    36      305
@001241C0       055D01AB        0157    006A    41      323
@001241C4       0611019A        0184    0066    47      311
@001241C8       064601AF        0191    006B    48      326
@001241CC       066901BB        019A    006E    50      335
@001241D0       06B401C0        01AD    0070    52      341
@001241D4       06FF01BB        01BF    006E    54      335
@001241D8       073C01C8        01CF    0072    56      347
@001241DC       078201BB        01E0    006E    58      335
@001241E0       07CD01DD        01F3    0077    60      363
@001241E4       082101E5        0208    0079    63      369
@001241E8       087401DD        021D    0077    66      363
@001241EC       08CC0

[zygot]

Really... I wasn't going to read this thread again, and then you replied to my post.

I haven't tried the 2023.1 version of Vitis/Vivado. It does seem that there were major changes. This is what Xilinx has always done; the new release breaks projects developed using older versions of the tools. For commercial users, this might encourage yearly license expenditures. For small companies and educational/recreational users who don't have $3500+ per seat to spend every year it's a major pain in the arse. 2019.2 saw the elimination of most of the old SDK and the introduction of Vitis. Huge impact for ZYNQ development flow. One version of Vivado, it might have been 2019.1 changed the IP database structure. More work for people using old HDL code containing Vivado IP ( like FIFOs ). It never ends.

The Z7020 is ancient. You don't get any benefit from using a recent version of the tools. In some ways you lose; like having IP depreciated.

I've been doing programmable logic development for longer than Xilinx or Altera has been making FPGAs. I've learned to incorporate some self-defense measures in order to keep my sanity ( or what's left of it ). One of these is not to keep up with the never ending "improvements" that break my IP and previous work. I use Vivado 2019.1 and 2020.2 almost exclusively. Both have their own bugs, and a few shared ones; but I know what most of them are by now. Yes, I do stumble into undiscovered bug once in a while ( this just happened a week ago with regard to IDELAY, using code that previously worked with a different clocking structure). A general rule of thumb is to use the version of the tools that came out with the device that you are using. This might mean using an older PC and OS. But the Vivado 2018.3 installer could fit on a DVD.

I would suggest using the same version of the tools that your FPGA board vendor uses for it's product demos and support. While Digilent is not going to make public its Waveforms support for the Eclypse-Z7, it does supply something for that board, and it's a lot more than they do for most of their other FPGA boards.

There are a lot of facets to FPGA develoment, especially ZYNQ development. Unfortunately, some of these involves figuring out how to be productive in spite of the best attempts by the tool vendors to throw boulders in you path. It's not just about the technical details for devices. Be kind to yourself and make the experience as happy as possible. Edited December 28, 2023 by zygot

[Xband]

@artvvb, 

updated the ip and things seem better, creating a new HDL wrapper and will try synthesis. 

Got through synthesis and implementation.  A timing failure.  Bitstream generated fine.  Will try to load onto the hardware when I get a chance.  Upgrading the IP seemed to resolve the issue.  I wasn't sure if it should be done, let sleeping dogs lie was my approach until spiraling into the doghouse. 

Thanks for the support. 

Edited December 28, 2023 by Xband
update

[Xband]

Seemed to have everything compiled in the hardware, went to Vitis in and rebuilt the project seemed ok, had issues getting anything to respond after programming the device over and over.  Switched to  another board which I have and then had this strange error which probably comes from the "calibration" routine.  Had previously run the demo which is why there are "button x pressed!" in the serial display.  

"Unsupported Zmod" was after reprogramming the fpga and then loading the software from Vitis. 

Try more later. 

 

 

[artvvb]

Looking into it. The calibration reader app expects the names to be "Zmod ADC 1410-105" and "Zmod DAC 1411-125", based on the original names of the products before a rebrand. The EEPROM contents that store the product model name must have been changed. Changing lines 23:27 and 53:57 of main.c in the calibration_reader app as follows should clear the issue, using this as the point of comparison if you've made any changes: https://github.com/Digilent/Eclypse-Z7-SW/blob/ddr-streaming/src/calibration_reader/src/main.c.

Quote

    else if (strcmp(DnaStrings.szProductModel, "Zmod Scope 1410-105") == 0) {
        info.family = ZMODFAMILY_ZMODSCOPE;
        info.maxSampleRateMHz = 105.0f;
        info.resolution = 14;
    }

Quote

    else if (strcmp(DnaStrings.szProductModel, "Zmod AWG 1411-125") == 0) {
        info.family = ZMODFAMILY_ZMODAWG;
        info.maxSampleRateMHz = 125.0f;
        info.resolution = 14;
    }

Thanks,

Arthur

[Xband]

@artvvb, 

continuing to beat this dead horse.  Attached is a pdf of the hardware design that seems to compile and can generate a bitstream, though with some errors.  The DDR software will run but since the multi streaming code is written for a 32bit buffer I cant extract it.  

I went into the HW code and made the original AXI stream 64bits depth and tried to change appropriate buffer indexing in the s2mm_cyclic_transfer code but cannot get to the data.  I"m not sure this was successful. 

I've changed the u32 for the Bufferlength to u64 and tried to propagate these changes to indexing through to multi streaming main code.  

Could you give a rough overview on which values should change to u64 (in the s2mm_cyclic_transfer main) or perhaps just a link to some code that would spit the actual hardware  buffer out so I can confirm it has changed to 64bit depth.  

I seen to recall you being against changing this depth to 64, which is entirely reasonable given the complexity that the stream depends on, I tried to preserve the 32bit axi stream data through all the original hw in the base fpga design and then combine the stream just befor the DMA transfer coming out of the ZmodScope Port A. 

I think the basic concept of pulling out the data, using the DSP accumulator and then combining back into the AXI stream should work, though I could be really screwed up with the timing even if it compiles.  I'm interested in using the simulation to try and ring out the timing but this DDR project is pretty deep and I'm not sure how to implement that, though I see you have the ILA ip scattered throughout for this purpose. 

Have a good weekend, sorry I always post these on Friday but I'll typically work through the week trying to get something going and then succumb to the frustration later in the week. 

Thanks for any help.

 

 

ZmodScope_PortA_2_9_2024.pdf

[artvvb]

On 2/9/2024 at 1:54 PM, Xband said:

Could you give a rough overview on which values should change to u64 (in the s2mm_cyclic_transfer main) or perhaps just a link to some code that would spit the actual hardware  buffer out so I can confirm it has changed to 64bit depth.  

Reading a captured stream as 64-bit pieces of data is relatively straightforward. The data would be interleaved - every other word is a raw sample and the rest are accumulated samples. I haven't tested this code, and it would depend on whether your real samples and accumulated samples are in the high or low word of the 64-bit piece, but it would look somewhat like the following:

// start at line 562 of https://github.com/Digilent/Eclypse-Z7-SW/blob/b42fb15a8ab4c52a38db2c15918cd7263e84f65e/src/s2mm_cyclic_transfer_test/src/main.c
    for (u32 i = 0; i < BufferLength; i++) {
        u32 index = (i + BufferHeadIndex) % BufferLength;
        if (index % 2 == 0) { // then it's probably raw samples
          	float ch1_mV = 1000.0f * RawDataToVolts(RxBuffer[index], 0, ZMOD_SCOPE_RESOLUTION, Relays.Ch1Gain);
          	float ch2_mV = 1000.0f * RawDataToVolts(RxBuffer[index], 1, ZMOD_SCOPE_RESOLUTION, Relays.Ch2Gain);
          	const u16 ch1_raw = ChannelData(0, RxBuffer[index], ZMOD_SCOPE_RESOLUTION);
          	const u16 ch2_raw = ChannelData(1, RxBuffer[index], ZMOD_SCOPE_RESOLUTION);
        } else { // it's accumulated samples
        	// do stuff with the samples
        }
        // ...
    }

That said, I'm not certain how the logic that finds the start of the buffer would handle it, and an off-by-one error could crop up. There's a note in the S2mmFindStartOfBuffer function to this effect: https://github.com/Digilent/Eclypse-Z7-SW/blob/b42fb15a8ab4c52a38db2c15918cd7263e84f65e/src/s2mm_cyclic_transfer_test/src/s2mm_transfer.c#L238.

Edit: Note that the DMA is currently transferring 64-bit chunks of data (in the demo project, it's sending two 32-bit samples at once), it's put sequentially into memory, and it's up to the software how to interpret this memory on the software side - it would also be possible to cast the buffer to a pointer of a 64-bit struct type, like the following, and iterate over it using that (though there's additional complexity from the "start of the buffer" potentially showing up anywhere in the memory region allocated for the it). The same possibilities of off-by-one, which word is the "top" of the 64-bit piece of data, and S2mmFindStartOfBuffer maybe needing changes show up here too.

typedef struct {
  u16 acc_data_0;
  u16 acc_data_1;
  u16 raw_data_0;
  u16 raw_data_1;
} rawAndAcc_Data;

rawAndAcc_Data *buffer = (rawAndAcc_Data*)RxBuffer;

// looping not shown here due to the aforementioned complexity around where the start of the buffer is located
xil_printf("%04x", buffer[0].acc_data_0);

 

On 2/9/2024 at 1:54 PM, Xband said:

I seen to recall you being against changing this depth to 64, which is entirely reasonable given the complexity that the stream depends on, I tried to preserve the 32bit axi stream data through all the original hw in the base fpga design and then combine the stream just befor the DMA transfer coming out of the ZmodScope Port A. 

The concern here was motivated by the potential loss of samples. The DMA master interface that connects to one of the PS HP ports is in fact 64 bits wide. In other demo projects, this was only 32 bits wide. The DDR demo increased the DMA master width of to 64 bits it to improve throughput - if the DMA is clocked at the same rate as incoming data (100 MHz for 100 MS/s or something) and the input interface to the DMA (AXI stream) and output interface are the same width, then protocol overhead needed for the AXI master interface makes it so that there will always be some clock cycles where data can't be pushed into the DMA. By protocol overhead, I mean that the DMA needs to take time to do things like specify the address it wants to write to. The way that this issue shows up is that if the DMA master interface is busy, its internal FIFOs are filling up - when they fill entirely, the ready signal on the DMA's AXI stream interface drops, and the rest of the input pipeline backs up. When the DMA becomes ready again, the stream starts up again, but there will have been samples that were lost during the stall.

It may be possible to increase the clock frequency fed to the DMA sufficiently to cover the extra cycles needed for the protocol overhead, but that also makes it harder to meet timing. "Upconverting" from 32 to 64 bits inside of the DMA made this much easier to do, since it doubled the potential throughput at the output, giving plenty of margin to account for overhead. This is why I suggested limiting yourself to one channel of input, so that you could use 16 bit raw samples and 16 bit accumulated data from the same channel, staying within the original 32-bit width.

Alternatively, it might be possible to copy the DMA hierarchy (effectively making a second instance of S2mmDmaTransfer_0) to handle a second 32-bit stream, but I would need to set up a project to see if/how it would work. It would also require software changes in main to manage a second DMA controller.

[artvvb]

On 2/9/2024 at 1:54 PM, Xband said:

 

I think the basic concept of pulling out the data, using the DSP accumulator and then combining back into the AXI stream should work, though I could be really screwed up with the timing even if it compiles.  I'm interested in using the simulation to try and ring out the timing but this DDR project is pretty deep and I'm not sure how to implement that, though I see you have the ILA ip scattered throughout for this purpose. 

To be clear, yes, this concept is sound. I'm only concerned about doing it for both channels simultaneously and passing data up to software through a single DMA controller.

Simulation for this kind of project would be best served by pulling apart the different pieces. Keeping it likely way too vague, you might write a wrapper and a testbench for everything in the stream between the ZmodScopeController and DMA, and verify that it responds to control signals correctly - for example, you'd set an "enable" bit in a testbench that simulates the pipeline instead of trying to include the software and PS in simulation through cosimulation. Doing simulations of small components regularly and on a small scale before integrating them together would be helpful to verify that assumptions we're making are correct.

It's at a really low level compared to the entirety of the project, but this guide might help for starting to poke around the simulator UI, in case you haven't touched it yet: https://digilent.com/reference/programmable-logic/guides/simulation.