Getting lwIP and zmodlib to be friends

[davwamai]

Hello again. I'm in the process of developing a system based on the Eclypse Z7 board along with the 1411 and 1410 ZMods. 

The aim is to display the collected ADC data on a host machine oscilloscope style. Dumping the ADC buffers over UART is too slow, so I've opted for ethernet instead. 

The lwIP TCP server template included in SDK/Vitis makes it really easy to get high speed transfers between the PS and the host up and running quickly, which was really nice to see after struggling with other communication methods for a while. Now all that's left is getting the zmods to do cool tricks and send their results to a host machine with the help of lwIP. 

Small problem... the lwIP library and its template functionality is written in C, where zmodlib is written in C++. I feel like this is such a silly thing to complain about. All I really need to do is wrap the ZMOD and ZMODADC1410 C++ classes/functions in an extern and handle name mangling, but it's just a very un-fun process, especially in a fairly dense and unfamiliar library. 

I just wanted to put this issue out there to see if anyone had any clever ideas or suggestions on a possible workaround, to simultaneously check if anyone could find some flaw in my thinking (since I tend to get too excited and forget to take into consideration very important details) and give me time to procrastinate writing the interface code for zmodlib. 

Thanks,

David

[zygot]

29 minutes ago, davwamai said:

Dumping the ADC buffers over UART is too slow, so I've opted for ethernet instead. 

Why do you believe that? Clearly moving 100 MB of data over a 12 Mbaud UART channel is time consuming. Moving 16 KB over a 12 Mbaud UART might not be a burden. It depends on what you need to do with that data.

29 minutes ago, davwamai said:

I just wanted to put this issue out there to see if anyone had any clever ideas or suggestions on a possible workaround

I've proposed some ideas in a previous thread that you started on this topic and you pretty much ignored it; so I'm guessing that you dismissed it out of hand. If you don't respond, then you are indicating that you don't want additional information about what's are concerns are..

The best solution is to figure out how the GEM on the Z7020 works, either in Linux or the stand-alone XIlinx supporting libraries. Evidently you've decided that this is too hard. That doesn't leave you with many options as there's no alternative hardware interfaces other that the 2 PMOD headers. There's also the Ethernet using the SYZYGY option but perhaps that's too hard as well.

You might be able to prompt better replies to your questions if you were more open to discussion and revealing information about your project requirements.

I've put in a lot of work trying to make the Eclypse-Z7 a useful platform so I am not uninformed on the subject.

I'm not in any way offended if you don't want my advice, by the way.

Edited July 12, 2023 by zygot

[davwamai]

4 minutes ago, zygot said:

I've proposed some ideas in a previous thread that you started on this topic and you pretty much ignored it; so I'm guessing that you dismissed it out of hand.

You're right, I did ignore it. It was of poor etiquette to not reply after you put in the time to provide me with some pretty valuable ideas/information, and for that I apologize.

25 minutes ago, zygot said:

The best solution is to figure out how the GEM on the Z7020 works, either in Linux or the stand-alone XIlinx supporting libraries. Evidently you've decided that this is too hard.

Which you also suggested in the referred thread. You're correct in your assumption that I decided it was too hard. When I did begin to read into the GEM, it was admittedly overwhelming, so I mistakenly decided to give up on even attempting to understand the system entirely. The fact of the matter is that I am trying to get a comparatively complicated (to me, at least) system up and running with what very little experience and very little time I have, so I am going to default to whatever solution seems to me as the path of least resistance (in this case, it seemed like lwIP). I understand now that more effort is required to implement something as non-trivial as what I am trying to do, so I will force myself to change my mentality and not give up so easily when faced with something daunting or unfamiliar. 

25 minutes ago, zygot said:

You might be able to prompt better replies to your questions if you were more open to discussion and revealing information about your project requirements.

Also true. My communicating the exact requirements of my system have been minimal, I will be sure to correct this when posting in the future.

For context, this is an undergraduate research project in which I have been tasked with using the EZ7 platform to implement an analog signal acquisition/generation system for a compact NMR. The ZMod DAC will be used to generate the stimulus signal for the chemical sample, and the ZMod ADC will be used to capture the response signal from the NMR system. Both SYZYGY ports are used for each of the cards, which is also why I dismissed your suggestion of using the 1 GbE Ethernet PHY pod with a SYZYGY interface. The captured signal must be sent and stored on a host machine for post processing.  The sampling rate for the ADC must be above 50Msps, which felt like an amount of data too great for a standard 12Mbaud UART channel to handle. In truth, I really do not know where to begin to determine this for sure. Instead of putting in the effort to figure out how to determine the capabilities of UART, I foolishly opted for ethernet instead. 

The current implementation is a doctored version of the provided Scope demo from Digilent, utilizing the zmod libraries to acquire polling into a buffer that can be used to process the contents or send them elsewhere. My plan was to take the buffer populated by the aquireImmediatePolling() function and pass it to lwip_send(), but I naively assumed that would be the end of my efforts, failing to consider opening a connection in the first place, configuing netif, etc.

42 minutes ago, zygot said:

That doesn't leave you with many options as there's no alternative hardware interfaces other that the 2 PMOD headers.

The PMOD headers, as you also suggested in the other thread, unfortunately do not share the same required capabilities as the zmods. With the AD5's sampling rate at 4.8kHz, it does not meet the 50Msps requirement. 

In the end of the previous thread you also advised me to learn the Ethernet stack, which I did immediately after reading. You wouldn't know that, because I did not reply. I began to play around with lwIP to develop my competencies and to make the top levels of the stack more familiar to me, which lead me to get too excited and start this particular thread. My knowledge is still inadequate to implement anything useful. 

Going forward I will figure out how the GEM works, as per your statement of that being the best solution. I do not doubt the amount of work you have put in to make the EZ7 a useful platform, for this I will trust your judgement. Again, apologies for the immaturity surrounding my interactions on this forum, as well as for my indifference towards learning what is unfamiliar. I will be sure to address these issues going forward. 

Thank you for the perspective. 

[artvvb]

For what it's worth, it looks like the problem of trying to get C and C++ to play nice in the same project is not an uncommon issue, there are a couple of Xilinx support forum threads on the topic:

• https://support.xilinx.com/s/question/0D52E00006hpOeeSAE/vitis-combining-c-and-c-in-a-project?language=en_US
• https://support.xilinx.com/s/question/0D52E00006hpjZ8SAI/tips-for-sdk-c-projects-using-c-source-files?language=en_US

Thanks,

Arthur

[zygot]

2 hours ago, davwamai said:

The sampling rate for the ADC must be above 50Msps, which felt like an amount of data too great for a standard 12Mbaud UART channel to handle.

The sampling rate is one aspect of your requirements. How many ADC and DAC channels you need to support is another. How many samples are used for the stimulus and how many are used for the response is yet another. The stimulus/response system is pretty typical. If you can fit all of your ADC and DAC samples in BRAM, then your project can be pretty straight-forward and easy, with all of the hard stuff in the PL. If you need 2 million samples for both converters, then things will be tougher. It all depends on how you design your hardware architecture.

The Eclypse-Z7 can be fine for such a project where all storage can fit into PL resources. Since the PL has no external storage other than the PS DDR. Larger storage requirements will involve 2 bidirectional DMA between the PL and the PS external memory controller. I was never suggesting that a UART could stream ADC and DAC samples in real time. But certainly this is an option if all you need to do is write to 1 32 KW BRAM storage element before you send the stimulus and read from another 32 KW storage element after the response sample time has ended. You can also do this using the PS UART, but at a considerably lower baud rate. All of this depends on your non-specified requirements. You don't need to provide those, but the more information that you can provide, the better the replies will be.

In theory, one 1 GbE can simultaneously stream 1 channel of ADC and DAC 16-bit 50 Msps samples with an arbitrary sample depth. Doing this through the PS  GEM is problematic because the GEM is DMA driven, the samples are DMA driven, the PS cores use the DDR and the DDR is where the samples are stored. It's an unfortunate "feature" of the Eclypse-Z7 design. The platform isn't designed stream data from 1 ADC ZMOD and 1 DAC ZMOD to an external device.

Really this comes down to whether or not the Eclypse-Z7 can meet all of your project requirements ( and you can come up with a suitable design architecture using its features ) or not.

Opal Kelly had a crowd funded BRAIN-1 project to fund it's SYZYGY development. This board has 3 standard ports and 1 transceiver port. Unfortunately, it's also ZYNQ based with no external storage available directly to the PL. I also uses the Z7012S which is a device with very limited PL resources. But it could support 1 DAC pod, 1 ADC pod and 1 Ethernet PHY pod, meaning that it could use an Ethernet PHY as a streaming interface as long as the total data rate each direction is < ~120 MB/s.

One last thought. If you can create your stimulus waveform in the PL logic, then this dramatically simplifies your overall design problems.

Edited July 12, 2023 by zygot

[zygot]

4 hours ago, davwamai said:

The aim is to display the collected ADC data on a host machine oscilloscope style. Dumping the ADC buffers over UART is too slow, so I've opted for ethernet instead.

Well, I overlooked this bit when I wrote the reply just preceding this.

I'm not sure how to interpret this based on what you've revealed so far.

[davwamai]

41 minutes ago, zygot said:

If you can create your stimulus waveform in the PL logic, then this dramatically simplifies your overall design problems.

This is another very important detail I have yet again failed to deliver. The stimulus signal is being generated in the PL via a DDS compiler. The DDS samples are sent directly to the low level DAC controller. Depending on the version of the controller IP, an AXIS interface is used between the DDS compiler and the controller. From what I understand the DDS compiler generates a LUT of some sort, which may be something that I would need to look into now that you mention the potential for high throughput in/out of DDR, assuming DDS samples are stored there. I'll look into this. 

16 minutes ago, zygot said:

Well, I overlooked this bit when I wrote the reply just preceding this.

I'm not sure how to interpret this based on what you've revealed so far.

This inconsistency is merely one way I see this project panning out. At some point storing the data for post processing as well as displaying it within a GUI in tandem will be the final implementation. Either way, the goal right now is to move the ADC data to a host machine and verify their arrival with a networking tool like wireshark. Achieving that will prompt the next step in developing systems to process and/or display the data. 

I was unaware of the BRAIN-1 until today. Thank you for mentioning it. It looks like a very good compromise at the expense of transfer rates. 

Edited July 12, 2023 by davwamai
clarification

[zygot]

Have you looked at this project? https://forum.digilent.com/topic/20299-fun-with-phasors/

You don't need the DDS IP or compiler to create sine-waves or even more complex wave-forms. It can all be done with some HDL code and BRAM. Of course I don't know what kind of waveform you need to create. Read the documentation for the project above and see if it's relevant to what you want to do.

[zygot]

33 minutes ago, davwamai said:

I was unaware of the BRAIN-1 until today. Thank you for mentioning it. It looks like a very good compromise at the expense of transfer rates. 

I mentioned it because this is a board that is much smaller than the Eclypse-Z7, uses a much smaller ZYNQ device, and yet is more capable. Opal Kelly hasn't updated any of the support material since it was released. I'm not recommending that you consider it as an alternative platform or comprehending that you avoid it.

Opal Kelly also makes the XEM7320, which I've used and might be a more suitable platform for your project. I ported the project in the above link to that board and there's a project posted nearby,

[davwamai]

22 hours ago, zygot said:

Have you looked at this project? https://forum.digilent.com/topic/20299-fun-with-phasors/

You don't need the DDS IP or compiler to create sine-waves or even more complex wave-forms. It can all be done with some HDL code and BRAM. Of course I don't know what kind of waveform you need to create. Read the documentation for the project above and see if it's relevant to what you want to do.

I will take a look at this. The signal we want to create is only a sine wave at 20-30MHz. We've found success with using the DDS compiler for this, but I will take a look into the phasor project out of curiosity. It looks like it would be a sound exercise for my familiarity with BRAM and HDL, and I also recently wanted to expose myself to OCTAVE.

[zygot]

1 hour ago, davwamai said:

The signal we want to create is only a sine wave at 20-30MHz.

Only 20-30 MHz? At 100 Msps how many samples does your DAC get per cycle? You can do the math for yourselves.

How sine-like does your stimulus waveform have to be? You're telling me that you've already done this with the DDS compiler. You've simulated this of course... right? Vivado simulator will render data in analog format is you tell it to.

There are techniques for synthesizing 20-30 MHz high quality sine waves but usually they don't involve a multi-GHz DAC. Have you confirmed that the analog bandwidth for your ZMOD 1411 DAC supports a 20-30 MHz signal? I suppose that if you look at your DAC output with an oscilloscope having an analog bandwidth of 5 MHZ what you might see, besides having a very small amplitude, might resemble a sine wave. Does that make it a ( good enough ) sine wave? Perhaps, but not very likely. 

Have you worked out the requirements for your ADC path?

I don't want to rain on your parade.... just asking a few questions that I'd have asked myself at the beginning of such a project. (Don't take this as criticism or a personal insult... someone who's done certain kinds of work for a few decades should be a bit more organized )  Some mistakes made a the beginning of a project can't be fixed late in the development stages....

 

Edited July 13, 2023 by zygot

[zygot]

51 minutes ago, davwamai said:

I also recently wanted to expose myself to OCTAVE

OCTAVE or MATLAB if you have a student license ( actually  recently I learned that they now provide a reasonable pricing plan for individual personal use ) is the place where you should start this kind of project. Prototype your system in OCTAVE and then start building FPGA HW.

[davwamai]

22 hours ago, zygot said:

Only 20-30 MHz? At 100 Msps how many samples does your DAC get per cycle? You can do the math for yourselves.

How sine-like does your stimulus waveform have to be? You're telling me that you've already done this with the DDS compiler. You've simulated this of course... right? Vivado simulator will render data in analog format is you tell it to.

Success was a horrible choice of word to describe the wave generation aspect of the project, I may have put too much confidence in the preceeding "some."

I have the DAC hooked up to an oscilloscope, and after following a few start-up guides, I found that some people went about generating their signals with a DDS compiler, so to my mind, it was the only way in existence to generate signal data. The raw signal coming out of the DAC sourcing data from the DDSC was less than desirable, an arbitrary signal that just barely resembled a sine wave. It was consistent with the required frequency, so, as per my PI's instructions, I attached a physical analog filter to the output to yield a 'cleaner' wave. I initially thought that this was an incredibly poor decision to make, since when the time came to change frequencies for another molecular sample, fiddling around in the DDSC's configuration window to find that magic combination of parameters yielded a completely different shape at our target frequency. I'd like to use this as an example to reinforce the notion that I have no idea what I am doing. 

After reading the first two sentences of the "Course in Communications" section of the Phasor toy README, I began questioning every design choice I have made for this project in the last few months. I've yet to have any time to play around with your design, but I'm really looking forward to learning how it works. 

22 hours ago, zygot said:

Have you worked out the requirements for your ADC path?

No. I now realize that I've been far too confident in my regard to this project. I'm going to start from scratch and begin researching appropriate DMA transfer techniques, and from there worry about the platform-host transfer medium. 

[zygot]

5 hours ago, davwamai said:

have the DAC hooked up to an oscilloscope, and after following a few start-up guides, I found that some people went about generating their signals with a DDS compiler, so to my mind, it was the only way in existence to generate signal data. The raw signal coming out of the DAC sourcing data from the DDSC was less than desirable, an arbitrary signal that just barely resembled a sine wave. It was consistent with the required frequency, so, as per my PI's instructions, I attached a physical analog filter to the output to yield a 'cleaner' wave.

Usually, students are put in a lab, with an experiment to perform, and given test equipment to use. I'm betting that rarely, if ever, do students get instructions on how to use test instruments. Oscilloscopes can be useful things. The problem is that an oscilloscope is an unrepentant and enthusiastic liar if the user doesn't know how to configure them for the purpose at hand, and is unaware about how the pictures that the user is looking at were created, and how to interpret them.  The default rendering mode for most digital scopes is the sinx/x presentation, AKA "Lie to me" mode. It can be especially confusing when the user is expecting a sine wave to appear. Scope users who are uninformed about what's going on behind the rendered display can make a lot of very bad assumptions and draw very wrong inferences with regard to these pictures. You got sucker punched... by a box with an on/off switch, and insufficient instructional preparation. This is a whole topic for discussion.... for another time.

The first thing that a lot of people do, if they are trying to do something that they have no experience in, is to jump right into an exciting problem with a quick plan and  start doing things.. Typically, lab sessions encourage this due to time constraints. It's a good idea to begin every endeavor with a bit of mental rumination about what's involved. The next step is a "sanity check" that often involves a bit of simple math and thought to see if the plan is realistic. Often, with a project involving analog <--> digital conversion some time spent with MATLAB or OCTAVE helps set some boundaries and helps define expectations. 

And that could bring us back to the oscilloscope which is basically a DSP device that does this: analog --> digital --> x/y plot of an "analog" signal that should represent what the scope probe is measuring, but often misleads the user, sometimes badly. Someone could make a whole course on signals and systems just using an oscilloscope as a point of inquiry.

Things like the DDS compile and similar IP has uses, but learning how to use shortcuts that do most or all of the hard stuff for you can be an impediment to learning. Yes, they can be invaluable tools that let you focus on higher level problems. But if you have no idea what the tools do, then they can be impediments understanding what, exactly you are doing.  Basically, the DDS just creates data that used in a lookup table and massaged a bit. Anyone with a modicum of HDL experience can do pretty much the same thing. It may seem like a waste of time building you own DDS compiler using HDL or Verilog and OCTAVE, but everyone thinking of using such IP should, at least once, do it for themselves. The secondary benefits can be more valuable than using such tools to solve complex problems. 

Edited July 15, 2023 by zygot