Using the Waveforms SDK to capture analogue data at 20Mhz

[Alanf]

Hi,

I have a Discovery 2 and have trialled the Waveforms application on Windows and it is very nice however I need to be able to get data out of it at 20MHz. The most that Waveforms can do is able 3-4MHz.

I have created a standalone program based of the example analogin_acquisition.cpp using the SDK and found that the FDwfAnalogInStatusData16 call takes less than about 200us to read and in most cases much faster. The buffer is 8192 in size so should take 409us to capture at 20MHz so in theory we should be able to capture data at 20Mhz easily.

A single acquisition works nicely.

My issue however is that if you use the ScanScreen mode then to use the FDwfAnalogInStatusIndexWrite to get the location of the buffer pointer (I assume this is the pointer where the Discovery 2 is up to) then you have to first call FDwfAnalogInStatus. A call to FDwfAnalogInStatus followed by FDwfAnalogInStatusIndexWrite takes about 2.3ms. 

Any thoughts?

Why is a read of data much faster than a call to get the status?

I have used the internal Windows QueryPerformanceCounter to measure 409us frames and read the data in and it seems to work fine at full speed (at least for 20ms). Assumption that the Windows performance counter is reasonably accurate. Still yet to validate the captured data as this is on my desk - not in the lab.

 

Update: OK the data is not valid. it looks like it copies the same data over and over. So it looks like using ScanScreen mode, you still have to use FDwfAnalogInStatus to update the buffer. 

Edited November 2, 2022 by Alanf
Addition

[Guest]

Hi @Alanf

I am not entirely sure about what you can achieve in terms of samples/second from the top of my head, but I could try quite easily.

What are you trying to do precisely? How long will your acquisitions be ideally (perhaps of indefinite length?) Do you need 1 or 2 channels? Do you need some form of triggering?

I think what you're trying to do suggests you should use the "Record" mode. One thing to be aware of is that only the FDwfAnalogInStatus call actually transfers data from the AD2 to the PC; the "Read" functions merely accesses a buffer on the PC-side that was transferred during the most recent status call.

Cheers, Sidney

Edited November 2, 2022 by reddish

[Alanf]

Thanks Sidney,

That makes more sense now and answers the questions to the timing.

I need to capture about 20ms of analog data from two channels at 20MHz min and trigger from an external source or one of the channels so looks like the Discovery 2 can’t do that with its buffer size.

Looks like the Pro has a much larger buffer so will probably work better.

Alan

[Guest]

Hi @Alanf

Okay, so you want to sample 20 MHz * 20 ms * 2 channels ==  800,000 samples or thereabouts.

First thing to know is that the Digilent devices can be opened in different configurations, which have different capabilities w.r.t. on-device buffering. Here's a small list of the available configurations of devices I've had in my hands:

https://pydwf.readthedocs.io/en/latest/background/DigilentWaveformsDevices.html

From this, you can see that the AD2 can be configured to have a local buffer allocation for the analog-in instrument of 16384 samples, which is twice the default. But that is not enough.

Unfortunately, the higher-end AD Pro 3450 does not meet your requirement either; in its best configuration for this application, it merely provides 65,536 samples in the device.

There's a at least two possibilities I see to proceed....

(1) It is possible to try what dual-rate sample rate is sustainable when streaming data, both on the AD2 and the ADPro 3450. In the case of the ADPro 3450, there's the extra option to stream via 2 different interfaces (USB2, Ethernet); and also, the device can be booted in two different modes which may have different performance characteristics.

(2) The AD 3450 Pro device runs Linux internally, and it is possible and documented to log onto that device, and run programs on it. Because this cuts out the USB/Ethernet path, I wouldn't be surprised if this greatly increases the sustainable bandwidth for capturing. It is, however, somewhat of a hassle to do this.

If it is helpful I could experiment a bit with what is achievable; I have access to an AD2 and an ADPro 3450. Let me know.

For your use-case, a relatively low-end dedicated digital oscilloscope may also do the trick. Those often provide more device buffering, but be sure to properly study the datasheet.


 

Edited November 2, 2022 by reddish

[Guest]

@Alanf

Some more info.

I sort-of forgot about this, but my pydwf Python package comes with an example that's pretty much spot-on for testing the analog-in performance in record mode ; see AnalogInRecordmode.py in https://pydwf.readthedocs.io/en/latest/background/Examples.html

This example does a two-channel loopback and records a configurable amount of data at a configurable sample frequency, reporting any performance issues encountered. The performance is limited by the communication (not by Python).

Recording 20ms of data on my AD2 I can do about 700 kS/sec. On an ADPro3450 via Ethernet I can do about 6 MS/sec without glitches.

I also tried to do it on the ADPro device itself, but that performs surprisingly bad, probably due to the most CPU (an older ARM core). Less than 5 MS/sec for sure.

So perhaps for your application, the Digilent devices simply aren't well suited.

 

Edited November 6, 2022 by reddish

[Alanf]

Thanks @reddish

I think you are right in saying that the Digilent devices are not well suited.

I though that the specs on the pro say 1Ms (my assumption was 1million samples buffer) but you are saying its only 65k buffer which as you say is not enough either.

I do have a low-end oscilloscope but its resolution of data is (can I say crap) not as good as the Discovery Pro 2. The resolution I need is 14bits over -10v to +10v.

I was also thinking that the analogue waveforms I am trying capture are frame based (repeat every 20ms) so in theory if the waveform is static (in alot of test cases it is) then I could capture the data in slices (capture one buffer at a time) using offset from trigger point and add them together. Not ideal but in theory might work.

Anyway thanks for your help with this.

Alan

[Guest]

Hi @Alanf,

Your resolution and range requirements are quite ambitious. What exactly do you mean by "14 bits" -- effective bits? If so, that makes matters a lot more complicated. 14 effecitive bits at 20 MS/sec is asking quite a lot.

The AD2 has a 14 bits ADC with two range settings (approx 5Vpp and approx 50 Vpp). You'd need the 50Vpp setting so from that alone you will lose a few bits, disqualifying the AD2 (and the ADPro).

If these are really all necessary (capturing 2x400k samples at 20 MHz, with 14 bits resolution in a -10..+10V range) I'm afraid you will be needing some rather fancy high-end equipment, with a matching pricetag. I recommend that you carefully review if all your requirements are strictly necessary or just nice-to-have, because what you're trying to do is only easy if money is not an issue.

Your idea to take snapshots of a static waveform and pasting them together can be made to work, but you need to be aware of timing jitter (of trigger to actual sampling), as the AD2 samples its trigger inputs at 100 MHz I think, meaning your fragments will be shifted by some random amount between 0 and 10 ns between different acquisitions, which (at 20 MHz) is borderline annoying.

You will have to repeat at least 50 captures while varying the trigger delay. I'd consider taking a lot more fragments with more overlap, and using the overlap between successive fragements to reconstruct the full signal. It's not trivial to get rid of artefacts, but perhaps that's not super important for your application.

[Guest]

Hi @Alanf

I looked at the spec (see https://digilent.com/shop/analog-discovery-pro-3000-series-portable-high-resolution-mixed-signal-oscilloscopes/), and it indeed does claim 128 MSamples of input buffer size in record mode (see below).

Perhaps @attilacan explain what that spec means, and how to use that 128 MS buffer? Because the behavior I am seeing on my ADPro 3450 is not that I can just record 128 MS at 20 MHz without glitches, unfortunately. With a buffer like that, I should be able to record a few seconds.

Cheers Sidney

[attila]

Hi @reddish

See the WF application Help:

[Guest]

Interesting, I'll test tomorrow what this means for @Alanf's usecase.

 

[Alanf]

OK so I have connected it to real signal and found it works way better than I expected.

I originally calculated the 20Mhz sample rate based on the specs of the source signal max slew rate but it turns out the actual slew rate is no where near that figure and the Discovery 2 capturing at 1.6Mhz gives an excellent result (10ms of capture). At 800khz is also very good.

I was wrong with the 14bit, its actually 12bit.

Do you know much about the math functions? Can I use math to show the delta change between samples on a channel?

 

[Guest]

Hi @Alanf

I am not sure what you mean by math functions, in this context. Once you have the samples in your PC and copied them into an array using your preferred programming language you can do whatever you want to do with them, of course. Other than that, the API is of little use there.

[Alanf]

I mean adding a math channel in Waveforms app.

yes I can do this in a custom app.

 

[Guest]

Hi @Alanf, @attila

I have verified that the AD3450 indeed can use a much larger buffer, provided it boots into "standard" mode rather than "linux" mode. I just succeeded in capturing a 2-second 2-channel 20 MHz signal without issue.

@attila -- The device reports that it has 32/64K samples of buffer space (via its configuration, in the enumeration API), irrespective of the mode that it's in. This number is valid in Linux boot mode, but in standard boot mode the actual buffer size available is much larger thanks to the DDR. Reporting a higher number in case a device is running in standard mode would be better I think?

Or at least, in the documentation, a remark could be added to point out that the AD3x50 series uses a much bigger buffer in DDR mode.

I for one didn't know this, and it's pretty awesome.

[attila]

Hi @reddish

The DDRam buffer is only used for analog/digital-in (Scope/Logic-Digital) recording, to assist as a secondary cache. This is intended to be transparent to the user.
The 32/64Ki (BRam) buffer can be used up to 125MHz on 4channels and 16dios. The recording with or wo the DDRam has some limitations.

[Guest]

Hi @attila

> This is intended to be transparent to the user.

But it really isn't, completely, since the behavior of the device changes between Standard and Linux mode. As an API user, relying on the API documentation, I have no way of knowing that the device can actually do this, as far as I can tell, since the use and limitations of DDR are not discussed.

A question: in the GUI program, what does the "DDR buffering" checkbox do, actually? In the API I don't see options to control use of DDR?
 

[tdeball]

Hi,

I was typing up a new question and decided to search again for the answer and found this thread which is basically identical to my question. This thread has been extremely helpful!

I am currently investigating the AD2 and AD3250 for an embedded system. I require at least 1MS/s (ideally faster) for 200ms (minimum buffer size of 200k). Signal is <5V and 12bit resolution is required.

If I understand correctly, this discussion rules out the AD2 for my application, but the AD3250 should far exceed my needs. Can someone please confirm?

I will be using the SDK and not the waveforms app, does this change anything regarding data capture rate capability?

[Guest]

Hi @tdeball

How many channels do you need?

There is no performance difference between the waveforms application and the API; the waveforms application also just uses the API to interact with the devices.

Which language do you plan to use?

Cheers Sidney

[tdeball]

Hi @reddish,

Great to know about the waveforms application and the API.

I only need 1 channel and an external trigger.

I am currently working in Python and progressing pretty well through my proof of principle prototype. I am open to changing to C++ if needed depending on the speed performance i get once i start manipulating the buffered data.

I can post a new question if needed, but is there a way to get more voltage ranges? There is a pretty big jump between 2V and 50V and this makes the LSB change significant. My signals will be between 0.1V and 5V so when using the AD3250, I will need to be in the "High Range" of +/-50V / 14 bits = 3.3mV resolution. If i could set it to +/-5V instead, my resolution could be 10 times higher.

[Guest]

Hi @tdeball

If you need just one channel, 1 MS/s is probably quite doable with the Analog Discovery 2. I can verify if that's useful for you.

If you're working in Python, chances are you're using the low-level API that's based on direct ctypes interfacing as shown in the examples provided by Digilent. I recommend you consider using pydwf, which makes life a lot simpler by taking care of the ctypes stuff and proper error handling. I made that package, so it's only natural that I recommend it; but if you try it I hope you'll agree that it is an improvement.

For the AnalogIn device, the pydwf package comes with a nice example that does two-channel acquisitions of a configurable duration and sample rate, which is quite useful to assess if a certain type of acquisition is sustainable. It should be easy to modify that example to the support the functionality that you need. See here: https://github.com/sidneycadot/pydwf/blob/master/source/pydwf-examples/AnalogInRecordMode.py and especially take note of the fact that the code looks much nicer without all the ctypes stuff. The example is pretty long because it also sets up nice analog-out signals and does live plots; but the core of what you need is perhaps 20 or 30 lines of code.

If you want to acquire 200,000 samples, you will have to use "record mode", which delivers blocks of consecutive samples. It's a bit of a puzzle to make sure that your acquisitions are well lined up w.r.t. your trigger signal. The example I linked to above show how to do that; it is unfortunately not well-documented in the authoritative DWF docs, and it took me some time to figure out. I recommend that you save yourself some effort by reading the example.

There should be no need to switch to C or C++, Python is plenty fast to handle a 1 MS/sec data stream. I've done 20 times as much without problems. Almost all performance-critical functionality is handled either by libdwf or numpy, which are both written in C; Python just glues those together at a very modest cost. In general I'd recommend separating the data acquisition scripts from the analysis scripts, but depending on your processing needs it could be possible to combine them.

Another viable option is to have the acquisition done in its own thread. While doing calls to a C library, ctypes will be nice and free the global Python interpreter lock (GIL), so doing this can actually give you quite a lot of performance headroom in your main thread. But stay away from that kind of stuff unless you are comfortable with thread programming.

The AD2 and ADPro support 2 voltage ranges: a 5Vpp range (-2.5 .. + 2.5V, approximately) and 50Vpp (-25V to +25V, approximately). There's no in-between I'm afraid. Depending on what you're doing and the characteristics of what you're measuring, you could perhaps apply an offset to the DUT signal, or a voltage divider if the output impedance is low enough to allow for that. Also, the AD2 internally samples at 100 MHz and if you request 1 MS/sec, will average 100 14-bit samples for each sample reported, giving extra resolution and SNR. The samples actually transferred are 16 bits, so I think at 1 MS you will actually get 16 useful bits per sample (but I have to confirm to be sure).

Cheers Sidney

 

Edited November 6, 2022 by reddish

[Guest]

@tdeball

To optimize your range, there is another option that I didn't mention; you can set an offset to each of your analog input channels, separately, so you should be able to center around 2.5 V, with the 5 Vpp range, which should serve your demands.

However, be careful to read back both the offset and range settings of your analog in channel after configuration. First thing you will notice is that the values are not precisely what you set earlier; this is due to device calibration. The "real" offset and range of the instrument (as reported) are what's used by the device to translate raw binary samples to analog voltage values.

Second, you may notice that the library silently sets your range back to the"high" range (+/- 50 Vpp) in some conditions, even if you commanded the "low" range. This behavior is not properly documented and I haven't bothered to reverse engineer what happens exactly. For now, just print the values and make sure it's in line with what you want and expect.

Edited November 6, 2022 by reddish

[Guest]

Hi @tdeball

I just checked acquisition of a single channel of 200 ms of data using the Record mode, on an AD2.

This works even at 2 MS/sec, although occasionally, once every 100 acquisitions or so, I see USB data drops (working in Windows now; my experience in Linux is generally better). However, those drops can be detected in the API (see example program I linked to earlier), so in most situations you can just deal with that by retrying the measurement.

 

[Guest]

4 hours ago, reddish said:

Also, the AD2 internally samples at 100 MHz and if you request 1 MS/sec, will average 100 14-bit samples for each sample reported, giving extra resolution and SNR. The samples actually transferred are 16 bits, so I think at 1 MS you will actually get 16 useful bits per sample (but I have to confirm to be sure).

Unfortunately, it turns out this was wishful thinking. I tested acquisitions with an independent  signal generator that generates a ramp signal, and 5-second acquisitions in Record mode at 100 kHz. My code is in Python, reading the data by calling FdwfAnalogInStatusData16().

In "Decimate" mode, the samples received are 16 bits, ranging from -32768..+32764 with steps of 4, consistent with a low-level ADC bit width of 14 bits, to which 2 zero bits are appended to scale to a 16-bits signed integer.

In "Average" mode (the default), the samples received are 16 also bits, ranging from -32768..+32760 with steps of 4. So whomever does the averaging (probably the device's FPGA) does so quite sub-optimally, thereby not yielding an increased number of bits that would potentially be possible. That's a shame.

I am also curious why the averaged sample values obtained using FdwfAnalogInStatusData16() will not exceed 32760, even when I have my signal generator generate a DC voltage that will exceed the configured range. To me it suggests that there is probably a bug in the VHDL or Verilog code that does the sample averaging. I will try to reproduce this using waveforms and report it as a bug.

See

 

Edited November 6, 2022 by reddish

[tdeball]

Wow Sidney!

I really appreciate the responses and amount of info you are providing.

I'll back up a step and provide a bit more info about my system. I have a current system that I am updating. My main test circuit produces a 50-100ms pulse (length and magnitude is test dependent and adjustable). The pulse has a 1mV to 40mV portion that rides on top of a relatively large square wave (.5 to 20V) (test dependent). I am keeping it somewhat vague and refraining from using industry key words to prevent this from being easily searchable by our customer. There is a processing section of my test circuit that cancels out the square wave using several variable gain instrumentation amps. The instrumentation amps also scale up the important portion of the signal by 100x. The magnification and cancelling the square wave significantly reduces the data capture (oscilloscope) dynamic range requirement.

Two options for updating the system with a modern USB oscope:

1) keep the test and cancelling circuit the same, use a AD2 or ADpro to replace the existing custom oscope board.

2) I am considering using a high end picoscope to accept the full dynamic range of the test. This would eliminate all the instrumentation amps and the square wave cancelling portion would be done by my software. This would greatly simplify my system circuit. The picoscope can be oversampled to 20bits. One thing @reddish said that is very interesting is utilizing the offset functionality of the ADpro. I know the expected voltage band that the square wave will fall. Could I set the offset at any value and use the "low range" to get a higher effective dynamic range? For example, get 14 bits from 2V to 7V for one test and then change the offset and get 14 bits from 6V to 11V for another test? That would be perfect!

@reddish also mentioned calibration. My equipment is calibrated annually. How accurate can the offset be calibrated?

Edited November 6, 2022 by tdeball

[Guest]

Hi @tdeball

I have some experience with Picoscope equipment -- I am not a huge fan, the PicoScope 5244B I worked with specifically had some rather nasty artifacts in its analog-in; and since their support people were quite dismissive of the report I made about it, I made the commitment not ever to touch their stuff again, if at all possible.

The offset functionality is not only available on the ADPro but also on the AD2. As discussed above, analog inputs have two ranges they can operate in: approx 5 Vpp and approx 50 Vpp. The offset in the "low-range" mode is limited to +/- 5.5 Vpp ; if you set an offset outset that range the range switches to the "high-range" mode. As far as I know this behavior is not documented but it's what I see. As I said before: the trick is to read back the offset/range for each channel after you configured both; it will then tell you the true range/offset in use for that channel (see https://pydwf.readthedocs.io/en/latest/pydwf_api/DwfDevice.analogIn.html#pydwf.core.api.analog_in.AnalogIn.statusData for the calculation).

What you describe /could/ be possible (even with an AD2), but I'd have to try to be sure; I still don't have a complete picture of the offset/range behavior of the analog-in channels, and the authoritative documentation is lacking in that area. Anyway it depends on your precise situation so your best bet is try yourself.

As to calibration of the AD2 and ADPro devices: I don't know how good it is, and how quickly the devices degrade. It is apparently possible to re-calibrate yourself but I never did that. I personally don't put too much trust in absolute accuracy of voltages I see on any instruments unless they are high-end and properly calibrated; my advice would be to check against a properly calibrated higher-end device if possible, it accuracy is important.

Personally I like the Analog Discovery devices for what they are: low-to-medium end devices with surprising versatility and software programmability that is on par or better than many high-end devices. Especially on the old pricepoint around 200 USD, or with academic discount, the AD2 was extremely high value-for-money. The fact that we're even considering if the types of measurements you need are possible is quite astonishing for a device that costs a few hunderd dollars. However, there are obviously cases where the AD2 or the ADPro are less well equipped than truely high-end instruments, and your usecase may or may not be such a case, depending on the details.

[tdeball]

Sounds good.

I'm not sure that offset range will be sufficient for the full dynamic range scenario. I also forgot that the circuit change (removing the instrumentation amps) would eliminate 100x gain so the requirement for the full dynamic range measurement would be higher than 14 bit.

Ultimately, I think the ADpro (and maybe the AD2) will meet my needs. I will buy the ADpro and keep prototyping to find out. This discussion was extremely helpful and informative. Much appreciated!