Capturing ONLY the relevant signals, or filtering them out (eg. chip select high)

[rvanee]

Hi,

I'm new to using the Digital Discovery. I have the Analog Discovery 2 as well, but picked up the Digital Discovery because the Analog Discovery couldn't handle the higher sample rates I needed.

I'm trying to capture a very specific condition. Let's say I have the following:

Signal A
Signal B
Bus (8 bits)
Signal E

I want to capture the values of signal A, signal B, and the bus on signal E's falling edge, whenever signal E has been high for more than 100 ns.

Capturing continuous data and trying to eyeball it is far too time-consuming and there is too much noise to comb through.

I've tried triggering for E going high, but then it just keeps recording and I have the exact same problem, even under "Repeated" mode. And these instances of E going high then low again could be in very fast succession and I don't want the software potentially dropping anything.

I'm reverse-engineering a device and this is my only source of information for what the device is actually doing.

I've also tried to just export data to CSV but it seems to cap out at a certain number of data points, which is well before anything that I care about has happened, and I only see one signal in there instead of all of my signals.

I'm ripping my hair out trying to have a manageable view into anything that might be going on, but all I've been able to do is manually scroll, and eyeball it... and it's far too easy to lose my place in the scrolling.

I don't mind having to script something, but even then I just want to get the data in a usable form.

Please help!

Thanks

Ryan

P.S. Bonus question: If I have a signal that can only be high or low... how do I apply text labels to both states (eg. "Read"/"Write")? Value2Text doesn't seem very intuitive... Value2Text(1,1) in the example seems to imply it's taking absolute values of 1 and 1 instead of values from signals... and how do I tell it which signal to take the values from?

If all of this is in the documentation, I apologize, but so far it seems it's not written in a way my brain can work with. I feel like I'm missing context for most things I read about.

[JColvin]

Hi @rvanee

 

Edit: apparently this message submitted with no additional text. Great job @JColvin

Edited November 18 by JColvin
clarity

[rvanee]

On 11/15/2024 at 2:59 PM, JColvin said:

Hi @rvanee

Hi! I don't know if this was supposed to have an answer in your reply... But I've been trying the SDK and I'm not getting anything that remotely resembles what I see in the Waveforms app.

This is my script... but I'm finding that the majority of what I'm getting returned to me is just a bunch of 0's. I'm using inputs 0-11 on the Digital Discovery.

# 0-7 = LCD Data
# 8   = Register Select
# 9   = Read/Write
# 10  = DEN
# 11  = E (needs to be inverted)

from ctypes import *
from dwfconstants import *
import sys
import csv

def getBitStr(intvalue, bitnum):
    return str(int(intvalue & (1 << bitnum) != 0))

def bitFlip(bitstr):
    if bitstr == '1':
        return '0'
    else:
        return '1'

outputfilename = ''
if len(sys.argv) == 2:
    outputfilename = sys.argv[1]
else:
    print("No filename provided.")
    quit()

if sys.platform.startswith("win"):
    dwf = cdll.dwf
elif sys.platform.startswith("darwin"):
    dwf = cdll.LoadLibrary("/Library/Frameworks/dwf.framework/dwf")
else:
    dwf = cdll.LoadLibrary("libdwf.so")

hdwf = c_int()
sts = c_ubyte()

version = create_string_buffer(16)
dwf.FDwfGetVersion(version)
print("DWF Version: "+str(version.value))

dwf.FDwfParamSet(DwfParamOnClose, c_int(1)) # 0 = run, 1 = stop, 2 = shutdown

print("Opening first device")
dwf.FDwfDeviceOpen(c_int(-1), byref(hdwf))

if hdwf.value == 0:
    print("failed to open device")
    szerr = create_string_buffer(512)
    dwf.FDwfGetLastErrorMsg(szerr)
    print(str(szerr.value))
    quit()

dwf.FDwfDeviceAutoConfigureSet(hdwf, c_int(0))# 0 = the device will be configured only when calling FDwf###Configure

print("Configuring Digital In...")

sampleFreq = 20e06
dwf.FDwfAnalogIOChannelNodeSet(hdwf, c_int(0), c_int(6), c_double(sampleFreq)) 
cAvailable = c_int()
cLost = c_int()
cCorrupted = c_int()
fLost = 0
fCorrupted = 0
hzDI = c_double()

dwf.FDwfDigitalInInternalClockInfo(hdwf, byref(hzDI))
print("Base freq: "+str(hzDI.value/1e6)+"Mhz")
print("Sampling freq: "+str(sampleFreq/1e6)+"MHz")

dwf.FDwfDigitalInAcquisitionModeSet(hdwf, acqmodeRecord)
dividerValue = int(hzDI.value/sampleFreq)
print("Divider value: " + str(dividerValue))
dwf.FDwfDigitalInDividerSet(hdwf, c_int(dividerValue))

dwf.FDwfDigitalInSampleFormatSet(hdwf, c_int(32))
dwf.FDwfDigitalInTriggerPositionSet(hdwf, c_int(0))
dwf.FDwfDigitalInTriggerSourceSet(hdwf, trigsrcNone)
dwf.FDwfDigitalInInputOrderSet(hdwf, c_int(0))
dwf.FDwfDigitalInConfigure(hdwf, c_int(0), c_int(1))

print("Recording...")

with open(outputfilename, 'w', newline='') as outputfile:
    fieldnames = ['LCDData','Register','Read/Write','DEN']
    outputwriter = csv.DictWriter(outputfile, fieldnames=fieldnames)
    outputwriter.writeheader()
    e_ones = 0
    e_zeroes = 0
    try:
        while True:
            if dwf.FDwfDigitalInStatus(hdwf, c_int(1), byref(sts)) == 0:
                print("Error")
                break
            dwf.FDwfDigitalInStatusRecord(hdwf, byref(cAvailable), byref(cLost), byref(cCorrupted))
            if cAvailable.value == 0:
                break
            if cLost.value :
                fLost = 1
            if cCorrupted.value :
                fCorrupted = 1
            cChunk = cAvailable.value
            dataBuffer = (c_uint32*cChunk)()
            dwf.FDwfDigitalInStatusData(hdwf, byref(dataBuffer), c_int(cChunk))
            result = []
            for point in dataBuffer:
                lcddata = getBitStr(point, 7) + getBitStr(point, 6) + getBitStr(point, 5) + getBitStr(point, 4) + getBitStr(point, 3) + getBitStr(point, 2) + getBitStr(point, 1) + getBitStr(point, 0)
                den = getBitStr(point, 10)
                e = bitFlip(getBitStr(point, 11))
                rs = getBitStr(point, 8)
                rw = getBitStr(point, 9)
                result.append({'e': e, 'lcddata': lcddata, 'den': den, 'rs': rs, 'rw': rw})
                bitstring = ''
                for i in range(0, 32):
                    bitstring = getBitStr(point, i) + bitstring
            for index in range(len(result)):
                if result[index]['e'] == '0':
                    if e_ones > 3:
                        if e_zeroes > 2:
                            regtext = ''
                            rwtext = ''
                            if result[index]['rs'] == '0':
                                regtext = 'Data'
                            else:
                                regtext = 'Instruction'
                            if result[index]['rw'] == '0':
                                rwtext = 'Write'
                            else:
                                rwtext = 'Read'
                            lcddata = 'b' + result[index]['lcddata']
                            outputwriter.writerow({'LCDData': lcddata, 'Register': regtext, 'Read/Write': rwtext, 'DEN': result[index]['den']})
                            e_ones = 0
                    else:
                        e_ones = 0
                    e_zeroes += 1
                else:
                    e_zeroes = 0
                    e_ones += 1
    except KeyboardInterrupt:
        pass

dwf.FDwfDeviceClose(hdwf)

if fLost:
    print("Samples were lost! Reduce sample rate")
if fCorrupted:
    print("Samples could be corrupted! Reduce sample rate")

 

Output:

DWF Version: b'3.22.2'
Opening first device
Configuring Digital In...
Base freq: 800.0Mhz
Sampling freq: 20.0MHz
Divider value: 40
Recording...

 

Recording ends when the user hits Ctrl+C.

I must be doing something wrong in acquiring the samples. I've tried:

- 32-bit integers and 16-bit integers
- Different sample rates
- Viewing every bit coming in just to ensure it wasn't an endianness issue, but it didn't seem to be

Occasionally, after a little while, I'll see what looks like random noise on bits 8-11. And there is no indication of lost of corrupted samples at the end of the script.

In WaveForms, the signals should look more like this:

Please help... I'm desperate!

Thanks

Ryan

Edited November 17 by rvanee
Mentioning how to end the script

[rvanee]

I've simplified the program (merging the two subloops into one... and removing the "bitstring" stuff which was a leftover from a test I did).

 

# 0-7 = LCD Data
# 8   = Register Select
# 9   = Read/Write
# 10  = DEN
# 11  = E (needs to be inverted)

from ctypes import *
from dwfconstants import *
import sys
import csv

def getBitStr(intvalue, bitnum):
    return str(int(intvalue & (1 << bitnum) != 0))

def bitFlip(bitstr):
    if bitstr == '1':
        return '0'
    else:
        return '1'

outputfilename = ''
if len(sys.argv) == 2:
    outputfilename = sys.argv[1]
else:
    print("No filename provided.")
    quit()

if sys.platform.startswith("win"):
    dwf = cdll.dwf
elif sys.platform.startswith("darwin"):
    dwf = cdll.LoadLibrary("/Library/Frameworks/dwf.framework/dwf")
else:
    dwf = cdll.LoadLibrary("libdwf.so")

hdwf = c_int()
sts = c_ubyte()

version = create_string_buffer(16)
dwf.FDwfGetVersion(version)
print("DWF Version: "+str(version.value))

dwf.FDwfParamSet(DwfParamOnClose, c_int(1)) # 0 = run, 1 = stop, 2 = shutdown

print("Opening first device")
dwf.FDwfDeviceOpen(c_int(-1), byref(hdwf))

if hdwf.value == 0:
    print("failed to open device")
    szerr = create_string_buffer(512)
    dwf.FDwfGetLastErrorMsg(szerr)
    print(str(szerr.value))
    quit()

dwf.FDwfDeviceAutoConfigureSet(hdwf, c_int(0))# 0 = the device will be configured only when calling FDwf###Configure

print("Configuring Digital In...")

sampleFreq = 20e06
#dwf.FDwfAnalogIOChannelNodeSet(hdwf, c_int(0), c_int(6), c_double(sampleFreq))
cAvailable = c_int()
cLost = c_int()
cCorrupted = c_int()
cSamples = c_int()
fLost = 0
fCorrupted = 0
hzDI = c_double()

dwf.FDwfDigitalInInternalClockInfo(hdwf, byref(hzDI))
print("Base freq: "+str(hzDI.value/1e6)+"Mhz")
print("Sampling freq: "+str(sampleFreq/1e6)+"MHz")

dwf.FDwfDigitalInAcquisitionModeSet(hdwf, acqmodeRecord)
dividerValue = int(hzDI.value/sampleFreq)
print("Divider value: " + str(dividerValue))
dwf.FDwfDigitalInDividerSet(hdwf, c_int(dividerValue))
dwf.FDwfDigitalInSampleFormatSet(hdwf, c_int(32))
dwf.FDwfDigitalInTriggerPositionSet(hdwf, c_int(0))
dwf.FDwfDigitalInTriggerSourceSet(hdwf, trigsrcNone)
dwf.FDwfDigitalInInputOrderSet(hdwf, c_int(0))
dwf.FDwfDigitalInBufferSizeInfo(hdwf, byref(cSamples));
print("Buffer size: " + str(cSamples))
dwf.FDwfDigitalInConfigure(hdwf, c_int(0), c_int(1))

print("Recording...")

with open(outputfilename, 'w', newline='') as outputfile:
    fieldnames = ['LCDData','Register','Read/Write','DEN']
    outputwriter = csv.DictWriter(outputfile, fieldnames=fieldnames)
    outputwriter.writeheader()
    e_ones = 0
    e_zeroes = 0
    try:
        while True:
            if dwf.FDwfDigitalInStatus(hdwf, c_int(1), byref(sts)) == 0:
                print("Error")
                break
            dwf.FDwfDigitalInStatusRecord(hdwf, byref(cAvailable), byref(cLost), byref(cCorrupted))
            if cAvailable.value == 0:
                break
            if cLost.value :
                fLost = 1
            if cCorrupted.value :
                fCorrupted = 1
            cChunk = cAvailable.value
            print('Chunk size: ' + str(int(cChunk)))
            print('Lost: ' + str(int(cLost.value)))
            print('Corrupted: ' + str(int(cCorrupted.value)))
            dataBuffer = (c_uint32*cChunk)()
            dwf.FDwfDigitalInStatusData(hdwf, byref(dataBuffer), c_int(cChunk))
            for point in dataBuffer:
                lcddata = getBitStr(point, 7) + getBitStr(point, 6) + getBitStr(point, 5) + getBitStr(point, 4) + getBitStr(point, 3) + getBitStr(point, 2) + getBitStr(point, 1) + getBitStr(point, 0)
                den = getBitStr(point, 10)
                e = bitFlip(getBitStr(point, 11))
                rs = getBitStr(point, 8)
                rw = getBitStr(point, 9)
                if e == '0':
                    if e_ones > 3:
                        if e_zeroes > 2:
                            regtext = ''
                            rwtext = ''
                            if rs == '0':
                                regtext = 'Data'
                            else:
                                regtext = 'Instruction'
                            if rw == '0':
                                rwtext = 'Write'
                            else:
                                rwtext = 'Read'
                            outputwriter.writerow({'LCDData': 'b' + lcddata, 'Register': regtext, 'Read/Write': rwtext, 'DEN': den})
                            e_ones = 0
                    else:
                        e_ones = 0
                    e_zeroes += 1
                else:
                    e_zeroes = 0
                    e_ones += 1
    except KeyboardInterrupt:
        pass

dwf.FDwfDeviceClose(hdwf)

if fLost:
    print("Samples were lost! Reduce sample rate")
if fCorrupted:
    print("Samples could be corrupted! Reduce sample rate")

 

I've tried commenting out FDwfAnalogIOChannelNodeSet because I'm not sure it made a difference (and this is a digital capture). It was only there because it was present in the "DigitalDiscovery_RecordToFile.py" example.

Output to the console looks like this now:

 

DWF Version: b'3.22.2'
Opening first device
Configuring Digital In...
Base freq: 800.0Mhz
Sampling freq: 20.0MHz
Divider value: 40
Buffer size: c_int(67108864)
Recording...
Chunk size: 7934
Lost: 0
Corrupted: 0
Chunk size: 580282
Lost: 0
Corrupted: 0
Chunk size: 2097152
Lost: 0
Corrupted: 0
Chunk size: 2097152
Lost: 0
Corrupted: 0
Chunk size: 2097152
Lost: 0
Corrupted: 0

Buffer size 67108864 / 32 bits = 2097152 so that checks out, I guess, if I'm dealing with a full buffer nearly every loop. I don't know if that means I'm losing samples. If I was, I would expect that lost and corrupted would be showing values other than 0.

And each loop through 2097152 takes several seconds. If it's supposed to be running at 20 MHz sampling frequency, I would expect to be going through about 10 of these per second. Occasionally, the file being output will have some entries, so I guess I'm not only getting 0's, but the entries look a bit nonsensical in the context of what I know about the chip these signals are destined for.

[attila]

Hi @rvanee

The next application version will add pulse length condition for Sync capture.

With custom app/script to sample only on pulses of at least 100ns, see DigitalIn_Sync.py and use the following trigger setup:
dwf.FDwfDigitalInResetSet(hdwf, 0, 0, 1 << indexOfE, 0) # reset on E rise
dwf.FDwfDigitalInTriggerSet(hdwf, 0, 1 << indexOfE, 0, 0) # valid while E high
dwf.FDwfDigitalInTriggerLengthSet(hdwf, c_double(100e-9), c_double(-1), 0) # for a minimum of 100ns, unlimited max, normal sync mode 0
dwf.FDwfDigitalInTriggerCountSet(hdwf, 1, 0) # count 1
 

[rvanee]

4 hours ago, attila said:

Hi @rvanee

The next application version will add pulse length condition for Sync capture.

With custom app/script to sample only on pulses of at least 100ns, see DigitalIn_Sync.py and use the following trigger setup:
dwf.FDwfDigitalInResetSet(hdwf, 0, 0, 1 << indexOfE, 0) # reset on E rise
dwf.FDwfDigitalInTriggerSet(hdwf, 0, 1 << indexOfE, 0, 0) # valid while E high
dwf.FDwfDigitalInTriggerLengthSet(hdwf, c_double(100e-9), c_double(-1), 0) # for a minimum of 100ns, unlimited max, normal sync mode 0
dwf.FDwfDigitalInTriggerCountSet(hdwf, 1, 0) # count 1
 

Thanks, I'll have a look at these.

I'm also thinking with my script above, I'm not letting the dataBuffer complete its copy before trying to utilize the data, hence all of the 0's. Should I be using acqmodeRecord for continuous capture without breaks in the data, or would something like acqmodeScanShift or acqmodeScanScreen make more sense?

I'm okay with continuous capture to a file until interrupted, I can at least analyze that after the fact, so long as the format is something I can work with.

Thanks

[attila]

Hi @rvanee

Use simple capture if you want separate captures, record/streaming for longer or continuous capture.
The record with divider -1 is 'Sync' mode which only stores one sample (8, 16, 32 or 40bit-channel) / trigger condition, like for E pulses at least 100ns.
This 'Sync' mode, similar to data compression, can work even at high capture rates since usually it requires lower data transfer bandwidth.
The scan modes are intended for visualization purpose.

[rvanee]

4 hours ago, attila said:

Hi @rvanee

Use simple capture if you want separate captures, record/streaming for longer or continuous capture.
The record with divider -1 is 'Sync' mode which only stores one sample (8, 16, 32 or 40bit-channel) / trigger condition, like for E pulses at least 100ns.
This 'Sync' mode, similar to data compression, can work even at high capture rates since usually it requires lower data transfer bandwidth.
The scan modes are intended for visualization purpose.

So if sync mode is on, it can send every instance of whatever meets the trigger criteria?

I reorganized my script so the actual checking of bits happens outside the capture loop, and I was able to get a lot more data this way, but I think Python isn't fast enough as-is to deal with this much data coming in. To use the approach I'm using now, I'd probably have to switch to C or something compiled that plays well with C, like Zig.

I'll play with sync and leave my current script alone for now.

Thanks!

[attila]

Hi @rvanee

Yes, Sync stores one sample / trigger.

[rvanee]

9 hours ago, attila said:

Hi @rvanee

Yes, Sync stores one sample / trigger.

So just to clarify, if the sync is set for:

- reset on E rise
- valid while E high

... does that mean the point in time it captures is when E goes low again, not just when E has been high for 100 ns?

Does increasing the "count" change the number of resets/triggers that get captured into a buffer? Or do I have to deal with each trigger individually before resetting?

[attila]

Hi @rvanee

The sampling is done in the device, there is no need to reset anything. The count doesn't matter.

 

 

[rvanee]

On 11/26/2024 at 5:22 AM, attila said:

Hi @rvanee

The sampling is done in the device, there is no need to reset anything. The count doesn't matter.

 

 

I tried reimplementing the full acquisition in C, and it's a bit faster, but the buffer still fills up before it has captured everything, and I suspect that once that happens, 70%+ is being dropped. So maybe USB is the bottleneck, not the programming language. I presume that even though the data width is set as 16 bits via FDwfDigitalInSampleFormatSet, internally it's still using 32 bits, as the buffer size is reporting as 67108864 samples, multiply by 16 = 1073741824 = 1 Gbit, not the supposed 2 Gbit buffer size.

However it is, I have my doubts that it could keep up anyway, so that leaves sync as the only possible method.

I'm still unclear about when sync takes the sample.

Would it be right at the 100 ns mark, or at the end of the period where E is high?

Thanks

 

[rvanee]

6 hours ago, rvanee said:

I tried reimplementing the full acquisition in C, and it's a bit faster, but the buffer still fills up before it has captured everything, and I suspect that once that happens, 70%+ is being dropped. So maybe USB is the bottleneck, not the programming language. I presume that even though the data width is set as 16 bits via FDwfDigitalInSampleFormatSet, internally it's still using 32 bits, as the buffer size is reporting as 67108864 samples, multiply by 16 = 1073741824 = 1 Gbit, not the supposed 2 Gbit buffer size.

However it is, I have my doubts that it could keep up anyway, so that leaves sync as the only possible method.

I'm still unclear about when sync takes the sample.

Would it be right at the 100 ns mark, or at the end of the period where E is high?

Thanks

 

I got the sync script working! And it was exactly what I was looking for!

I noticed that every time E went high, it was almost exactly 500 ns so I just set the length to 420 ns since all other signals would have changed by then.