Digilent Instrumentation Protocol

[David Bradway]

Can anyone point me at more information on the Digilent Instrumentation Protocol that is described here? https://reference.digilentinc.com/reference/software/digilent-instrumentation-protocol/protocol

I am reading the Oscilloscope via a JSON HTTP request using Python. I want to know the format of the data returned in the byte blob when reading the Oscilloscope via the JSON protocol. Are the oscilloscope samples 12 bits? How are the bits ordered and packed into bytes for each sample? I am trying to convert the byte blob into an array of samples for plotting in Python.

Thanks!

David

[Kristoff]

Hey David,

You're probably the 4th person to ever use this documentation (after Keith, Dharsan and me :D) so don't hesitate to ask questions.  We plan to keep updating it to be more user friendly.

Check out the Device>>Enumeration response.  It contains info about the data types returned by each instrument.

"osc": {
                "1": {
                    "resolution": 12,
                    "effectiveBits": 11,
                    "bufferSizeMax": 32640,
                    "bufferDataType": "I16",
                    "sampleFreqMin": 6000,
                    "sampleFreqMax": 6250000000,
                    "delayMax": 4611686018427388000,
                    "delayMin": -32640000000000000,
                    "adcVpp": 3000,
                    "inputVoltageMax": 20000,
                    "inputVoltageMin": -20000,
                    "gains": [
                        1,
                        0.25,
                        0.125,
                        0.075
                    ]
                },

• resolution: 12
  • The actual sample resolution
• bufferDataType
  • The data type in the binary data

So the binary data returned in the Osc Read will be signed 16 bit numbers and each one represents a single sample in mV.

If you get some python code working we'd love to see it!

-Kristoff

[David Bradway]

I got my Python code working and put up a simple repository: https://github.com/davidbradway/python_openscope_example Hopefully this will be useful for some people. Please clone, fork, and submit Pull Requests to improve it!

I have a couple more questions and comments about the API which I will put in separate replies below.

[David Bradway]

I haven't figured out the triggering through the API, so I tested my code with waveformslive running with a continuous trigger.

I see there is an API endpoint for the trigger command Run. Does it work?

[David Bradway]

When the Openscope is on the WiFi network there doesn't seem to be any security. So anyone (or any botnet/etc) can post JSON commands at it? Are there any plans for secret keys, oauth2, etc?

[Kristoff]

Thanks for sharing the Python code, can't wait to check it out!

Run is not currently implemented on OpenScope MZ.  When you click 'run' in WaveForms Live we do a 'single' and as soon as it returns we do another 'single' until the user clicks stop.  We do plan to implement this but since we're currently only support synchronous communication (every data transfer must be initiated by WFL) 'run' won't provide a huge performance boost just yet.

This feature is particularly complicated because ideally we'd use websockets for run.  This means adding web socket support to our embedded network stack, OpenScope, WFL and the Digilent Agent.  Then we have to update the command protocol and our state machines on both ends to support asynchronous data.  Then we could implement 'run' the right way.  So it's on the road map, but may be a while.  We may implement the 'run' portion first to save one HTTP call per buffer.

You are correct that once the OpenScope MZ is on the network it's open to anyone to send HTTP requests.  Right now the OpenScope will respond to the requests in order and it is possible to send conflicting commands from two device.  Our advice for now is to only connect from with one device at a time.  We have talked about adding a username / password or something similar to limit access to the device but it didn't make the cut for 1.0.  I welcome any feedback people have on how valuable this is (especially compared to other features like 'run' protocol analysis, etc).  I created a feature request issue for this here.

 

-Kristoff

[David Bradway]

I started a Python module to interact with the Digilent Instrumentation Protocol (https://reference.digilentinc.com/reference/software/digilent-instrumentation-protocol/protocol) API! Come check it out and contribute to it here: https://github.com/davidbradway/pyopenscope

If you know of anyone who can use this and help me finish it, send them my way! Only some of the API is implemented so far. Check out the README and example_script.py

[nophead]

Thanks for creating this David.

I can't get the example to work. It gets a response code indicating a syntax error in the awgSetRegularWaveform and oscRead commands. These seem to be the only ones with multiple parameters. I have looked at the data sent in Wireshark and it is valid json.

{"awg": {"1": [{"signalFreq": 1000000, "vOffset": 0, "command": "setRegularWaveform", "vpp": 3000, "signalType": "square"}]}}

The response is {"statusCode":2684354567,"Char Location":37}

Character 37 is either the comma or the space after 1000000. I am surprised requests doesn't remove spaces. Could this be the problem?

 

 

 

[nophead]

I modified requests to remove all the spaces and I have the same problem now at character34.

{"awg":{"1":[{"signalFreq":1000000,"vOffset":0,"command":"setRegularWaveform","vpp":3000,"signalType":"square"}]}

Does it count from 0 or 1?

I am using the latest firmware but I note this project is older. Has the command syntax changed?

 

[nophead]

This is the same command captured when waveformslive sends it.

{"awg":{"1":[{"command":"setRegularWaveform","signalType":"square","signalFreq":1000000,"vpp":3000,"vOffset":0},{"command":"run"}]}}

The standout thing is "command" is first. From what I read this can only be guaranteed with Python 3.6 onwards and I am on 2.7.

 

[David Bradway]

Hi. Thanks for your interest. I saw that you submitted a pull requested to the Github repo. Did you figure out your quote type problem? https://github.com/davidbradway/pyopenscope/pull/1

I will be back in the office next week and I will try to take a look at it it and merge your pull request.

Thanks! If you want, please implement more of the API and open another PR! 

[nophead]

It looks like Python 2.7 defaults to ascii encoding and Python 3.6 defaults to utf-8. So my PR is only relevant if the rest of the code can be made to work on Python2, although it does no harm.

I don't see how it can work on Python2 unless the firmware is changed to not require the command first.

 

 

[Kristoff]

The "command" token is part of the Digilent Instrumentation Protocol and is required for the OpenScope to process the request.

-Kristoff

[David Bradway]

On 5/1/2018 at 10:52 AM, nophead said:

It looks like Python 2.7 defaults to ascii encoding and Python 3.6 defaults to utf-8. So my PR is only relevant if the rest of the code can be made to work on Python2, although it does no harm.

I don't see how it can work on Python2 unless the firmware is changed to not require the command first.

 

 

I do recommend you migrate to Python 3.6 as 2.7 is approaching end of life. Not sure if that will fix things and sorry I haven't had time to troubleshoot. Did you try this code too? https://github.com/davidbradway/python_openscope_example

[David Bradway]

2 minutes ago, Kristoff said:

The "command" token is part of the Digilent Instrumentation Protocol and is required for the OpenScope to process the request.

-Kristoff

I think he's referring to the order of commands within the JSON. I don't think python dicts are ordered and I didn't think JSON are either... 

[Kristoff]

Ahh interesting. I didn't look close enough and missed the 'command' in the first packet.

You are correct that JSON does not guarantee the order of the tokens.  That being said every JSON library I've used leaves them in the same order, but Python 2.7 may be special

The OpenScope parser should handle this, but it looks like it may not.  I'll have to ask Keith about this when he is back in the office next week.

-Kristoff

[nophead]

Python dicts are used to pass the JSON to requests. Python dicts were not ordered until version 3.5 where they now preserve the key order. So the only way to ensure the command comes first is to use Python 3.5 onwards.

Using Python 3.6 it gets further into the example but the trigger command returns InstrumentNotConfigured status. This seems to be because the oscilloscope state is idle.

Also my OpenScope won't autoconnect to wifi anymore. I think this is since I updated to the latest firmware. I have to connect it to USB and tell it to connect to wifi, which makes Wifi useless.

[BecauseIHaveto]

I got nearly the same problem. I want to access the OpenScope mz over Wifi with my c++ programm. the communication works fine but everytime I want to read the parameters from the oscilloscope I get

{"osc":{"1":[{"command":"read","statusCode":2684354573,"wait":0}]}}

I've looked it up here: https://reference.digilentinc.com/reference/software/digilent-instrumentation-protocol/status-codes

and it says the oscilloscope is in use, so I guess I can't access it. But what do I have to change in order to get the parameters?

Thanks in advance!

[Fa-b]

@BecauseIHaveto,

Have you reset the instrument prior to using it?

Have you configured and armed the trigger before attempting to read the osc?

As far as I remember there is no 'No Trigger' mode implemented in the OpenScope. If you want this, you will have to manually force the trigger periodically.

 

Regards

Fabian

[BecauseIHaveto]

@Fa-b yes I tried to reset it but this didn't change anything.

I also tried to set the trigger but if I try to forceTrigger it always says "InstrumentNotArmed", and I have no idea what this means?
Maybe you can explain it to me so I can try this again.

Thanks in advance.

[Fa-b]

@BecauseIHaveto,

Have a look at point 19 of the Getting Started Guide. It is Typescript, but the order of commands to be sent to the OpenScope will be similar:

- Set Parameters of the Oscilloscope Module

- Set Parameters of the Trigger Module

- Arm the Trigger Module ('single' trigger is used since 'run' is not implemented in the Openscope, you will have to re-issue the 'single' trigger arming after you successfully read, see @Kristoff's post above)

- continuously call read the Oscilloscope Module (The guide uses a 100 ms Timeout for the calls)

The read will return with a success status (statuscode: 0) with the count of samples taken when the trigger condition is met and a chunk data transfer following the JSON packet. It will return with a statuscode other than 0 if the trigger condition is not met (Probably: Instrument In Use).

You can then force the trigger instead of waiting for a real trigger condition to occur.

It is important that you always wait for the response of the instrument before you send out the next command. The Typescript library uses Observables for that purpose.

 

Hope this helps,

Fabian

[BecauseIHaveto]

First of all, thank you for your helpful answer.

I looked it up and implemented it in my test setup. Sadly, I still get an statusCode != 0. 
In the response I get from the Osilloscope the StatusCode is "2684354571". I did some research
and found a file which shows the meaning of the status. This one means "InstrumentArmed".

The code I used looks like this:

	request(ip_address, json_mode);
	Sleep(500);
	cout << request(ip_address, reset_instruments) << endl;
	Sleep(5000);
	cout << request(ip_address, set_regular_waveform_funct(1, "sine", 350000, 2500, 0, 50));
	Sleep(3000);
	cout << request(ip_address, set_parameters_osci_funct(1, 0, 0.25, 350000, 5000, 0)) << endl;
	Sleep(5000);
	cout << request(ip_address, set_parameters_trig_funct("osc", 1, "risingEdge", 470, 500, "osc", 1)) << endl;
	Sleep(5000);
	cout << request(ip_address, trig_single_funct(1)) << endl;
	Sleep(3000);
	string post_request = request(ip_address, read_osci_funct(1,101));
	string response = extract(post_request, "{", "}");
	cout << response << endl;

`request` is an selfwritten function which sends an HTTP Post Request to the mentioned IP-Address, and returns the answer of this Request.

`extract` just cuts unwanted text befor and after the response text, in order to use the string as an JSON-Object.

The following text is the response from the code above:

{"device":[{"command":"resetInstruments","statusCode":0,"wait":1000}]}
{"awg":{"1":[{"command":"setRegularWaveform","statusCode":0,"actualSignalFreq":350000,"actualVpp":2500,
"actualVOffset":0,"wait":0}]}}{"osc":{"1":[{"command":"setParameters","statusCode":0,"actualVOffset":-14,
"actualSampleFreq":350003,"actualTriggerDelay":0,"wait":0}]}}
{"trigger":{"1":[{"command":"setParameters","statusCode":0,"wait":0}]}}
{"trigger":{"1":[{"command":"single","statusCode":0,"acqCount":1,"wait":-1}]}}
{"osc":{"1":[{"command":"read","statusCode":2684354571,"wait":0}]}}

 

Has somebody pleas some tips or an advice for me? Some things I should change or things I could try?

Looking forward to your responses.

[AndrewHolzer]

Hi @BecauseIHaveto,

What it sounds like to me is that the oscilloscope hasn't acquired any data yet because the trigger conditions haven't occurred. When the trigger is armed, the instrument will wait for the specified trigger conditions to occur, and when that happens some amount of data is acquired and is then made available for the subsequent oscilloscope read command.The error you are seeing makes sense in this case. However, I also see that you are setting of the wavegen instrument and I am assuming that you are feeding this signal back into the OpenScope via oscilloscope channel 1, so we should see some acquired data. I do not see a run command for the awg being sent in the code snippet you sent above, and this right here looks to be the cause of the issue. If you drop in a awg run command after the awg setRegularWaveform, your osc read should (hopefully) come back with some data.

Let me know if that works out for you, or whether there is another issue that needs to be fixed.

Regards,
AndrewHolzer 

[BecauseIHaveto]

Thank you @AndrewHolzer for your response.

Can't believe that i overlooked this. However I changed this, and set the waveGen to running, but I still get the same error as above. This didn't change anything at all.

So the search continues?. But still thanks for replying. If you have any other tips or assumptions please let me know.

Best Regards,
BecauseIHaveTo

[AndrewHolzer]

Hey @BecauseIHaveto,

First, I must apologize for my delayed response. I'm still scratching my head on this. Ideally, before reading the oscilloscope you would issue a getCurrentState command and check to see whether the trigger has fired and data has been made available, then perform the read. With how you've got things configured, the 3 second delay before reading would allow enough time for data to be made available to you. I suggest that you try a getCurrentState before the read, looping on that while there is no data and once there is, then performing the read to fetch it. I also suggest that you make sure that you are looping the output of the WaveGen back into channel 1 of the oscilloscope. It is a simple solution, but its easy to overlook because of its simplicity.

I'll be looking forward to your update.
AndrewHolzer