Pmod GPS: NMEA message corrupted at high baud rates

[alexandre.willeme]

Dear all,

I'm trying to integrate Pmod GPS on my Xilinx ZCU102 platform with a Linux OS.  To do so, I'm using the resources available in Digilentinc's vivado-library zip (PmodGPS_v1_1 IP + software drivers using xuartns550).

I managed to make it work correctly in polling mode (calling GPS_getData func) with a clock freq of 49.995 MHz and a baudrate of 9600 symbols/s.

Nonetheless, when I try to increase the baudrate to 115200, the NMEA messages in the ->recv buffer get corrupted. For instance, some messages are not correctly formatted because one comma has been flipped to another char and it makes the GPS_formatSentence function to seg fault.

It's really important for my project to achieve higher rates (I'm working on a 60 fps camera and I would like to associate a GPS data to each frame and can not afford to spend too much time in the GPS_getData function in I want to preserve real time operation).

Here are the hypothesis I make on the cause of this problem:

a) The clock at both ends of the UART link are not synchronized. As far as I understand, this problem can also occur at low baud rates so it's probably not the main cause (cf PercentError computed in XUartNs550_SetBaudRate).

b ) There is corruption on the physical link (my PMOD in connected to the ZCU102 using female-female jumper wires).

Here are the workaround I tried so far:

a) Check the checksum of the NMEA message before calling GPS_formatSentence and discard corrupted messages. This improve the reliability of the system but some problems still appear from time to time which is definitely not acceptable. I feel like it will be quite complex to design a check that is able to handle any kind of corruption (what if a $ sign or a * gets corrupted ?)

b ) Use baud rate = 9600 and call GPS_getData and GPS_formatSentence in another thread (so that the main thread is not waiting for GPS_getData to complete). Unfortunately, I only have little experience in multi-threaded programming and I did not manage to make it work properly. I also think it's not very good practice, it would be better to make it work at higher baud rates.

Do you have any complementary idea about how to fix that ?

Thanks in advance,

Alexandre

[D@n]

@alexandre.willeme,

Is there any possibility you are overrunning your buffer by not reading from the GPS NMEA stream often enough?

Dan

 

[Notarobot]

@alexandre.willeme

According to the datasheet Pmod GPS has default baud rate of 9600. Unless you can change this rate on Pmod it is the only rate you can use for communication reliably.

I would suggest using UART with interrupts to avoid waiting instead of multishreading which is tricky to program.

[alexandre.willeme]

Hi D@n and Notarobot!

Thanks a lot for your answer.

@Dan: It's not overrunning as the PMOD is supposed to write the buffer every 1/10 sec and I try to read it every 1/60 sec.

@ Notarobot : the issue is that, when interrupt is triggered in single-thread operation, my CPU will read the UART buffer at 9600 symbol/s and it will freeze the video pipeline during more than 1/60 sec which will make the video to freeze. The only way around that at 9600 symbols/sec involves multi-threading. Am I wrong? Furthermore, I really think that this PMOD can be setup at >9600 symbols/sec as it is done in the Digilentinc 's vivado_library (cfr.    GPS_changeBaud(InstancePtr, 115200); in PmodGPS.c)

What I do ATM is to operate at 115200 symbols/s and discard NMEA messages that are not correctly formatted (using REGEX) or not correct (according to their checksum). This works and is an useful check anyway but the error rate in the NMEA messages is still quite high ...

Any other suggestion to reduce this error rate ?

Thanks,

Alexandre

[Notarobot]

Hi @alexandre.willeme

1. The interrupt will be generated when the UART buffer is full. The CPU reads the buffer at high speed not at 9600 symbols/s. Please, check documentation on UART.

2. Personally, I would make RTL UART module for reading GPS data independently from the CPU and saving these data in block RAM. Processor can poll these data at any time or read on interrupt generated by the RTL module. Reading BRAM is like reading RAM takes very short time.

3. I might not fully understood your idea but I am sure that correcting errors due to incompatible data rate would never pass the critical design review in the industry.

[D@n]

@alexandre.willeme,

The problem at this point is that you cannot tell where the problem is occurring.  There are many places where this problem might reside, and it's hard to say which one is the culprit.

Therefore my suggestion would be to try to get a trace from the serial port to see where the error lies.  In particular, I'd want to run a logic analyzer internal to the FPGA capturing 1) the incoming UART pin, and 2) the decoded characters from the PmodGPS, and I'd want this analyzer to continuously capture until your software detects a bug.  Such a trace would tell you whether the problem exists in the line from the PModGPS, the UART decoder, or your software processing.

@Notarobot's suggestion above is getting at the same issue--trying to isolate what part of the design is producing the fault. 

Dan

P.S.  I got a thorough chuckle from @Notarobot's third point: "... I am sure that correcting errors due to incompatible data rate would never pass the critical design review in the industry."  Well said!

[alexandre.willeme]

Hi D@n and Notarobot,

I'm sorry about my bad practices, my work is mainly for prototyping / academic research so my expectations in terms of quality are sometimes pretty low even though they should not (but thank you for reminding that :).

Anyway, I found the problem in my project, the PMOD is sending characters by UART to the UART module on the FPGA. These char are stored at the baud rate in the fifo on the UART module that has a depth of 16 char. Apparently, when this fifo is full, the PMOD keeps sending char and the fifo overflows so that I loose char and the NMEA messages gets corrupted.

This means that, on the software side, if I do not react to the UART interrupt quickly enough or if I do not poll the UART fifo often enough, I will loose some characters ... I'm currently running a linux on the CPU and I use UIO (http://heim.ifi.uio.no/~knuto/kernel/4.14/driver-api/uio-howto.html) as driver to control the UART module on the FPGA.

As far as I can see, the ways to avoid this FIFO overflow issue are the following (taking into account that I can poll the FIFO every 1/60 sec):

1) Set a UART baud rate that is low enough so that 1/60 is not enough for the fifo to overflow (that is currently what I'm doing with a baud rate of 4800 symbol/sec). Of course it is not best practice :).

2) Find a good way to catch the UART interrupts when it arises on Linux (nonetheless, it is not that simple to do on the application side using UIO driver, I already spent a significant amount of time working on that without success ...). That's basically what is done on the demo in Digilent's vivado-library but without Linux (bare-metal).

3) Do as Notarobot suggests and replace the small fifo by a larger blockram that can store whole NMEA messages, that can be polled whenever the CPU want. This last option is probably the best but I'm afraid it can also be the most time-consuming ...

 

What do you think about option 2? Do you have any clue how I can handle these interrupts as soon as they occur on Linux using UIO driver ?

 

 

 

 

[Notarobot]

Option 2 is typical approach when dealing with async communication.
Check this Wiki http://www.wiki.xilinx.com/PS+UART