Necessary fpga equipment for a particle measurement sensor

[MarcMVT]

Hello guys,

I'm confronted with following problem.

I want to build a data processing setup. I have up to 16 (later maybe 32) channels of an analog signal which will be digitalized with 1 MS/s and 10 or 12-bit.

Since the setup is for a single particle counter there will be only one set of signals with a duration of about 34 µs every 150-200 µs.

The signal processor (maybe a FPGA?) should find the maximum of each signal and put those in a vector with the length of channels I have. Which is up to 16(32).

Afterwards about 1000 of these vectors should be put together in an array to be send to a postprocessing PC via Ethernet ( about 2,56 MBits/s of data) or similar as a reasonable datapackage. Since each row in this array represents a single particle the order isn't important. 

I thought about using an Arty A7. Is this FPGA suitable or can you suggest a better device to me?

Also I have the possibility to design and build ADCs at our institute. But if there is a prebuild solution from digilent it would be interesting for me, too.

Thank you for your help!

Best regards,

Marc

[zygot]

All of this appears to be reasonable. The Arty A7 has a 10/100 Mbps Ethernet PHY so you may have an issue with streaming gap-less data for 32 channels using normal short packets. I'd be thinking of a 1 GbE Ethernet if Ethernet is the required interface. Data pacing requirements are a bug factor in any design. I don't think that there would be an issue processing your data as you describe it in an all HDL design. The Ethernet part might be more complicated than you anticipate, depending on your requirements and experience but is also doable. Usually, people use a soft processor if they need Ethernet because the free IP makes it easier. The problem there is that, for a small device, a MicroBlaze could consume most of the resources. You'd have to prototype a concept for various devices to get a sense. If you are doing point to point Ethernet then this isn't an overly difficult thing to implement in HDL. If you are connecting to a network though switches then it might be a problem. If you can simplify the Ethernet data flow, then things get easier.

If I were to select from platforms for a project like this that I have available I'd be thinking of the MAX10 Development Kit or ( currently MAX10 devices and boards are as plentiful as hen's canines ) or the Mimas A7 ( currently over-priced if available at all ). Both are about $200 and have 1 GbE. There are some inexpensive ZYNQ based boards that might be an option for 1 GbE. ( Actually, I'd probably use the Genesys 2 because it's currently on my work bench; but I'm trying to meet you in the low end budget space as that's where you are starting ). Since the ZYNQ PS has a MAC and 1 GbE you might over estimate the task involved in transferring data over Ethernet. But, from a HW design standpoint it certainly would be simpler. Some Ethernet applications using ZYNQ are trivial like implementing a web server. Other applications not so much. ZYNQ Ethernet is complicated. If you are comfortable creating a custom Linux OS using Yocto or other framework then perhaps this is the best platform to use. It all depends on the Ethernet environment and requirements.

A less elegant approach might be a really cheap Artix board and a Raspberry Pi for Ethernet connectivity. Three wires and a GND could connect the FPGA and RPi using SPI at up to 31.25 MHz data rates. The repetition rate of your data works in favor here. I'd be spending some time trying to figure out how to disconnect the Ethernet part from the FPGA part.. but really this is based on very little information.. and personal preferences.

One nice thing about FPGA development is that you can mock up a prototype design and build it for any device or board in order to get a handle on resource usage, as well as simulate it, before selecting a hardware platform.

[edit] Opps, I forgot about the ADC. One problem with Digilent FPGA boards are those outdated 12-pin PMODs. They aren't very friendly for experimenter's wanting to use a cheap board with custom hardware interfaces. There certainly are mult-channel ADC devices consistent with your requirements available but using the PMODs may be problematic. Not all PMODs on all boards have clock capable pin assignments that might complicate things. Digilent has the CMOD form factor and, if (almost) all of the available IO are inputs **might** be suitable. In general , I don't consider these to be suitable as general purpose 'components' in a system design for a number of reasons. The Terasic DE0 Nano is at a similar price and more robust. It has a better header design for most of the things that I want to do. You'll need to work this out. In terms of the Artix devices, as long as you don't use a MicroBlaze, any size should work. ADC design for a specific application isn't necessarily straight-forward so you are on your own there :). Edited April 29, 2022 by zygot

[zygot]

So I just realized that the ADC part of your project is more difficult than I thought. 32 channels of 12-bit 1 Msps is 32 12 Mbsp stream pins; and that's just for the data. You don't mention if you need coherent sampling which complicates things considerably. Possibly, your converters can output streams of parallel data but you likely need more than one converter. Either way you are likely to run into a shortage of GPIO connector pins and likely not enough clock capable pins if you want to use a really cheap FPGA platform. Again, this is where the PMOD universe isn't amenable to other than simple projects. In principle, 12 Mbsp data streams are within the capability of any Digilent PMOD.

My thought for such a project would be to make a custom PCB tying all of the interfaces to a cheap FPGA board suitable for application as a system component. I don't think that this is part of the market that Digilent wants to serve. There are Digilent boards with FMC connectors, but designing a custom FMC mezzanine card isn't trivial. And these boards aren't something that you'd want to stick onto a larger custom PCB.

I suppose that it could be possible to design a SYZYGY pod that would work on the Eclypse-Z7 board... but that's a lot of work and at a considerable cost premium. Edited April 29, 2022 by zygot

[MarcMVT]

On 4/29/2022 at 3:15 PM, zygot said:

All of this appears to be reasonable. The Arty A7 has a 10/100 Mbps Ethernet PHY so you may have an issue with streaming gap-less data for 32 channels using normal short packets. I'd be thinking of a 1 GbE Ethernet if Ethernet is the required interface. Data pacing requirements are a bug factor in any design. I don't think that there would be an issue processing your data as you describe it in an all HDL design. The Ethernet part might be more complicated than you anticipate, depending on your requirements and experience but is also doable. Usually, people use a soft processor if they need Ethernet because the free IP makes it easier. The problem there is that, for a small device, a MicroBlaze could consume most of the resources. You'd have to prototype a concept for various devices to get a sense. If you are doing point to point Ethernet then this isn't an overly difficult thing to implement in HDL. If you are connecting to a network though switches then it might be a problem. If you can simplify the Ethernet data flow, then things get easier.

If I were to select from platforms for a project like this that I have available I'd be thinking of the MAX10 Development Kit or ( currently MAX10 devices and boards are as plentiful as hen's canines ) or the Mimas A7 ( currently over-priced if available at all ). Both are about $200 and have 1 GbE. There are some inexpensive ZYNQ based boards that might be an option for 1 GbE. ( Actually, I'd probably use the Genesys 2 because it's currently on my work bench; but I'm trying to meet you in the low end budget space as that's where you are starting ). Since the ZYNQ PS has a MAC and 1 GbE you might over estimate the task involved in transferring data over Ethernet. But, from a HW design standpoint it certainly would be simpler. Some Ethernet applications using ZYNQ are trivial like implementing a web server. Other applications not so much. ZYNQ Ethernet is complicated. If you are comfortable creating a custom Linux OS using Yocto or other framework then perhaps this is the best platform to use. It all depends on the Ethernet environment and requirements.

A less elegant approach might be a really cheap Artix board and a Raspberry Pi for Ethernet connectivity. Three wires and a GND could connect the FPGA and RPi using SPI at up to 31.25 MHz data rates. The repetition rate of your data works in favor here. I'd be spending some time trying to figure out how to disconnect the Ethernet part from the FPGA part.. but really this is based on very little information.. and personal preferences.

One nice thing about FPGA development is that you can mock up a prototype design and build it for any device or board in order to get a handle on resource usage, as well as simulate it, before selecting a hardware platform.

[edit] Opps, I forgot about the ADC. One problem with Digilent FPGA boards are those outdated 12-pin PMODs. They aren't very friendly for experimenter's wanting to use a cheap board with custom hardware interfaces. There certainly are mult-channel ADC devices consistent with your requirements available but using the PMODs may be problematic. Not all PMODs on all boards have clock capable pin assignments that might complicate things. Digilent has the CMOD form factor and, if (almost) all of the available IO are inputs **might** be suitable. In general , I don't consider these to be suitable as general purpose 'components' in a system design for a number of reasons. The Terasic DE0 Nano is at a similar price and more robust. It has a better header design for most of the things that I want to do. You'll need to work this out. In terms of the Artix devices, as long as you don't use a MicroBlaze, any size should work. ADC design for a specific application isn't necessarily straight-forward so you are on your own there :).

 

On 4/29/2022 at 5:07 PM, zygot said:

So I just realized that the ADC part of your project is more difficult than I thought. 32 channels of 12-bit 1 Msps is 32 12 Mbsp stream pins; and that's just for the data. You don't mention if you need coherent sampling which complicates things considerably. Possibly, your converters can output streams of parallel data but you likely need more than one converter. Either way you are likely to run into a shortage of GPIO connector pins and likely not enough clock capable pins if you want to use a really cheap FPGA platform. Again, this is where the PMOD universe isn't amenable to other than simple projects. In principle, 12 Mbsp data streams are within the capability of any Digilent PMOD.

My thought for such a project would be to make a custom PCB tying all of the interfaces to a cheap FPGA board suitable for application as a system component. I don't think that this is part of the market that Digilent wants to serve. There are Digilent boards with FMC connectors, but designing a custom FMC mezzanine card isn't trivial. And these boards aren't something that you'd want to stick onto a larger custom PCB.

I suppose that it could be possible to design a SYZYGY pod that would work on the Eclypse-Z7 board... but that's a lot of work and at a considerable cost premium.

Hello zygot,

thank you very much for your detailed answer. I know the 384 MBit/s of Data to the FPGA seem a lot. But is this stream really a big problem, since it is "only" 12 MBit/s per chanel? I thought about connecting the ADC with 1 wire per chanel with a common ground. Also the FPGA has to filter most of the input data stream as there is only about 34 µs of valid data every 150-200 µs at the maximal allowed particle concentration. After the FPGA did it's "calculations" the remaining set of data to be sent to the post processing PC will be rater small. Since I'm only looking for the peak of each channel for a single particle the data stream of the ethernet connect should be less than 3,84 MBit/s. As each particle produces only 384 Bit every 100 µs at the highest particle concentration.

 

So I thought that the FPGA could have a threshold on every chanel and if this threshold on any of the chanels gets exceeded the FPGA will create a vector with 34 elements for each chanel. Afterwards find the maxima and add those to an array of e.g. 32x100 elements. After the Array is filled, the dataset (38,4 kBit) gets sent via Ethernet to the post processing PC or maybe an RPi (I'm not sure about that already). 

I'm by far no expert in computer science, but I thought this problem could be solved with a rather slim solution after first talks with more experienced colleages.

 

 

[zygot]

I'm pretty sure that just about any low end FPGA can do the processing. Data storage requirements would, on the face of it appear to be minimal. Any FPGA platform with a 1 GbE interface can send the processed data to wherever it's going, though there might be details to work out. The issue is that 48,000,000 bytes of data transfer during the active phased of ADC sampling that is making this complicated.

What are you thinking of as an ADC that has 16(32) serial data output pins ( one per ADC channel ) and a continuous 1 Ms Fs? I can't think of any. There are some nice 1 Ms multi-channel ADC devices out there ( though you have to read the datasheet very carefully to make sure that this is the continuous rate at which channels can be sampled and transferred out of the device ). I don't know of any 32 channel ADC devices like that. And then you have to consider if all of the channels need to be samples simultaneously. If you need to use more ADC devices, then you will need more control/status pins.

The larger concern is the FPGA platform IO. Assuming that you don't want to select new hardware for the 32 channel version of your project, you'll need 32 GPIO plus as many control and status pins to control the ADC device(s). This is where the cheaper Digilent boards will let you down. In fact, I can't think of a cheap FPGA board with and Ethernet port and 40-50 GPIO tied to connectors in a convenient arrangement. The Terasic DE0 Nano would work but doesn't have and Ethernet PHY. The Mimas A7 might work though the GPIO isn't all that great for tying everything together with a custom PCB.

Since Digilent likes to use MicroBlaze, even it's cheap boards tend to have an Ethernet PHY; this is great. Unfortunately, as I mentioned before, these boards serve a different purpose than as a platform for your kind of project, because they are meant to connected to 1 or more PMOD add-on boards, each with only 8 GPIO, and randomly placed clock capable pins if any at all.

If it were currently possible to find an MAX10 devices available for sale I'd recommend designing your own FPGA/ADC platform using something available in a flatpack package. Unfortunately, these are on my current, and lengthy, list of things that money can't buy. Edited May 2, 2022 by zygot

[zygot]

I realize that this might sound, on the face of it, like a stupid idea but I'm used to making do with what's available.

If I were doing this kind of project, the first choice would be to make my own custom FPGA / ADC board. But that's a lot of work and time ( though you can spend a lot of time trying to make unsuitable hardware into something usable as well ), nor is it cheap and simple. My second choice might, depending on options and restrictions, be to use a simple FPGA board, like the DE0 Nano, connected to the converters and an RPi ( which has the Ethernet connectivity already packaged ) using a custom PCB without an FPGA device. You can get about 3 MB/s data transfer between the FPGA and the RPi via one SPI interface. There's also the mysterious Secondary Memory Interface that can do 16+ MB/s though not comfortably. Unfortunately, there's no kernel support for it so you have to accept the consequences of bypassing kernel services. I know.... this still requires a custom PCB, albeit a much more simple one ) and involves 3 boards. But sometimes such a solution is the best one available. Edited May 2, 2022 by zygot

[zygot]

If it were possible to find one anywhere I'd say that the $200 MAX10 development board is a great general purpose Ethernet based data server. It has external DDR, 2 1 GbE PHYs ( though I've never been able to use both simultaneously ) and an HSMC connector with > 70 GPIO. The MAX10 device is nice because it has internal FLASH for configuration so you can program it, wire it up to whatever, put it in a box and use it as an appliance. The MAX10 is billed as a CPLD but realistically acts like a reduced performance FPGA. FPGA vendor development boards were never intended to be used this way... but for quick on-off projects why not?

I don't know why Xilinx got out of this area of the programmable market. Maybe AMD will have other ideas? Edited May 3, 2022 by zygot