Arty A7 100 Ethernet to LEDs Circuit

[silantyeved]

Hi,

I have progressed to wiring FSMs on an Arty board and I would like to delve deeper into ethernet and get a first hand understanding of it. As a base project I am thinking of creating a circuit that would blink board LEDs if a packet with certain content is received.

I also see that many existing tutorials use ready Ethernet IP cores, which is not what I'd like to do - I want to recreate ethernet from the ground up. How to go about it?

 

Ed

[zygot]

This is a good project, though perhaps LEDs aren't the best indicator of functionality...

The Ethernet PHY is a great general purpose interface. It's basically a cable modem. If you understand, or want to understand some basic modem concepts it's a great interface to learn about. A 1 GbE PHY can do over 120 MiB/s, sustained data rates each direction in full-duplex operation. Unlike most serialized interfaces, Ethernet can be highly predictable in terms of timing.

The place to start is to read about how Ethernet works. Unlike most popular serial interfaces, specifications and descriptions of the protocol and physical layer are public knowledge and freely available. It's well worth the effort as all FPGA vendors only provide Ethernet support connected to encrypted MAC IP . With UltraScale, ADM/Xilinx further obfuscates and abstracts even basic PHY DDR connectivity making understanding what they do almost impossible. Who wants that?
The Digilent Project Vault is an un-curated, hard to use mishmash of projects with source code, projects that need someone to debug the code, and inappropriate posts that belong somewhere else so finding useful information there is difficult... but.. there are a number of projects with HDL code that might help bootstrap your project. Regardless, of how you proceed, understanding basic serial communications concepts, and in particular Ethernet physical layer operation and basic packets types is essential if you want to know how to tak advantage of this interface. Ethernet is, by design, packet based but you don't have to use standard packets to use it effectively for point to point communication.

Have you seen this? https://forum.digilent.com/topic/16802-ethernet-phy-test-tool/

There are a few such projects of varying usefulness in the Project Vault. There are also HDL Ethernet projects with code on the Internet. Edited November 1, 2022 by zygot

[zygot]

The Ethernet PHY on the Arty is a 10/100 Mbps interface compared to the Nexys Video 1 GbE PHY. They are quite different devices. If you do a search on Ethernet physical layer interfaces you can read about the differences.

I pointed you to that project because, basically, the first hurdle is getting the PHY to talk to your FPGA design. This is not a trivial design project and testing it is challenging. There are two other projects in the vault that might be if interest. One points to a non-existent source location but this one might be of interest as it is also a 10/100 BASE-T design: https://forum.digilent.com/topic/3968-ethernet-udp-echo-server/

I have a theory about FPGA development that suggests that the designer not only needs to develop a competency in logic design, simulation, and the tools, but most importantly a level of competency in knowing how to debug a design. If you can write HDL source and testbench code and simulate the design, that doesn't mean that it will work in hardware. If your hardware isn't doing what you expect it to be doing and have absolutely no idea about what to do next, it's a big problem. If you have sufficient past experience developing debugging strategies and HDL debug IP to help analyze what your code is doing, then you have a chance of making progress. I don't make assumptions about anyone's ability to develop their skills, but this is what I've discovered in my personal journey.

Ethernet is not a trivial topic. Don't expect to read a few tutorials and then be able to sit down and write code that does all possible Ethernet applications. Even if you find a project with HDL source that you can implement on your board there are a lot of not so obvious details to be discovered. 6 months might be a reasonable time period for a person with decent FPGA development skills to make enough progress to get two FPGA boards communicating reliably and exchange data to implement a particular application concept. Implementing a sub-set of Ethernet capabilities that just do point to point communication is a much simpler task; but you still need to understand how the physical layer works and what the basic packet structures are, even if you don't use standard packet types.

The UART is a much easier to grasp interface. It might be worth considering deveoping some level of expertise with that one before jumping into the more exotic interfaces. Edited November 1, 2022 by zygot

[zygot]

6 hours ago, silantyeved said:

I am not sure what's my responsibility to implement and what comes already out of the box. 

That's an easy question to answer. Everything is your responsibility. That's one reason why FPGA vendors make it so hard to use this interface; it pushes inexperienced FPGA developers and students into using soft-core processors and expensive ( in terms of resource utilization ) IP and bus structures that are incompatible with all other FPGA vendors. There was a version of Quartus, a long time ago, that offered a RGMII-GMII IP with HDL source ( script generated but readable ). ISE provided basic PHY interface source that could, with some difficulty, be worked out.

The first step to using an Ethernet PHY is to get the data interface working. GMII is a SDR ( one bit per clock ) interface and appeared on older Digilent boards like the original Geneis and ATLAS. RGMII is a DDR ( 2 biits per clock ) interface and uses half the required data bit pins and is more common on current boards as it saves IO and is easier to route PCB signals. So, the first step is to understand how all of the PHY interfaces work. You can find this information in some PHY device datasheets ( Marvel is notorious for requiring a NDA to get complete datasheets but became popular because Xilinx boards used it ). The Ti PHY data sheets describe the data interface pretty well.

Now, there's a bit if a issue with PHY devices that support multiple data interfaces. The old Marvel 88E1111 PHY on the ATLYS can do GMII, RGMII or SGMII. The bad news is that the device may come of reset configured with the wrong data interface and you might have to reprogram the control registers using the slow MDA interface. The good news is that Digilent board are set up to use the correct data interface at the maximum data rate. Which brings up the second issue. Ethernet PHY devices are backward compatible to support slower data rates. The PHY on your board only supports 10/100. The 88E111 supports 10/100/1000. The same problem of how the device is configured out of reset applied to data rate. 10BASE-T uses a different modulation scheme than 100BASE-T or 1000BASET, and a different clock. Again, Ethernet PHYs on Digilent boards are configured to use the fastest data rate available for that PHY. There's other similar issues like auto-crossover, what data speed the PHY advertises, etc. These all can be changed via the MDA interface.

What I do is make using the Ethernet PHY as simple as possible. No MAC, only use the maximum data rate ( no clock switching ), minimal functionality in terms of standard packets, no fragmentation support, no re-sending bad packets, etc.

So, let me offer a task list to consider to get you started.

The First step is to get your FPGA design to talk to the PHY. For you board this means the data interface. The second Project Vault link that I posted earlier can help with that. You also need a HDL serial MDA interface to configure the PHY registers as you will see below.

• Learn about the different PHY data interfaces and how your PHY is set up.
• Read and write the PHY control and status registers using the MDC and MDI (bi-directional) pins.
• Create a data interface supported by how the PHY is connected on your board. You probably have to learn how to use the DDR IO capabilities of your FPGA. This is not trivial and I'd suggest a side project doing just that to get experience. DDR typically requires timing constraints. In trivial designs you can get away without using timing constraints. Generally, advanced IO features require timing constraints like setup for inputs and delay for outputs relative to a clock. If the PHY clock period is much longer than FPAG routing and IO delays you might get away with pretending that timing constraints aren't needed. For 10BASE-T perhaps. For 1000BASE-T no. The more data bit in the interface the more spread in clock to output between them.
• Once you believe that you have a workable data interface it's time to try and send/receive data. Ethernet PHYs have a loop-back capability that is useful at this point. Whatever data your FPGA application sends to the PHY will be returned. This is a good way to test out just how robust your data interface is. Nothing happens on the cable end of the PHY but the FPGA/PHY data interface can be extensively tested.
• Once you are writing/reading PHY registers and reading the correct loop-back data from your PHY you are ready to connect it to another Ethernet PHY. This could be a uController, another FPGA board, or a PC. The PC is a whole ball of wax as Ethernet is handled by the OS. My PC has 2 Ethernet PHYs and one uses DHCP for Internet use while the other uses a static IP address and is used to communicate with an FPGA board, Raspberry PI, or other development board. It's always a direct cable connection because switches and routers present more complication and require more supporting packet types.

Here's a tip. You don't need standard packets to use an Ethernet PHY, but the PHY is a modem so it needs timing information that needs to be refreshed periodically. This means that you can't just send 1 MB through a PHY. You need to send packets of data of reasonable size, say <16 KB) Every packet has to start with the same synchronization preamble. This identifies the first data bit in the serial stream and keeps the modulation/demodulation working over time. After the preamble you can have all data, header fields and payload data, a CRC field or not... anything that is appropriate for your application. If you FPGA wants to talk to a standard Ethernet node, then your FPGA application will have to support a few standard packets. You don't need a processor to do this. Verilog or VHDL is more than capable.

I learned of of this stuff the hard way. It took me a year or so to find and work out all of the surprises. If you want to proceed with this be prepared to expend some time and energy on it and not expect to be an expert or complete a real world application any time soon. Don't have a goal of connecting your FPGA board to the internet. Windows, Linux and other OSs can't keep my PCs safely connected to the Internet so the chances of my FPGA doing that is pretty  much zero.

 

Edited November 2, 2022 by zygot

[zygot]

4 hours ago, Udayan Mallik said:

Your best bet is to use the Cores provided by the Manufacturer.

Perhaps. It all depends on your work flow, design flow and what you want to do. I'm not sure who or what you are referring to when you mention "Cores provided by the Manufacturer". If Digilent has any RGMII, SGMII to GMII cores I've not run into them. All FPGA vendors that I know of take great pains to tie their Ethernet PHY "cores" to encrypted MAC cores and their proprietary soft-processor IP. In the case of UltraScale, Xilinx obfuscates it's actual PHY implementation and embeds a "virtual" serial management interface with a pass-through to the actual physical PHY. If you want to use their Ethernet cores, you have to implement your own serial management interface and program a couple of internal core registers in order to make them work. What are you going to do when your boss tells you that starting tomorrow you'll be using programmable devices from a different vendor, and you have a week to port your designs?  Why would anyone consider this a best bet?

Sure, if you aren't using your FPGA to do anything else you can feed lots of resources to a MicroBlaze and  have fun creating and debugging a loadable executable with the SDK or Vivis. If all you want to do is be able to rebuild someone else's project or a simple application perhaps this is OK.

None of that works for me. If what you want to do is learn about Ethernet and implement an easy to use point to point full-duplex communication interface that works at a much higher bandwidth than I suggest ignoring everything that FPGA board vendors and FPGA vendors are offering you. With a bit of work you'll end up with something that doesn't have any vendor IP, uses a fraction of the FPGA resources, doesn't have any FPGA software, and ( with a bit of work ) is portable to any FPGA vendor tools. 

Sometimes free stuff is worth the price; often it end up being too expensive and not all that useful.

[silantyeved]

Thanks @zygot

I have taken a look at your project and that info went way over my head. I ideally need much more hand holding here. I am surprised there are no resources providing this information. 

For the time being I am looking into the specs of TI Ehternet PHY that comes with Arty A7 and thinking of the simplest step I can take to get me off the ground. Any further suggestions welcome.

 

Ed

[silantyeved]

Thank you @zygot and your assumption of my skill level is very precise and thanks for managing my expectations.

For starters, I am not aiming as high as getting two FPGA boards to communicate with each other in an application setting. As mentioned before, my aim is more basic. To expand on an earlier example project I am after: an FPGA board receives a UDP packet with certain content in its body and flashes LEDs with certain light according to that content and/or writes something to UART. Currently, I am still not clear where I would start to build such a project from scratch. My understanding of Ethernet PHYs is not much clearer after a few days of attempts to understand how they work and what's more, I am not sure what's my responsibility to implement and what comes already out of the box. 

As far as testing / validation is concerned, I would like to set a realistic standard to work towards. I am no stranger to UDP and tools like Wireshark that can help troubleshoot UDP traffic. However, I have never worked with raw Ethernet. On that topic, what would be a minimal set up to validate that FPGA successfully clocked the sent Ethernet packet? How do I even send a raw ethernet packet?

[Udayan Mallik]

Your best bet is to use the Cores provided by the Manufacturer.