Qt on Zyboz7-20 via LX FrameBuffer to HDMI monitor

[Musko]

Finally working!

Brief description:
On right monitor is ssh session from my devel PC to Zyboz7-20 where I start Qt applications (framebuffer and sysinfo on attached picture).
Front monitor is connected to ZyboZ7-20 HDMI output port. Monitor resolution is SXGA (1280x1024@60fps).

FPGA:
- Build with Vivado 2016.4
- Data path for HDMI output:
/dev/fb0 DDR image buffer --> Zynq AXI HP port --> AXI Protocol Converter IP (AXI3 to AXI4) --> VDMA IP (mm2s only) --> AXI4 Stream to Video IP --> Digilent RGB to DVI IP --> HDMI connector
Video control signals are from Video Timing Controller IP (1280x1024p, Pixel clock is 108MHz).

On SD card:
- Boot image containing: FSBL, U-boot (Xilinx git tag xilinx-v2017.3), LX 4.6 kernel (configured and build from Xilinx git tag xilinx-v2016.4) and Buildroot-2017.08.1
- Modified Simple FrameBuffer driver.
- Xilinx DMA driver.
- My custom driver to control FPGA modules (VTC, Xilinx Performance monitor and some others I have in design).
- Cross compiled Qt-4.8.6 examples (analogclock, framebuffer) and Qwt-6.1.3 examples (sysinfo, cpuplot).

TODOs:
- Simple FrameBuffer driver does not starts Xilinx DMA driver transfers, so I have to configure VDMA IP registers manually.
But this is good enough for my first run and proof of concept. I will switch to and continue with DRM device driver.
- Inputs (mouse and keyboard) to control Qt application.

 

[jpeyron]

Hi @Musko,

Welcome to the Forums! Looks like a fun project. Thank you for sharing what you did! If you are able/want to share your project we have a project vault for complete community projects!

cheers,

Jon

[Musko]

Thank you for welcome.

Today was my first successful run of Qt GUI application on my ZyboZ7-20 after months of digging through the whole stack (FPGA - LX kernel - User Land).

Currently I am trying to make frambuffer driver to start moving data via xilinx_dma driver. At the moment I have to program VDMA IP registers (MM2S_StartAdresses, MM2S_STRIDE, MM2S_HSIZE and MM2S_VSIZE) manually. There is something not yet ok in framebuffer driver and binding it with xilinx_dma driver. I will try with DRM driver (ADI or Xilinx as start point) since it is more decent technology as framebuffer.

Once the project is in good shape I will consider adding it to Digilent project vault.

[sbobrowicz]

The Zybo Z7-20 HDMI output works with the Xilinx DRM driver, as long as you use the Digilent Linux fork (It includes some patches that are necessary):

https://github.com/Digilent/linux-digilent

You also need to have the output pipeline and device tree setup a certain way, you can reference this petalinux project to see how:

https://reference.digilentinc.com/reference/software/petalinux/start

[Musko]

Thank you for info.

In the meantime I managed to make it work with ADI AXI HDMI DRM driver by modifying it and combining it with Digilent encoder code.

I use/build all involved drivers as loadable (insmod/rmmod) modules, so I was able to trace down all needed changes.

Now it works perfectly with Xilinx VDMA with all other features (Plug-able EDID monitor detect, handling DPMS modes, ...).

My GUI needs on ZyboZy-20 are satisfied with Qt and frame buffer (no need to extend to X infrastructure).

Attached is device tree.

zyboz7-20.devicetree.zip

[sbobrowicz]

So Qt can be used without X, that's interesting... Could you post the rootfs_config from your petalinux project (it should be in project-spec/config/). It might be helpful to some stuff I'm working on. 

Were you following any Xilinx or Embedded Linux guides or did you bake this yourself?

Edit: Also, did you need to patch the Xilinx drivers or Digilent encoder?

[Musko]

I don't use Petalinux, never used it and also not familiar with its config/build files and procedures. I don't know what info Petalinux project-spec/config/ file contains.

I build my own FSBL (from Vivado exported HDF file), U-Boot (from Xilinx git tag xilinx-v2017.3), LX kernel (from Xilinx git tag xilinx-v2016.4) and Buildroot (buildroot-2017.08.1) and than pack them to SD card (~16MBytes all together).

I am attaching my LX kernel .config file (How my LX kernel is configured?).

For video I don't use Xilinx DRM but modified ADI AXI HDMI DRM driver with (quite a lot) different encoder/connector part. The only Xilinx driver involved in my video driver stack is DMA driver (for VDMA IP core).

I looked at Xilnx DRM driver first, but it was overkill (planes, re-samplers, format converters, display port, vtc re-config, ...) for what I need.

Not using Xilinx DRM driver also gives me freedom on implementation of my FPGA video pipeline, e.g. my VTC in FPGA is hard configured to 1280x1024@60fps.

All I need for my embedded video is a framebuffer for Qt application on fixed monitor resolution (SXGA 1280x1024@60fps). I don't need xterm on my embedded Zynq.

I use Qt-4.8.6 and Qwt-6.1.3 cross-compiled as static libraries (to ease Qt config on target).

I start Qt application with -qws option which than use framebuffer device instead of X server.

 

lx_kernel.config

[sbobrowicz]

Awesome, thanks, that is all I needed . I was mostly curious about what you had built into your root file system, and it sounds like all you need is the Qt/Qwt libraries. Didn't know they could target a framebuffer device directly. 

FYI, petalinux is basically just a wrapper for yocto that helps glue together a vivado project to your Linux system. It does things like auto device tree generation (based on what is found in your Vivado IPI block diagram) and automatic configuration of u-boot. It also provides a nice little KConfig menu for tossing in a bunch of user space libraries and utilities (Including X and Qt), which is the config file I was asking for. It spits out an image.ub (kernel, device-tree and ramdisk all in one) and BOOT.bin for you, and also supports things like TFTP boot, second partition of SD card for persistent Root file system, etc.. It's free too.

If your build process is working I wouldn't bother messing with success, but I found petalinux helpful to streamline the "Build Vivado Project"->"Build Linux Components"->"Boot Linux system" process myself.  

[cj01]

Hello @Musko, I'm currently trying to set up a video design on the Zybo z7-20 with Yocto. Did you choose a random address in DDR for the framebuffer. Also did you work with a dma proxy driver where the transfer is managed through ioctl calls or did you go the userspace route using /dev/mem to set the control registers?

Any help would be very much appreciated.

Cleo

[Musko]

Hi,

For sw part I use Xilinx DMA driver (interface to VDMA IP core) and modified ADI AXI HDMI DRM driver for exposing frame buffer device to GUI sw (e.g. Qt).  You can see driver bindings in above attached zyboz7-20.devicetree-1.zip (pl.dtsi).

All video memory transfers to FPGA are managed by this two drivers.

[cj01]

@Musko Thanks for the quick reply. I am not using GUI but I will look at the DRM driver to see how it interacts with the VDMA driver. I have not yet been able to write a functional kernel driver to use the Xilinx VDMA driver.

[Musko]

I have ported the above also to RedPitaya.

I had to do a bit of HW there too.

[cj01]

@Musko Thanks for the heads up