Zybo and UVC(USB Video Class)

[merflinch]

Hello, everyone!

I am now working on a project which have an action cam connected to HDMI Input and then filtered with HLS Video Library and then i wanted to use the Zybo as a generic webcam to the pc host.

What I wanted to ask is how do I configure my UVC linux kernel so that it receives the video output from my PL that I connect to my PS through AXI Mem Interconnect, and then ready to use as USB Webcam?

Also, can I run an OpenCV apps which generated from Vivado SDSoC and output the video to the uvc?

Any helps and hints appreciated, Thanks!

[sbobrowicz]

Sounds like a very cool project. I especially like the idea because it allows you to get rid of the output video pipeline to VGA, which will free up valuable FPGA resources in the 7010 for your HLS algorithm. You are probably going to run into some throughput issues on the USB 2.0 interface unless you compress or use smaller/slower/lower color depth input modes (640x480@60Hz and 24-bit color already uses 442 mbps of the 480 mbps theoretical limit).

As far as implementation goes, I would recommend getting your input pipeline working in baremetal, and then just using the UIO driver to implement the same code from user space. This means loading the generic UIO driver for the VDMA and VTC cores on your input pipeline. The trick will be allocating your input frame buffer in a way that plays nice with Linux, because it is not really supported to allocate physically contiguous memory from user space. I've had success with pointing an input pipeline directly at an output pipeline (with DRM driver loaded) by mmaping the /dev/fb0 device to get the physical address of a valid framebuffer, but I don't think the same method will work for you because you don't have a DRM output device. If you want to learn about accessing the /dev/fb0 device at a low level and getting access to the physically contiguous framebuffer, I highly recommend this guide: http://betteros.org/tut/graphics1.php .

Here are a couple of hacks I can think of that might work for allocating a framebuffer:

1) create a "dummy" video output device. Not sure exactly how to go about this, but the goal would be to get linux to create a /dev/fb0 device that doesn't really output anywhere, and then use it to allocate a contiguous frame buffer using the methods described in the guide above. A cursory google search suggests the vfb (virtual frame buffer device) might be worth looking into for this.

2) Reserve a chunk of memory in the device tree, and just use that. This seems like it might be easier and cleaner. Check out this documentation: https://www.kernel.org/doc/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt .

Once you have your input video pipeline working and a valid memory region to it at, you will need to point your HLS IP core at it. I'm not sure how this is typically done, but I imagine a UIO driver would probably be easiest to do this as well.

The last step will be to point the output of your HLS pipeline somewhere where the UVC driver can access the data. I honestly have no experience interacting with USB gadget drivers, so you will need to look around the internet on how to do this. If the UVC driver doesn't allow for the allocation of physically contiguous memory buffers, you will need to also reserve a buffer for the output of your HLS pipeline (using the methods described above), and then figure out how to point the UVC driver at that physical address (you should be able to mmap a reserved memory region). 

I just realized that I assumed your pipelines were setup like this:

Input video -> memory ->HLS ->memory -> UVC

But it is possible you are doing the following (which would be much more efficient from an FPGA resource perspective):

Input video -> HLS -> memory -> UVC

If the latter is the case, every thing I described should still work, you will just only need to reserve one memory region that the UVC driver can point to.