Video Pipeline: OV7670-ZYBO

[Haris]

Greetings everyone,

This is the first ever post of a beginner who has set on the path to learn embedded systems. Please forgive me if haven't followed rules of posting. I took the embedded system plunge few weeks back. Bought a strong laptop, a Zybo board, OV7670 camera, installed Vivado. Read online tutorials like 'blinking LEDs' and 'HDMI-to-VGA out'  (and the others ones in Zynq Book) to get myself accustomed with Vivado etc. [Abbreviations in the text: PS= Processing System, PL= Programmable Logic]

I have been visiting a blog lately and have found that quite helpful. A couple of weeks back I started this project mentioned on the same blog (http://lauri.võsandi.com/hdl/zynq/xilinx-vdma.html). What I'm doing is a bit simpler version of this as I omitted some part of the design which I thought wasn't required, I'll mention about that later. Pretty much like a mixture with another project involving test pattern generator (http://lauri.võsandi.com/hdl/zynq/xilinx-video-capture.html) My aim in this project is to get the stream from OV7670 camera, take it through PL (AXI VDMA IPs) to the PS, and then view the stream inside some window within Xillinux (linux). I feel doing so will ensure a bit of learning in both the PL and PS.

Following is my progress along with doubts I have so far:

1- I have pasted the picture of my block design (called VDMA_Trial). After playing hit and trial, I managed to go around all the initial errors and successfully generated the bitstream. Which was a big relief. If you compare my block design with the one mentioned on the link I pasted above, you'll notice mine has lesser number of IP blocks. I did not need the RGB, HSYNC and VSYNC outputs from the PL so I omitted that part and only focused on taking the camera stream to the PS. Do you think this makes sense? Or do I need to have the complete IP blocks even if I don't wish to see the stream on HDMI (or VGA)?

2- I'm a normal computer user who has used Windows PCs most part of the life. So I don't have much idea of CLI based Linux. After reading from online resources, I booted Xillinux image (downloaded from xillybus.com) on the Zybo through SD card. Since I wish to watch the stream from the camera inside the GUI Xillinux. From a bit of reading I got to know about something called V4L2, which I'm still trying to figure how to install on Xillinux. Now assuming the design in point 1 is fine, can I simple paste the bitstream of the project in the SD card along with the Xillinux boot files (there are some other files in SD card too for Xillinux: devicetree, ulmage and xillydemo.bit). Will the hardware design in the PL activate and start writing the video stream information to the PS memory (DDR) when I boot up the Zybo with this SD card? If not, what step must I follow? Do I need to launch SDK to write some code to tell PS what to do? But I will already have the Xillinux OS running on PS, am I correct when I say I'll have to write and compile some code in Xillinux to tell PS to fetch the stream which is being written to PS memory by VDMA (from PL).

3- While reading different stuff, I came across this thing related to V4L2 and OV7670 (http://www.cs.fsu.edu/~baker/devices/lxr/source/2.6.31.13/linux/drivers/media/video/ov7670.c). To a layman like me, it looked like the code to set up OV7670 camera using V4L2 driver. Will I need to compile and run this code in Xillinux to watch the stream? I tried, but it just did not compile.

My apologies for making it long and prob'ly silly. Just started traversing a steep learning curve, will take time to learn. I really look forward to seeing your enlightening responses. If there's any more info you need please let me know (in easy English ).

 

Regards,

Haris.

[D@n]

Greetings Haris!  Looks like you have a fun project ahead of you.

While I'm no expert, I thought I might hazard an answer to some of your questions:

1. If you don't need logic, then by all means get rid of it!  That's one less thing that can break, so the sooner it's gone, the better.

Just be aware: I've been bitten more than once after getting rid of logic I thought I didn't need only to find out, later, that I really needed it.  I'm also pretty sure I'm not the only one whose been bitten by that bug.

2. You have no experience with Linux?  Then there's a fun learning experience ahead of you.  I've used Multics, then Unix, and finally Linux just about all my life.  I think you'll like it.  The Command Line Interface (CLI) takes a bit to learn, but I personally enjoy not having to interrupt what I'm doing to take my hand off the keyboard and move the mouse.  I tend to work faster if I can keep my hands on the keyboard, and the CLI helps me do that.  If you get into a struggle, relax--it is very well documented.  You shouldn't have any problems.

As for simply pasting the boot files onto the SD Card ... I tried that once.  I made lots of changes to the kernel and the device tree, then pasted my results to the card (a BeagleBone Black) to watch it boot.  In my case, I was not only unsuccessful but I also couldn't tell why not.  (The kernel just hung on bootup, right after the bootloader was given the command to boot the kernel.  It was as though I messed up one of the files, but could never tell which.)  Be aware that if things don't work, you may have a ... fascinating debugging experience in front of you.  (I still need to return to my experience.)

Coming back to Linux, most things don't start automagically.  You will actually need to do some work to get that to happen.  In other words, just because you have V4L2 installed and in your kernel on boot up, that doesn't mean that it is going to show up on the screen automagically.  You'll still need to program that.  Realistically, from your description, it sounds like that's what you want though: a program that maps a windows contents onto a memory area which your PL can write to and that your video driver can read from.  You'll need to work through whatever graphical interface you are using (Gnome, KDE, GTK+, etc.) in order to make sure you've got that mapping done right.

While there might be an appropriate demo program within V4L2 that will work for you, my two cent wager is that you will need to compile and run some code for Xillinux.  You might be able to compile it on  your desktop, or you might choose to compile it from within your embedded platform, but you will need your secret sauce somewhere in this process telling the CPU how to do that graphical mapping.

3. As for the source code you found, it looks like you have some work cut out for you!  If it actually is the camera controller that you want and need, you might find that you can rebuild it for your hardware to get it to work for you.  Be aware, though, that you will probably still want something to interface with your programmable logic.  That will need to be a part of whatever driver you write or use as well. 

Have fun, and please post back when you are successful!

Dan

[Haris]

Thank you very much for your insightful and elaborated reply Dan. All your suggestions mean a lot as they have the motivating power to keep me busy in learning while not stopping just because things wouldn't work 'auto-magically'.

 

[JColvin]

Hi Haris,

I personally don't have the experience to help you out, but I do know that one of our Forum members, hamster, posted a project on his website using the Zedboard (the zynq 7020 rather than the Zybo's zynq 7010) and the OV7670 here. That being said, I don't know how much change would be involved with code migration (if you wanted to do that), but I imagine that project would be a good reference point for you if you need it.

Thanks,
JColvin

[Haris]

Thank you JColvin for the response. I'm aware of the forum member and their project you mentioned. In fact, I'm grateful to @hamster as he has been helpful in guiding me before on doing a similar project with Zybo, which involved directing camera stream to VGA output. I did that project before starting with the project I'm currently stuck in. I was successfully able to do that project.

[Mohs]

Hello Haris,

I am new to ZYBO and interested in developing a real-time video processing project on ZYBO. I want to interface my GoPro Hero (720p) camera with ZYBO and display its output to VGA using VDMA,, VTC, digilent HDMI sink and VGA display IP's.

I started by working on the project mentioned in this ( http://lauri.xn--vsandi-pxa.com/hdl/zynq/xilinx-test-pattern-generator.html ) post. The project displays a pattern generated from TEST PATTERN GENERATION ip. The block diagram is attached for your review. The problem is that the project is successfully synthesized but no pattern is displayed on the monitor. 

As you have mentioned that you have implemented a project which involved directing camera stream to VGA output.

your help on this regard will be highly appreciated. 

On 6/6/2016 at 8:17 AM, Haris said:

Thank you JColvin for the response. I'm aware of the forum member and their project you mentioned. In fact, I'm grateful to @hamster as he has been helpful in guiding me before on doing a similar project with Zybo, which involved directing camera stream to VGA output. I did that project before starting with the project I'm currently stuck in. I was successfully able to do that project.

 

[Haris]

On 26/07/2016 at 9:50 PM, Mohs said:

Hello Haris,

I am new to ZYBO and interested in developing a real-time video processing project on ZYBO. I want to interface my GoPro Hero (720p) camera with ZYBO and display its output to VGA using VDMA,, VTC, digilent HDMI sink and VGA display IP's.

I started by working on the project mentioned in this ( http://lauri.xn--vsandi-pxa.com/hdl/zynq/xilinx-test-pattern-generator.html ) post. The project displays a pattern generated from TEST PATTERN GENERATION ip. The block diagram is attached for your review. The problem is that the project is successfully synthesized but no pattern is displayed on the monitor. 

As you have mentioned that you have implemented a project which involved directing camera stream to VGA output.

your help on this regard will be highly appreciated. 

 

Hi Mohs, thanks for writing. I'm sorry for responding late as I was overseas for vacation. I'll take a look and respond to you. : )