Zybo Z7 HMDI Demo - the starting point for a video manipulation project

[Boneoh]

Hi to all!

I'm new to FPGA and I'm motivated to learn as much as I can about it. I saw the Zybo Z7 HDMI demo project, I think its awesome!  I just ordered the Zybo Z7 -20 board. I've downloaded the project source and have been reading through it. I realize I've got a lot to learn but I'm patient, persistent, and have time to work on it. 

My goals are

1. Learn this project as best I can to understand the details.

2. Make small changes over time to implement some manipulations to the video. I'm thinking about glitchy processing for video art. Simple things like bit shifting of the image data, quantization of pixel values, etc. 

3. In the longer term I'd like to implement some manipulation functions based on the luma values, things like increasing or decreasing the luma of the video buffer or filtering base on luma, etc.

I already have a few questions.

1. The HDMI Demo seems to never use index 0 of the frame buffer array. This is frameBuf in video_demo.c,  is that used for some special purpose? 

2. My video workflow is all 1080p30, the vga_modes.h file has an entry for 1080p60. My knowledge is very limited, but it seems like I should be able to just change the .freq entry, the remaining parameters for porch etc. are the some. Am I correct?

3. My background is strong in software development, limited in hardware and electronics. How should I approach adding manipulation to the project?  Should I be learning about DSP slices? Approaching it on the software side? 

Boy, I've got a lot to learn! Thanks in advance for any pointers.

Pete 

[Eran Zeavi]

Hi, in vga_modes.h you can find a list of all supported video formats.
I have the same issue as you, I Iam software and embedded software developer, but my knowledge with video is limited.
I am right now looking to downscale the frame rate from 60Hz down to 5 Hz t and still trying to figure it out.
What is your project about ?
 

[Boneoh]

Hi, Eran! I want to learn about HDMI in FPGAs so I can use the frame buffers to manipulate the video in real time. This will be for glitchy video art.

I have a Zybo Z7-20 and am trying to load it up. It took two days on my old PC to setup Vitas + Vivado and update to 2024-1-1. I need to replace the hard drive with an SSD !

It seems that the file "Zybo-Z7-20-HDMI-sw.ide.zip"  for 2024-1-1 is missing the  "Zybo-Z7-20.src" folder. This is in the -10 version of the zip file, the folder contains the source code. I will create a new post specific to this, I'm guessing that the src folder is the same for -20 as the -10 version, but I really don't have a clue.

Missing .src folder?

Edited August 19 by Boneoh
Added link to new topic.

[Boneoh]

I used to 'src' folder from the -10 version and it worked great. The HDMI Demo is up and running.

Quote

I am right now looking to downscale the frame rate from 60Hz down to 5 Hz t and still trying to figure it out.

I think you can add a new entry by editing  the source file that has the video settings, it is file "vga_modes.h" in the "src\display_ctrl" folder. I am trying something similar in a couple days, I want 1080p30. I should be able to add a new entry and change the .freq setting. Then in file "src\video_demo.c" there is function "DemoCRMenu" that will need to be changed to add the new mode to the prompt. Last, in function "DemoChangeRes" it will need to be changed to use the new mode when the user enters the response to the prompt.

Here is a link  that shows the parameters for many video modes. 

[Boneoh]

I've been making progress getting up the learning curve. I've been able to add the LEDs, buttons, and switches to the HDMI Demo project. 

My challenge is how to generate an interrupt from the PL to the PS to handle the framebuffer. I'd like to have the software in PS receive an interrupt to know when to switch framebuffers, so the software will be working in one buffer while the PL is displaying another. 

I've tried experimenting with the code, but I'm too new at this to get a good understanding of what to do. Here is what I've found.

A. The v_tc_in is generating an interrupt when receiving the lock. This works great. The v_tc_in.irq is connected to xlconcat0.In3 and the xlconcat0.dout is connected to processing_system7_0.IRQ_F2P[4:0]      I'm guessing that these are mapped when the function fsInitInterruptController in intc.c is called.  But it's not clear to me what is happening there. The five bits coming out of the xlcontact0.dout are going to bit 4:0 of the PS.  I seem to be missing the critical point here...

B. I see the code in video_demo.c where the "ivt" is setup for both GPIO and VTC. The interrupt id is set to HDMI_IN_VTC_IRPT_ID which was set to XPAR_FABRIC_V_TC_IN_IRQ_INTR. 

C. Code in video_capture.c function named VtcIsr receives the interrupt. 

Getting more specific here:

1. Should I be using an interrupt from v_tc_in to detect a new frame and switch buffers? It seems that v_tc_out is used for the output timing. The only use I can see for the v_tc_in is simply for the 'locked' interrupt. 

2. What interrupt is best to use? The product guide shows an example video on page 54, Figure 3-8. Here the "fsync" values are used by the video pipeline. 

3. What do I need to do to implement this?  a) add a new entry to the ivt. b) add a connection for one or more of the fsync bits to the xlconcat. c) create a handler to receive the interrupt.

Thanks in advance!

Pete 

[Eran Zeavi]

Hi,

There is a new recent issue (about a week ago) with the HDMI demo for zybo-20, after months it has been working for me.
I reinstalled fresh Vivado and HDMI demo , not solving the issue.
Vivado 2024.1 fails synthetising the project due to "ila_pixclk module not found" error showed on the picture.
My guess is that the ILA IP upgrade killed it but not sure 

Any idea ?

Thanks

Eran
 

Edited Thursday at 04:58 PM by Eran Zeavi

[Boneoh]

@Eran Zeavi 

I am running Vivado + Vitas 2024.1.1 but seem to be getting the same messages. 

I am so new to FPGAs that I don't understand the implications of these messages. The demo software does seem to be working for me.

Are you able to run the demo at all? Does anything appear to be broken? Is there anything you would like me to try on my machine? 

I get these critical warning messages in Vivado when generating the synthesis and implementation. 

Edited Thursday at 05:22 PM by Boneoh
Replaced image

[Boneoh]

I just found this and will dig deeper. https://github.com/Digilent/vivado-library/issues/25

 

[Boneoh]

I've been struggling to try to get the PL to PS interrupts added to the HDMI demo. I've added the getting started with baremetal code into the HDMI demo and can turn the LEDs on and off using the buttons and switches.  The goal is to have the VTC in the demo generate an interrupt to the PS. This would use a Frame Sync out from the VTC into the concat and then the IRQ_F2P of the PS. I've struggled with this for a few days and decided to try a more limited test case.

I've changed the Getting Started With Baremetal example to try to test the interrupt with no success. If someone would take a look, I would really appreciate it. 

Thanks!

Pete

GettingStartedBaremetal.ide.zip

[artvvb]

I haven't had much of a chance to look at the HDMI project itself, apologies. The warnings about the ILA should at least be ignorable, as Eran Zeavi pointed out. I've also let the folks who did the last round of demo updates know about the missing .src folder.

1 hour ago, Boneoh said:

I've changed the Getting Started With Baremetal example to try to test the interrupt with no success. If someone would take a look, I would really appreciate it. 

There are some missing API calls - you should be enabling the AXI GPIO interrupts through the xgpio.h header, at a minimum, some calls to XGpio_InterruptEnable and XGpio_GlobalInterruptEnable are missing. Once they're working, the interrupts need to be manually cleared in the handler by the processor as well. If you go poking around in the driver headers and c files, there's a lot of good information. This modified version of your main file works on my machine:

#include "xparameters.h"
#include "xil_printf.h"
#include "xgpio.h"
#include "xil_types.h"
#include "xscugic.h"

// Get device IDs from xparameters.h
#define BTN_ID XPAR_AXI_GPIO_BUTTONS_DEVICE_ID
#define LED_ID XPAR_AXI_GPIO_LED_DEVICE_ID
#define SWT_ID XPAR_AXI_GPIO_SWITCHES_DEVICE_ID

#define BTN_CHANNEL 1
#define LED_CHANNEL 1
#define SWT_CHANNEL 1

#define BTN_MASK 0b1111
#define LED_MASK 0b1111
#define SWT_MASK 0b1111

XScuGic InterruptController;
static XScuGic_Config *GicConfig;

u32 btn_data;

void ExtIrq_Handler(XGpio *InstancePtr)
{
	u32 irq_sts;
	irq_sts = XGpio_InterruptGetStatus(InstancePtr);
    XGpio_InterruptClear(InstancePtr, irq_sts);
	btn_data = XGpio_DiscreteRead(InstancePtr, 1);
    xil_printf("ExtIrq_Handler: %08x\t", irq_sts);
    xil_printf("%08x\r\n", btn_data);
}

int SetUpInterruptSystem(XScuGic *XScuGicInstancePtr)
{
  Xil_ExceptionRegisterHandler(
		  XIL_EXCEPTION_ID_INT,
		  (Xil_ExceptionHandler) XScuGic_InterruptHandler,
		  XScuGicInstancePtr
  );
  Xil_ExceptionEnable();
  return XST_SUCCESS;
}


int interrupt_init(XGpio *InstancePtr)
{
  int Status;

  GicConfig = XScuGic_LookupConfig(XPAR_SCUGIC_0_DEVICE_ID);
  if (NULL == GicConfig) {
    xil_printf("interrupt_init error - XScuGic_LookupConfig\r\n");
    return XST_FAILURE;
  }

  Status = XScuGic_CfgInitialize(&InterruptController, GicConfig, GicConfig->CpuBaseAddress);
  if (Status != XST_SUCCESS) {
	xil_printf("interrupt_init error - XScuGic_CfgInitialize\r\n");
	return XST_FAILURE;
  }

  Status = SetUpInterruptSystem(&InterruptController);
  if (Status != XST_SUCCESS) {
	xil_printf("interrupt_init error - SetUpInterruptSystem\r\n");
	return XST_FAILURE;
  }

  Status = XScuGic_Connect(
		  &InterruptController,
		  XPAR_FABRIC_AXI_GPIO_BUTTONS_IP2INTC_IRPT_INTR,
		  (Xil_ExceptionHandler)ExtIrq_Handler,
		  (void *)InstancePtr
  );
  if (Status != XST_SUCCESS) {
	xil_printf("interrupt_init error - XScuGic_Connect\r\n");
	return XST_FAILURE;
  }

  XScuGic_Enable(
		  &InterruptController,
		  XPAR_FABRIC_AXI_GPIO_BUTTONS_IP2INTC_IRPT_INTR
  );

  return XST_SUCCESS;
}

//void XGpio_InterruptGlobalEnable(XGpio *InstancePtr);
//void XGpio_InterruptGlobalDisable(XGpio *InstancePtr);
//void XGpio_InterruptEnable(XGpio *InstancePtr, u32 Mask);
//void XGpio_InterruptDisable(XGpio *InstancePtr, u32 Mask);
//void XGpio_InterruptClear(XGpio *InstancePtr, u32 Mask);
//u32 XGpio_InterruptGetEnabled(XGpio *InstancePtr);
//u32 XGpio_InterruptGetStatus(XGpio *InstancePtr);

int main() {
	XGpio_Config *cfg_ptr;
	XGpio led_device, btn_device, swt_device;
	u32 data;

	xil_printf("Entered function main\r\n");


	// -------------------------
	// 'Normal' processing
	// -------------------------

	// Initialize LED Device
	cfg_ptr = XGpio_LookupConfig(LED_ID);
	XGpio_CfgInitialize(&led_device, cfg_ptr, cfg_ptr->BaseAddress);

	// Initialize Button Device
	cfg_ptr = XGpio_LookupConfig(BTN_ID);
	XGpio_CfgInitialize(&btn_device, cfg_ptr, cfg_ptr->BaseAddress);

	// Initialize Switch Device
	cfg_ptr = XGpio_LookupConfig(SWT_ID);
	XGpio_CfgInitialize(&swt_device, cfg_ptr, cfg_ptr->BaseAddress);

	// Set Button Tristate
	XGpio_SetDataDirection(&btn_device, BTN_CHANNEL, BTN_MASK);

	// Set Button Tristate
	XGpio_SetDataDirection(&swt_device, SWT_CHANNEL, SWT_MASK);

	// Set Led Tristate
	XGpio_SetDataDirection(&led_device, LED_CHANNEL, 0);

	XGpio_InterruptEnable(&btn_device, 0b1111);
	XGpio_InterruptGlobalEnable(&btn_device);

	// -------------------------
	// Interrupt processing
	// -------------------------

    interrupt_init(&btn_device);

	// -----------------
	// Loop forever
	// -----------------

	while (1) {
		data = btn_data;
		data &= BTN_MASK;
		if (data != 0)
		{
			data &= LED_MASK;
		}
		else
		{
			data = XGpio_DiscreteRead(&swt_device, SWT_CHANNEL);
			data &= SWT_MASK;
			if (data != 0)
			{
				data &= LED_MASK;
			}
			else
			{
				data = 0;
			}
		}

		XGpio_DiscreteWrite(&led_device, LED_CHANNEL, data);
	}
}

Serial prints, ignore the top part, just the bottom five lines represent booting the board, pressing and releasing BTN3, then pressing and releasing BTN0.

[Boneoh]

@artvvb Thank you so much for the quick and detailed reply! I will try this asap and confirm the results. 

Regards,

Pete

[Boneoh]

@artvvb Yay! This works great!

Thanks a  bunch for getting me over this hurdle!

Edit: I've added the switches to the interrupts following your example. I've attached a zip file for the ide.

Regards,

Pete

GettingStartedBaremetal.ide.zip

Edited 5 hours ago by Boneoh
Added .zip file