FMC clock input to Genesys2

[tido]

I am working with a custom-made FMC (HPC) that attaches to a Genesys2 board with VADJ=3.3V. Among various signals, I have two LVDS_25 clock inputs on the pairs L25/K25 (FMC pins K4/K5) and F12/E13 (FMC pins K28/K29):

set_property -dict {PACKAGE_PIN L25 IOSTANDARD LVDS_25}  [get_ports clk0_p];   #IO_L12P_T1_MRCC_AD5P_15 Sch=fmc_clk_p[2]
set_property -dict {PACKAGE_PIN K25 IOSTANDARD LVDS_25}  [get_ports clk0_n];   #IO_L12N_T1_MRCC_AD5N_15 Sch=fmc_clk_n[2]
create_clock -period 9.523 -name clk0_105mhz [get_ports clk0_p];

set_property -dict {PACKAGE_PIN F12 IOSTANDARD LVDS_25} [get_ports clk1_p]; #IO_L14P_T2_SRCC_18 Sch=fmc_hb_p[06]
set_property -dict {PACKAGE_PIN E13 IOSTANDARD LVDS_25} [get_ports clk1_n]; #IO_L14N_T2_SRCC_18 Sch=fmc_hb_n[06]
create_clock -period 9.523 -name clk1_105mhz [get_ports clk1_p];

To my understanding, both pin pairs are clock-capable and Vivado synthesizes/implements without issues. The HB bank is powered by an external voltage of 3.3V from the FMC card.

For sanity checking, I pass both clocks through IBUFGDSs...

wire w_clk0_ibufg;
IBUFGDS clk0_ibufgds_inst (.I(clk0_p), .IB(clk0_n), .O(w_clk0_ibufg));
wire w_clk1_ibufg;
IBUFGDS clk1_ibufgds_inst (.I(clk1_p), .IB(clk1_n), .O(w_clk1_ibufg));

...divide the clock and power a couple of LEDs to confirm signal availability:

reg [25:0] r_clk0div = 0;
reg [25:0] r_clk1div = 0;
assign led[0] = r_clk0div[25];
assign led[1] = r_clk1div[25];
always @(posedge w_clk0_ibufg) begin
  r_clk0div <= r_clk0div + 1;
end
always @(posedge w_clk1_ibufg) begin
  r_clk1div <= r_clk1div + 1;
end

To my surprise, I see only led[0] blinking (with irregularities), but not led[1]. I see a LVDS signal on the FMC connector pins of both clock inputs. The signals are somewhat distorted (oscilloscope struggles to trigger at the proper period) and led[0] blinks irregularly at times.

Due to reasons, the FMC is connected to the Genesys2 board by an 250mm cable [1]. My guess is that the additional connector introduces reflections and the cable attenuates the clock signals to a degree that the FPGA does not detect a LVDS signal. Is it really this "simple", or am I missing something?

[1] https://www.mouser.ca/ProductDetail/200-HDR-169468-01

Edited May 26, 2022 by tido
add details

[zygot]

The cable between your Genesys2 and the FMC mezzanine board is not a great idea, even if it has great signal integrity characteristics. (NOTE: I say this even though I've successfully used a fairly long PCIe cable to connect an FPGA boards with 4-lane and 8-lane PCIe Gen2 interface to PCs and SBCs)

The Genesys2 doesn't provide LVDS termination for any signal pairs. This of course allows for flexibility in using the pins. Proper termination is part of all differential signalling standards. Ideally, differential receivers should have the termination as close to the input pins as possible. Putting termination on an FMC mezzanine card for LVDS signals driven by the card isn't ideal but is a necessary evil for general purpose hardware. Inserting an additional 250 mm, assuming that your card does indeed have the termination, between the terminator and the FPGA pins might well be a problem. This is something that can be simulated.

Be sure to read the AMD/Xilinx IO and clocking reference manuals before trying out hardware, or designing FMC mezzanine cards.

[tido]

Thank you zygot. It is really helpful that you confirm my concerns regarding the cable! I have joined a project mid-way and have not designed the original FMC. There are several things that I feel have not been taken into account before... hence no opposition to any sort of changes on my end. 😉 The cable is not there to stay, but was a workaround to swap pin pairs and use the available hardware sample for as much testing as possible.

The LVDS lines are driven by FIN1108 [1] fairly close to the FMC connector. I take it, that no additional termination would make sense on the end of the FMC card, right?

I find that a lack of information on PCB and cable materials (insert loss, return loss) makes simulation a challenge in this case. I have several years of experience in EM simulation for high-speed interconnects and am able to come up with models for the segments. I am certain that I would be able to extract data to illustrate the detrimental effect of the cable. However, this doesn't give me much new information to work with. In other words, I already know that I do not want the cable between FMC card and Genesys2. 🙃

[1] https://www.onsemi.com/pdf/datasheet/fin1108-d.pdf

Edited May 27, 2022 by tido

[zygot]

LVDS requires a (typically for FPGA IOSTANDARDs ) 100 ohm resistor between the _p and_n pair, ideally as close to the terminus ( receiver pins ) as possible. If this doesn't appear on your FMC mezzanine card then you must add it because it doesn't appear on the Genesys2. Without the terminator the signals aren't really LVDS. As I've mentioned previously, terminators do sometimes appear on mezzanine boards because general purpose carrier boards usually can't include them. Not the best arrangement but usually acceptable.

If you can ditch the cable, then do that. The HPC FMC connector has a LOT of pins. I have no experience trying to use a cable with FMC. I have created a custom PCB to connect FMC carrier boards together. This is rife with danger as it's easy to have shorts between pins at the connectors. For such an arrangement no voltage carrying pins should be connected and a lot of double checking is involved to avoid catastrophic outcomes. Anyway, I can't assert for certain that the cable is your problem, but if your clock signals are properly terminated that's the next priority to check into.

Do read through ug471_7Series_SelectIO.pdf and ug472_7Series_Clocking.pdf ( to understand limitation with the SRCC clock inputs ). All FPGA devices have clocking regions and some devices are much more limiting than others. You will also find, in the SelectIO reference a guide to proper termination for all AMD/Xilinx supported IOSTANDARDs.

One last thought. There's no 3.3V LVDS IOSTANDARD so make sure that the IO banks receiving your clocks are set to Vccio = 2.5V. On the Genesis2 this is determined by jumpers on JP6. The Genesys2 reference manual has a good write-up about the power supplies. On devices like FPGA, that support a wide range of logic standards, it's necessary to limit certain standards to specific IO bank power supply voltages. There are plenty of 3.3V LVDS logic devices that are not compatible with direct connection to FPGA pins without adding additional passive components. Now that I think about it, this is the 2nd thing to check once you've found the terminator resistors.

One (more) thing: ds182_Kintex_7_Data_Sheet.pdf is where you can find the AC and DC specifications for IO. This might help with your simulation analysis.

Last (last, last) comment. Anytime you do a new design using FPGA board IO DO check pin locations in the constraints files against the schematic. Always. Edited May 27, 2022 by zygot

[zygot]

Tired of writing the word last. I looked at the FIN1108 datasheet. It's a 3.3V device and not ( probably as I haven't looked into the details ) directly compatible with programmable logic differential IOSTANDARDs. That doesn't mean that you can't use it with the Genesis2; it just means that you will need to add conditioning to make it compatible with the Kintex LVDS_25 IOSTANDARD. AMD/Xilinx does have application notes on how to do things like that. I suppose that this is a good time to learn good old fashioned digital design basics... if you haven't already, or forgotten how to. Edited May 27, 2022 by zygot

[tido]

5 hours ago, zygot said:

There's no 3.3V LVDS IOSTANDARD so make sure that the IO banks receiving your clocks are set to Vccio = 2.5V. On the Genesis2 this is determined by jumpers on JP6. The Genesys2 reference manual has a good write-up about the power supplies. On devices like FPGA, that support a wide range of logic standards, it's necessary to limit certain standards to specific IO bank power supply voltages. There are plenty of 3.3V LVDS logic devices that are not compatible with direct connection to FPGA pins without adding additional passive components. Now that I think about it, this is the 2nd thing to check once you've found the terminator resistors.

ACK. This is indeed a problem when wanting to transmit from the Genesys2 to a FMC. To my understanding, using LVDS_25 is permitted for a Genesys2's Kintex7 at Vccio=2.5V as well as Vccio=3.3V, if it is used as an input pin, i.e. receive, cf. https://support.xilinx.com/s/article/43989?language=en_US.

Thank you for the extensive advice. I will be sure to consult all of it!

Edited May 27, 2022 by tido
add link to source at Xilinx

[zygot]

From ug471_7Series_SelectIO:

The LVDS I/O standard is only available in the HP I/O banks. It requires a V CCO to be
powered at 1.8V for outputs and for inputs when the optional internal differential
termination is implemented (DIFF_TERM = TRUE).

The LVDS_25 I/O standard is only available in the HR I/O banks. It requires a V CCO to be
powered at 2.5V for outputs and for inputs when the optional internal differential
termination is implemented (DIFF_TERM = TRUE).

It is acceptable to have differential inputs such as LVDS and LVDS_25 in I/O banks that are
powered at voltage levels other than the nominal voltages required for the outputs of those
standards (1.8V for LVDS outputs, and 2.5V for LVDS_25 outputs). However, these criteria
must be met:
• The optional internal differential termination is not used (DIFF_TERM = FALSE,
which is the default value).
• The differential signals at the input pins meet the V IN requirements in the
Recommended Operating Conditions table of the specific device family data sheet.
• The differential signals at the input pins meet the V IDIFF (min) requirements in the
corresponding LVDS or LVDS_25 DC specifications tables of the specific device family
data sheet.
• For HR I/O banks in bidirectional configuration, internal differential termination is
always used.
One way to accomplish the above criteria is to use an external circuit that both AC-couples
and DC-biases the input signals. Figure 1-72 shows an example circuit for providing an
AC-coupled and DC-biased circuit for a differential clock input. R DIFF provides the 100Ω
differential receiver termination because the internal DIFF_TERM is set to FALSE. To
maximize the input noise margin, all R BIAS resistors should be the same value, essentially
creating a V ICM level of V CCO /2. Resistors in the 10k–100KΩ range are recommended. The
typical values for the AC coupling capacitors C AC are in the range of 100 nF. All
components should be placed physically close to the FPGA inputs.

So, my reading of this is that first you need to determine whether your clock inputs are on an HP IO bank or an HR IO bank. If on an HR IO bank then you have the option of using the FPGA internal termination, but only for Vccio = 2.5V. If you don't use the internal termination then you are free to use Vccio = 3/3V... with the proper external  termination, as long as you don't exceed the DC specifications.



I had forgotten that Kintex HR IO banks had the option of selecting internal differential termination. So, thanks for making me take my own advice :)...

Anyway, the analysis is in the details, that is the respective datasheets for the FPGA device and an external circuit. The last thing anyone wants is a flaky clock clocking logic.



When connecting external devices to an FPGA you have to leave the FPGA development mindset for a while and put on your digital design hat to make sure that you are not violating any specifications. It's often easy to arrive at a wrong conclusion by referencing one article and failing to do the necessary analysis covering all the details. Sometimes the official literature appears to make the details confusing.

[edit] Clearly, using on-chip termination is the ideal way to implement a differential receiver in an FPGA; if it's an available option. There's a constraint for that, or you can use the Implementation view in Vivado to select this option. Try it, it might improve your results ( assuming that there isn't another external termination extant ). You still need to go through the digital design analysis to make sure that you aren't exceeding input voltage range specifications.

 
Edited May 27, 2022 by zygot

[tido]

4 hours ago, zygot said:

I looked at the FIN1108 datasheet. It's a 3.3V device and not ( probably as I haven't looked into the details ) directly compatible with programmable logic differential IOSTANDARDs.

Following up on this for the sake of completeness and in case anyone else runs into this: Xilinx' flow chart [1] illustrates the requirement for AC coupling and DC rebiasing. In essence, you will need to rebias if the absolute maximum output voltage from the external circuit could be higher than what the FPGA input can handle.

In case of the FIN1108, this means common mode voltage + 50% of differential voltage swing = 1.375V + 450mV = 1.825V.
On the other hand, the Kintex7 HR input pins can handle VCCO+0.55V = 3.3V+0.55V = 3.85V [2]. The flow chart is more restrictive and requests VCCO+0.2V=3.5V. Either way, this is higher than the FIN1108's output, so no AC coupling is required. In addition, the supply voltage of the FIN1108 is recommended to be set to VCC=3.3-3.6V. Given that I am supplying 3.3V, this does not exceed the Kintex7's input rating either.

That said, an external termination as shown in Fig 1-70 of Xilinx SelectIO Resource Guide [3] is still worthwhile to be used. (Thanks pointing this out, zygot!) Notably, these are not in place on the FMC that I am dealing with. I am beginning to think that this could be the actual issue for the flaky clock that I witness.

The cable might make things worse, but could be indifferent for two reasons:

• The cable does not act as a fully developed transmission line here: I am running the LVDS near 100MHz. i.e. free-space wavelength/2=1.5m or approx. 75cm in the cable. Hence, the cable is 1/6 of a wavelength in length. This is good as it prevent reflections from building up.
• The fact that the cable's phase delay is less than a full period means that it does not attenuate the signal much, if at all. Hence, reflections from the FPGA input pin return to and impact the transmitter.

The latter aspect leads me to consider the missing termination to be the dominant cause for the observed misbehaviour.

[1] https://support.xilinx.com/s/article/43989?language=en_US
[2] https://docs.xilinx.com/v/u/en-US/ds182_Kintex_7_Data_Sheet
[3] https://docs.xilinx.com/v/u/en-US/ug471_7Series_SelectIO

[zygot]

There are a couple of things to consider here.

•     Absolute maximum/minimum voltages for inputs. This includes things like overshoot and undershoot for ill-conditioned signals. This is somewhat more complicated for differential signalling.
•     recommended maximum/minimum voltages for inputs in order to correctly recognize a correct logic level; again a bit more complicated for differential signals.

Table 1 of the Kintex datasheet states that Vin must be between -.40V and Vcco+.55V. There are 3 notes stating caveats with respect to those numbers.

Table 4 provides some additional guidance for HR IO banks:

Maximum AC Voltage Overshoot = Vcco+.55V for 100% of UI within operating temperatures

Maximum AC Voltage Undershoot = -.40V for 100% of UI within operating temperature. If the undershoot is -.55V the duration must be limited to 11% of UI.

Table 12 provides the LVDS_25 specification. It only applies to Vcco = 2.5V. The datasheet doesn't have information about using LVDS_25 inputs with Vcco = 3.3V

Vidiff must be between 100 and 600 mV. Typical value is 350 mV.

Vicm must be between 300 mV and 1.5V. Typical value is 1.2V

The FIN1108-D only specifies DC characteristics for Ta=25 C. The maximum output offset voltage Vos is between 1.125V and 1.375V and Vod is between 250 mV and 450 mV. So, if there is no overshoot the maximum voltage out of the driver is 1.6V.

As for the Xilinx article that you reference I'm not 100% sure what TxVOCM and TxVOD reference. I'm assuming that this relates to the FIN1108-D Vos and Vod values respectively. If so, The worst case Vos seems to be in an acceptable range for Vidiff and LVDS_25 at Vcco=2.5V. Also the Vod of the driver appears to be in range for Vicm. So, if I have this right, the FIN1108-D would appear to be compatible with the LVDS_25 IOSTANDARD given Vcco = 2.5V. This means that you should be able to use the on-chip termination and Vadj= 2.5V on the Genesys2.

One way or another you need the termination for your LVDS clock inputs and the on-chip termination is the most preferable.

I've written way too many words to arrive at "DIFF_TERM = TRUE" but the exercise was well worth the time.

Edited May 27, 2022 by zygot

[tido]

Once again, thank you so much for the hints and the food for thought, zygot! This is much appreciated.

I, too, assume that TxVOCM and TxVOD in the Xilinx article refer to the Tx=transmitter  VO=voltage output for CM=common mode ("bias") and D=differential. After all, they make statements about what their (Xilinx') hardware is able to handle. Furthermore, this seems reasonable due to the way they define the limit on the input voltage and the criterion for rebiasing.

I agree that the over-/undershoot conditions and the AC nature of the signal make this more involved than comparing static voltage levels. However, I think those assumptions are reasonable given the "low" speed at which I operate the link: ~100MHz, cf. my earlier posting. Over-/undershoot can only happen when Tx and Rx are separated by a transmission line, i.e. when you have a substantial phase delay between the two. The given configuration (cable and operating frequency) resembles more of a DC than high-speed digital.. 😉

Looking at the datasheet again, I have to correct myself. The information on VCCO=3.3V is given in Table 1 right below the line that you refer to: the upper limit for VIN is 2.625V. Strictly speaking, this puts the device at risk of being damaged if the FIN1108 would output its maximum 3.3...3.6V. However, the nominal maximum output is --as you say-- only 1.6V with up to 1.825V still being covered by its specification. So no damage under practical (and reasonably to assume) operating conditions.

Last but not least, I was able to tweak the design and run part of it with the DIFF_TERM=TRUE. I have yet to confirm full function, but: the clock seems quite stable. (I compare the input clock with an internally generated clock of the same frequency.) It looks like the cable is not the culprit, but rather it was the missing termination.

[zygot]

I'm not quite in agreement with your transmission line analysis. Back in the dark days before programmable logic I had no problem creating overshoot and undershoot using common LSI/MSI logic on much shorter signal paths. Advanced Shottkey logic was particularly good at this, and at producing EMI which, for commercial products, needs to be constrained. I've re-worked many a PCB design to meet EMC requirements... Today, simulation is automated and even integrated into the PCB layout tools.

I view almost any signal path as a transmission line. There are two considerations here. One is discontinuities in impedance along the transmission line and the other is bandwidth. If you transmit a clock along a transmission line the duty cycle is not unimportant for the analysis of whether or not your signal will be ill-behaved, but the slew rate is even more important and relates directly to the transmission line bandwidth.

Good to hear that you've gotten past your initial problem though you still have some work to do to verify what you observe with the current termination.

[edit] You wrote that you were using a 250 mm cable and I keep thinking that it's a 250 cm cable... so this alters the comments here somewhat, but the concepts are still valid.. Edited May 28, 2022 by zygot