Articles in the VHDL category

• Why FPGA boards are so expensive?

  Published on mar 05 juillet 2016 in VHDL, (Comments)

  When looking for an FPGA board, I was really surprised to see that they are quite expensive. At first, I thought that the FPGA was the most expensive part (it is) and that the rest was just a simple carrier with some connectors on it. It is more or less the case for low-end cards, but I quickly discovered that the price of FPGA dev kits is always about twice the price of the FPGA they sport. So, a 40€ FPGA will be soldered on a 40€ carrier to make a 80€ dev kit, and a 250€ FPGA will be installed on a 250€ carrier to make a 500€ dev kit.

  My question was: why? Why do we pay so much for development kits on which expensive FPGAs are installed (for some definitions of "expensive", there are dev kits out there that cost more than a car). I started to do a bit of research to try to discover where this price comes from. In this blog post, I show all the information I got. If you are only interested in the conclusion, feel free to jump directly to it. In short: the price is plenty justified!

  • Prices and methodology
  • Shiny controllers
    • 512MB DDR3 SDRAM
    • Network controller and HDMI buffer
    • PIC microcontroller
    • OLED screen
    • Audio controller
    • FTDI chips
    • QSPI flash
  • The boring stuff
    • Voltage regulators
    • The board itself
    • Power supply
  • Engineering costs and software
  • Conclusion

  Prices and methodology

  I base my research on the Digilent Nexys Video board, the only FPGA board I own (this allows for some nice pictures). What is also interesting about this board is that its price can be directly compared to other Digilent and non-Digilent boards:

  Board	Price (€ taxes included)
  Digilent Nexys Video (Artix 7 200T)	533.28
  Digilent Nexys 4 DDR (Artix 7 100T)	341.98
  Artix 7 200T + 1GB RAM micromodule	332.01
  Micromodule carrier lite	117.81
  Less minimal micromodule carrier	236.81

  All the prices are obtained from Trenz Electronic, the only place where micromodules are available. The rest of this article will use dollar prices from Digilent and other manufacturers, but we can already observe some relations:

  • On Digilent boards, going from an Artix 100T to 200T adds 191 € to the price, even though the 200T FPGA is only about 70$ more expensive than the 100T FPGA. When looking at ratios, the Nexys Video is 1.6x the price of the Nexys 4 DDR, and the Artix 7 200T is 1.6x the price of the 100T. What is around the FPGA must therefore scale at the same rate as the FPGA itself.
  • The Trenz micromodules are available at a very good price, but the carrier boards seem ridiculously expensive! Paying more than 200€ for an HDMI port, two LEDs and an USB-to-serial chip does not seem justified.

  The price of components are obtained from Digikey (in dollars). When possible, I take single-unit prices. This allows me to have an idea of a price that Digilent may make us pay for the component (instead of what they paid), and this takes into consideration that Nexys boards are not the highest-volume product of the century. FPGA fans are quite rare, and FPGA fans willing to spend 500€ on a card are even rarer.

  Shiny controllers

  Now, let's start with the shiny stuff. In this section, I show the price of some components found on the Nexys Video board (prices from Digikey, obtained on July 4th, 2016). How much do we pay for the OLED display or the audio chip that we may not use? This section answers that.

  512MB DDR3 SDRAM

  

  512MB of 800 Mbps DDR3 RAM is available on the Nexys Video. The exact part used (as seen on the schematics published by Digilent) is a MT41K256M16HA-187E at 8.13 USD. The chip on the picture above looks like a RAM chip (it is quite large, has a ball grid array package and is close to the FPGA) but does not have any meaningful markings. However, it is identified as IC25 on the board, and the schematics say that the RAM chip is soldered at this location.

  Network controller and HDMI buffer

  

  The Ethernet port of the board is controlled by a Realtek RTL8211E PHY at about 2 USD. The HDMI ports are driven by an AD8195 buffer chip at 4.39 USD (sorry, I forgot to take a picture of this chip).

  PIC microcontroller

  

  A PIC microcontroller gives a bit more body to the Nexys Video board and makes its use simpler. Basically, it manages the FPGA. When the board is booting-up, the microcontroller programs the FPGA from an USB drive or SD-card (FAT32 format, selects the first .bin file it finds in the root directory), or let it program itself from the QSPI flash device or a JTAG connection. Once the FPGA is programmed, the microcontroller changes mode and acts as a PS/2 device. We can plug an USB mouse or keyboard to the Nexys Video and use them as PS/2 devices on the FPGA side.

  The microcontroller itself, a PIC24FJ128, costs 4.81 USD, but we have to factor in the development cost of the firmware. This is a price I cannot really evaluate as I don't know the development effort required from Digilent, the number of Nexys boards they sell, and plenty other information.

  OLED screen

  

  The OLED screen is typically something that I didn't need, and I was afraid that it contributed significantly to the price of the board. However, the UG-2832HSWEG04 screen with included controller costs only 3.85 USD. We have not yet found any very expensive part on the board.

  Audio controller

  

  The various jacks available on the Nexys Video board are controlled by an ADAU 1761BCPZ audio controller. This is a fairly complex and powerful chip that is even able to resample signals, route them and do plenty other stuff. This is like a mini-soundcard, programmable over I2C. The chip costs 11 USD, a hefty price compared to other components.

  FTDI chips

  

  Two FTDI chips are used, a FT2232HQ for a fast USB-to-parallel connection (used by the FPGA itself and as a USB-to-JTAG port), and a FT232RQ for a much slower USB-to-serial asynchronous connection (used by the FPGA when a very simple serial interface is needed). The large FT2232HQ chip costs 6.70 USD, the smaller FT232RQ costs 4.50 USD.

  QSPI flash

  

  32 MB of QSPI flash is available on the Nexys Video. Part of it is used to store an FPGA bitstream, and the rest can be used for whatever we want. The flash chip is also very fast, as programming the FPGA from it takes even less time than over USB-to-JTAG (and much less time than from an USB thumb drive). The FL256SA costs 3.86 USD

  The boring stuff

  The previous section talked about chips and devices that sound cool and provide user-friendly features. They are devices we care about, because adding them adds features and removing them removes features. When summing up their cost, we get 49.24 USD, or about 9.8 % of the price of the board (if considering the American 490 USD price).

  Now, my research taught me that the devil is in the details, and here the details consist of power consumption. The FPGA can burn as much as 10W (and become so hot that you cannot touch the heatsink anymore), other devices (especially the RAM, the FTDI chips and the microcontroller) also consume several watts together, and everything need a very high quality power supply. For instance, the FPGA core needs between 0.980 and 1.050 volts, at up to 8 amps! Producing such a stable voltage at such high currents is very challenging. Moreover, a dozen other voltages are required (1.0V, 1.2V, 1.3V, 1.5V, 3.3V, 5.0V among others).

  Voltage regulators

  

  All the voltages required on the Nexys Video board are produced by a succession of ADP voltage regulators. They are chained or run in parallel, and need to be very efficient. The input voltage (from the power brick) is 12V, and 1V must be produced at 8 amps. A simple approach that dissipates the "excess voltage" as heat would not work, as about 88W would be needed for that apprach (losing 11V at 8A). This is more power than consumed by a high-end Intel CPU!

  Here is a table of all the high-quality switching power regulators found on the Nexys Video board

  Reference	Cost (USD)
  ADP5052ACPZ-R7	8.58
  ADP2384ACPZN	4.63
  ADG804YRM	3.36
  ADP2370ACPZ-5.0-R7	3.73
  ADR127BUJZ	6.04
  ADP123AUJZ	1.13
  ADP123AUJZ	1.13
  ADP123AUJZ	1.13
  Total	29.73

  To that, we have to add high-quality and large capacitors, and 1% tolerance resistors. I did not find the exact reference of the resistors, capacitors and inductances used, but it seems that they are worth at least 10 USD.

  The board itself

  We tend to forget that, but printing PCBs is quite expensive. Moreover, we are not talking about a simple one-sided PCB printed at a low-cost factory. As seen on the picture below, there are many traces, many vias that are really close to each other, and routing likely requires the use of several PCB layers. To give an idea, the default Power Analysis configuration in Vivado assumes that 12 to 15 PCB layers are used. This is absolutely huge!

  

  I tried to take a picture of the edge of the Nexys Video board, but we reach here the limits of my image acquisition apparatus (a 35x zoom Canon SX30-IS with some close-up add-on lenses):

  

  I cannot really extract much information from the above image as I'm not (at all) a PCB designer. For instance, I don't know if each darker line indicates a separation between layers. I assume that the lighter portion of the image comes from when the PCB was cut, but I don't really know. For what I see, there may be around 8 layers in this PCB.

  After a very quick research, it seems that an 8-layers board the size of the Nexys Video would cost about 30 USD in 500 unit batches. But as with software costs, only Digilent knows how much the PCB costs.

  Power supply

  The power supply shipped with the Nexys Video board is able to output at most 36W at 12V, 3A. One day, I let it plugged in the wall overnight, with the board shut down (so, no load on the brick). The next morning, the brick was completely cool. I became curious and loaded a very power-hungry design on the Nexys Video board. The FPGA became really hot (it reached 73 °C, 163 °F), the various voltage regulators slightly warm, the RAM quite warm too. And the brick was still completely cool.

  

  It seems that Digilent really paid attention to the power delivery system of the Nexys Video board, spending about 40 USD on on-board voltage regulators, and using a very high quality power supply that remains cool whatever happens. To give you an idea, my laptop consumes about 9W at idle, and its power brick (from Dell) is a very nice hand warmer. When the Nexys Video consumes the same amount of power, its power supply remains completely cool.

  Based on what can be seen on the above picture, I have found a very similar (if not identical) power supply on Amazon for 15 USD.

  Engineering costs and software

  Here is a very small section in which I point out that designing a board does not simply consist of buying and soldering components. The components must be carefully chosen, the PCB must be place and routed, the microcontroller firmware has to be developed, along with Vivado programming drivers (and the Digilent Adept suite that can also be used to program the FPGA). Designing such a board may require several years, several failures and adjustments, and represents a large investment for a very small user base.

  Here, I talk about Digilent, but the same is true for other brands. In fact, there is no bargain FPGA board on the market. Nobody managed to produce an Artix 7 200T board at 250$ (the consumer price of the FPGA plus some headroom). I also know some people that decided to build their boards themselves, but they finally took several years to do so and spent more than the price of an already-existing board.

  Conclusion

  Here is a table summarizing the costs listed in this blog post

  Description	Reference	Cost (USD)
  FPGA	XC7A200T-1SBG484C	218.75
  512 MB DDR3-800 SDRAM	MT41K256M16HA-187E	8.13
  Ethernet PHY	RTL8211E	2
  Board microcontroller	PIC24FJ128	4.81
  OLED display	UG-2832HSWEG04	3.85
  Audio in/out chip	ADAU 1761BCPZ	11
  HDMI buffer	AD8195	4.39
  FTDI USB-to-parallel	FT2232HQ	6.70
  FTDI USB-to-serial	FT232RQ	4.50
  32 MB QSPI flash	FL256SA	3.86
  Total shiny stuff		49.24
  12V to 3.3V, 1V, 1.8V, 1.5V	ADP5052ACPZ-R7	8.58
  12V to VADJ	ADP2384ACPZN	4.63
  Choose VADJ	ADG804YRM	3.36
  12V to 5V	ADP2370ACPZ-5.0-R7	3.73
  3.3V to 1.2V	ADP123AUJZ	1.13
  3.3V to 1.25V	ADR127BUJZ	6.04
  3.3V to 1.0V	ADP123AUJZ	1.13
  3.3V to 1.8V	ADP123AUJZ	1.13
  Capacitors and inductances		10
  Power supply		15
  Total power delivery		54.73
  Printed Circuit Board		30
  Grand total		352.72

  This blog post evaluates the price of the components on the Nexys Video to 352.72 dollars (which is a very, very, very rough estimate). This explains about 70% of the price of the board (490 USD on the Digilent website), the rest being engineering costs, return on investment, software and maintenance costs, etc.

  What is interesting to see is that the shiny stuff represents only a very small portion of the costs (less than 10%). A large portion covers the power supply, which was a very big surprise to me. For what I know, larger FPGAs consume more power than smaller ones (and we also tend to want more shiny stuff around expensive FPGAs). This may explain why the ratio between the price of the FPGA and the price of the board remains constant: the beefier the FPGA, the most expensive the power delivery system.

  Also, a larger FPGA has a larger package, with more pins, which requires more effort when making the PCB (routing, metal layers, etc). This, for me, also explains why FPGA dev boards are so expensive.

  But are they? When looking at the parts list (that does not count all the resistors, capacitors, buttons, LEDs, connectors and screws available on the board, nor the packaging, human costs, etc), I get the feeling that I paid less than I should have (especially since the Nexys Video is on sale on Trenz Electronics). At the end, all those FPGA boards from all those brands do a very nice integration job, and feel finished, polished, well built (the PCB is sturdy and feels luxurious), and some of them even have cases and fans!

• FT2232H USB to AXI master

  Published on dim 26 juin 2016 in VHDL, (Comments)

  A couple of days ago, I presented a small VHDL project that allows to exchange data with an FPGA over a fast USB link. The goal of that post was to present some difficult-to-find information about the FT2232H chip, and a simple project that allows to benchmark the resulting USB connection (results show that 14 MB/s are possible using that chip).

  In this post, I go a bit further and present an USB to AXI master bridge that I developed to test various AXI slaves on the Digilent Nexys Video board. This project can be compared to the Xilinx JTAG AXI master IP, but has the advantage of being simpler to use (using a small and portable C tool instead of Vivado commands).

  • Background
    • AXI
    • Xilinx Vivado IP
  • Protocol
  • VHDL implementation
    • USB-to-AXI IP
    • Test Design
  • Software tool
    • Upload
    • Download

  Background

  Last post introduced the Nexys Video board and what features it has, and presented the FT2232H USB-CDC chip that allows a microcontroller or FPGA to easily communicate with a computer over USB. In this section, I give more details about the AXI bus, used in the industry to connect various devices on a SoC, and the Xilinx Vivado IDE used in IP mode.

  AXI

  AXI is a bus protocol developed by ARM, that allows devices to communicate using addresses and data. At the most basic level, an AXI bus is a link between one master and one slave. The master initiates transactions (read/write data at a specific address), the slave responds to them and provides data. A typical AXI slave is a memory controller, that takes addresses and data to read/write to memory and performs these operations. The most common AXI master is a microprocessor, that initiates memory transactions according to the program is executes.

  On paper, the AXI bus seems limited, as it only describes one master and one slave. However, the devices themselves can be much more interesting than a simple master or slave. A device may have several master ports (instruction and data ports, or two independent ports that allow to bridge two AXI networks), several slave ports (a memory controller taking instructions from different masters), or a mix of master and slave ports (a powerful AXI router that redirects master requests to the proper slaves depending on their addresses, with arbitration if different masters want to access the same slave at the same time). Other AXI devices include clock domain crossing devices (a fast master is connected to a slow slave, for instance), bus width adapters (a master reads/writes 32-bit chunks of data while the slave uses a 128-bit bus, which requires several master transactions to be merged before being sent to the slave), etc.

  The following figure shows an example of a set of AXI devices that have been connected in order to provide an interesting system: a fast Microblaze processor is connected to a memory cache, connected to a memory controller; a slow Microblaze processor (running at half the speed for instance) is connected to clock adapters then to the memory cache.

  

  Xilinx Vivado IP

  Logic circuit development can be split in two parts: implementing the interesting stuff (and their testbenches), and wiring everything together. This last operation requires much copy/pasting, is tedious, and can introduce subtle errors in the design, for instance mis-matched clock periods, difficulties to know whether "data_to_send" goes from or to the device currently being wired, etc.

  Most FPGA or ASIC IDE providers have solutions to this problem, and they nearly all consist of allowing the user to graphically wire boxes. Each piece of the design is a box with input and output ports, and lines can be drawn between the ports. This produces a very nice schematic and allows to quickly see the entire design. Once this wiring is done, a VHDL or Verilog file is generated for use with standard synthesis and simulation tools.

  In Xilinx Vivado, schematic entry requires the use of the IP Integrator perspective. Xilinx heavily focuses on this IP Integrator and provides plenty of components ready to be used. In fact, most of their example designs now consist of integrating a couple of components with a Microblaze processor, so that people don't have to enter VHDL code anymore.

  Once you have created your project (a standard RTL project), the IP Integrator can be opened by clicking on IP Integrator » Create Block Design in the left panel of Vivado:

  

  This opens a blank sheet on which you can add components (called IP, for Intellectual Property I think) and wire everything together. The tool is very powerful and really tries to prevent you from making errors. For instance, each port has a type and additional information, so that Vivado can detect when you connect ports from different clock domains together. It also checks that your AXI parameters are correct (bus widths match, for instance).

  If you right-click on the blank sheet, a menu allows you to add some IP provided by Xilinx (or by you, you can create your own IP even if I will not describe that in this blog post), or to add any of your VHDL or Verilog modules (Add Module..., your file will appear as a box with input and output ports). Some Xilinx IP propose nice automations. For instance, adding a Microblaze microprocessor, then clicking on "Run component automation" in the green bar that appears, will add plenty of other components to that you get a complete and functional Microblaze SoC, with memory, debug and some AXI infrastructure.

  Because you can add your own RTL components in IP Integrator, you can wire up your design in this tool without having to create IP packages or use any Xilinx IP. However, I recommend that you look at some Xilinx IPs (more details about them can be found in IP Catalog, in the sidebar). There are complex stuff, like video, image encoders and network stacks, but also very handy basic things like memory caches, multipliers and dividers, AXI-to-nearly-anything bridges, etc.

  Protocol

  The goal of this project is to design an AXI master that issues transaction based on an USB connection in the simplest way possible. The AXI protocol is quite complex and supports many features, like burst transfers (sending the address once, then several data packets), partial writes, cache coherency signals, memory protection (the master says whether it is in privileged or unprivileged mode), etc. However, I don't implement any of these features, only the strict minimum for testing. Burst transfers are not really needed as USB is much slower than even the slowest memory device.

  The USB protocol is as-is:

  • The computer sends an address encoded as 4 bytes, little endian. The MSB (bit 31) is set to 1 when writing, to 0 when reading.
  • When writing, the computer sends a data packet, N bytes, with N depending on the width of the AXI bus (4 for a 32-bit bus, 16 for a 128-bit bus, etc)
  • When reading, the computer reads N bytes, that will arrive when memory has been read

  No error management is used, the protocol is the absolute minimum required for testing AXI slaves and memories.

  Because the FTDI chip has read and write buffers, the protocol can be slightly optimized for free. When writing, the computer can send a bunch of addresses and data packets at once (every data packet must follow its address). When reading, the computer can send a bunch of addresses, then read a bunch of data packets. Beware that the addresses sent to the FTDI chip must fit on its read buffer (32 work, 64 is unreliable). If too many addresses are sent, the buffer may become full, which may block the computer and prevent it from reading data. The computer waits for the address buffer to become empty, the FPGA waits for the computer to read data, and we have a deadlock.

  VHDL implementation

  I have implemented the USB-to-AXI master component as an IP, that can be imported and used in Vivado. You can download the IP here. The hdl/ft2232h_mem_v0__1.vhd file contains the actual component and can be used under the GNU LGPLv3 license. Other files are generated by Vivado and I don't really know what their licenses are. I advise you to extract the VHDL file and re-build an IP if you need to use it for something else than experiments.

  USB-to-AXI IP

  The AXI bus contains many signals and four independent communication channels: read addresses (from master to slave), read data (from slave to master), write addresses (from master to slave) and write data (from master to slave too). An AXI master can send as many read/write addresses on the address channels as long as the slave is ready. When the master wants to read data, it sets its ready flag and reads from the read data channel. The master can also write data on the write channel as long as the slave is ready. This is a bit weird, but as long as transactions are kept in order, you can send a bunch of data, then a bunch of write addresses (at which point the slave will perform the write operations). The only thing you must keep in mind is to send read addresses before reading data (seems legit).

  The VHDL component is a large state machine that covers two state chains:

  • Get (write) address from USB, send write address to AXI slave, wait for AXI slave to accept the address, get data from USB, send data to the slave, wait for data to be accepted, return to idle
  • Get (read) address from USB, send read address to slave, wait for slave, accept data, wait for data to arrive, send data to USB, return to idle

  There are many states and nearly no parallelism, but this is to keep things simple (for instance, the component could have read data from USB while the write address is sent to the slave instead of waiting for the slave to accept the address before reading data from the USB bus).

  1 2 3 4 5

  type usb_state_t is ( usb_get_address, usb_address, usb_send_waddr, usb_send_raddr, usb_wait_waddr, usb_wait_raddr, usb_get_data, usb_data, usb_send_data, usb_wait_data, usb_accept_data, usb_wait_for_data, usb_read_data, usb_read_data_wait, usb_after_read );

  Another thing to pay attention to is that the bus to the FTDI chip is much narrower (8-bit) than the AXI bus (up to 512-bit). This requires some additional state variables that allow to read/write data packets byte-by-byte before sending them on the AXI bus:

  1 2 3 4 5 6 7 8 9 10 11 12

  -- State of the USB transmission constant addr_bytes : integer := (C_M00_AXI_ADDR_WIDTH/8); constant data_bytes : integer := (C_M00_AXI_DATA_WIDTH/8); signal usb_state : usb_state_t; -- Address/data state signal address_byte : integer range 0 to addr_bytes-1; -- Index of the address byte being read signal data_read_byte : integer range 0 to data_bytes-1; -- Index of the data byte being read signal data_write_byte : integer range 0 to data_bytes-1; -- Index of the data byte being read signal address_buffer : std_logic_vector(m00_axi_awaddr'range); signal data_read_buffer : std_logic_vector(m00_axi_wdata'range); signal data_write_buffer : std_logic_vector(m00_axi_rdata'range); signal enable_write : std_logic;

  Before presenting the complete state machine, I also want to show that most of the AXI output ports can be set to a fixed value. For instance, the component always generates bursts of length 1, does not bother with cache coherency or security, ignores many things (like error signals), etc:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

  m00_axi_awid <= (others => '0'); m00_axi_awlen <= (others => '0'); m00_axi_awsize <= burst_size(data_bytes); m00_axi_awburst <= (others => '0'); m00_axi_awlock <= '0'; m00_axi_awcache <= (others => '0'); m00_axi_awprot <= (others => '0'); m00_axi_awqos <= (others => '0'); m00_axi_awuser <= (others => '0'); m00_axi_wstrb <= (others => '1'); m00_axi_wlast <= '1'; m00_axi_wuser <= (others => '0'); m00_axi_bready <= '1'; m00_axi_arid <= (others => '0'); m00_axi_arlen <= (others => '0'); m00_axi_arsize <= burst_size(data_bytes); m00_axi_arburst <= (others => '0'); m00_axi_arlock <= '0'; m00_axi_arcache <= (others => '0'); m00_axi_arprot <= (others => '0'); m00_axi_arqos <= (others => '0'); m00_axi_aruser <= (others => '0');

  With burst_size an array that maps widths (in bytes) to burst size values:

  1 2 3 4 5 6 7 8 9 10 11 12 13

  type burst_size_t is array(Natural range <>) of std_logic_vector(m00_axi_awsize'range); constant burst_size : burst_size_t(1 to 128) := ( 1 => "000", 2 => "001", 4 => "010", 8 => "011", 16 => "100", 32 => "101", 64 => "110", 128 => "111", others => "000" );

  Finally, here is the complete state machine. I refer you to the VHDL file linked at the beginning of this section for a list of input/output ports and the complete implementation:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130

  prog_rdn <= '1'; prog_wrn <= '1'; prog_oen <= '0'; prog_d <= (others => 'Z'); case usb_state is when usb_get_address => -- Wait for address bytes if prog_rxen = '0' then address_buffer(address_byte*8 + 7 downto address_byte*8) <= prog_d; enable_write <= prog_d(7); prog_rdn <= '0'; usb_state <= usb_address; end if; when usb_address => -- Read an address byte, and wait for next address bytes -- or data bytes if address_byte = addr_bytes-1 then -- Last byte has been read, send address address_byte <= 0; if enable_write = '1' then -- Write operation, on the write address bus usb_state <= usb_send_waddr; else -- Read operation, on the read bus usb_state <= usb_send_raddr; end if; else -- Wait for next byte address_byte <= address_byte + 1; usb_state <= usb_get_address; end if; when usb_send_waddr => -- Send a write request m00_axi_awaddr <= address_buffer; m00_axi_awvalid <= '1'; usb_state <= usb_wait_waddr; -- Clear the top bit, that is always at one otherwise m00_axi_awaddr(m00_axi_awaddr'high) <= '0'; when usb_wait_waddr => -- Wait for write request to be accepted if m00_axi_awready = '1' then m00_axi_awvalid <= '0'; usb_state <= usb_get_data; end if; when usb_send_raddr => -- Send a read request m00_axi_araddr <= address_buffer; m00_axi_arvalid <= '1'; usb_state <= usb_wait_raddr; when usb_wait_raddr => -- Wait for the read request to be accepted if m00_axi_arready = '1' then m00_axi_arvalid <= '0'; usb_state <= usb_accept_data; end if; when usb_get_data => -- Wait for data bytes if prog_rxen = '0' then data_write_buffer(data_write_byte*8 + 7 downto data_write_byte*8) <= prog_d; prog_rdn <= '0'; usb_state <= usb_data; end if; when usb_data => -- Read a data byte if data_write_byte = data_bytes-1 then -- Last byte has been obtained, send data data_write_byte <= 0; usb_state <= usb_send_data; else -- Wait for next byte data_write_byte <= data_write_byte + 1; usb_state <= usb_get_data; end if; when usb_send_data => -- Send data to be written m00_axi_wdata <= data_write_buffer; m00_axi_wvalid <= '1'; usb_state <= usb_wait_data; when usb_wait_data => -- Wait for data to be accepted if m00_axi_wready = '1' then m00_axi_wvalid <= '0'; usb_state <= usb_get_address; end if; when usb_accept_data => -- Tell the AXI bus that we are ready to read m00_axi_rready <= '1'; usb_state <= usb_wait_for_data; when usb_wait_for_data => -- Wait for data on the AXI bus if m00_axi_rvalid = '1' then m00_axi_rready <= '0'; data_read_buffer <= m00_axi_rdata; prog_oen <= '1'; -- Tell FTDI chip that we will write to it usb_state <= usb_read_data; end if; when usb_read_data => -- Transmit data on the USB connection prog_oen <= '1'; -- Tell FTDI chip that we will write to it if prog_txen = '0' then prog_d <= data_read_buffer(data_read_byte*8 + 7 downto data_read_byte*8); prog_wrn <= '0'; usb_state <= usb_read_data_wait; end if; when usb_read_data_wait => -- Let time to prog_txen to be updated prog_oen <= '1'; if data_read_byte = data_bytes-1 then -- Everything has been read, start a new request prog_oen <= '0'; data_read_byte <= 0; usb_state <= usb_after_read; else -- Read next byte data_read_byte <= data_read_byte + 1; usb_state <= usb_read_data; end if; when usb_after_read => -- Do nothing just after a write, let time to OEN -- to propagate to the FTDI chip usb_state <= usb_get_address; end case;

  Test Design

  The USB-to-AXI IP can be used in any design to test AXI slaves. For my experiments, I have used it with a Memory Interface Generator, that allows the FPGA on the Nexys Video to communicate with the onboard 512MB DDR3 RAM. Before showing the IP diagram that I used, here are some considerations:

  • The FTDI chip produces at 60 Mhz clock, that is used to clock the USB-to-AXI IP. This means that the AXI bus for which the IP is master runs at 60 Mhz
  • The Nexys Video board provides a very stable 100 Mhz reference clock to the FPGA.
  • The Memory Interface Generator needs a 200 Mhz input clock, that must be produced. I decided to use the 100 Mhz reference clock as base clock, not the 60 Mhz one, because memory interfaces are quite sensitive to clock signals (jitter, duty cycle) and I don't think that the primary goal of the FTDI chip is to provide a rock-solid clock.
  • The MIG is very easy to configure if you properly configure your project for your board. Digilent provides board files for all its boards, and these files describe the precise timings and parameters of the onboard DDR RAMs, if any. They also provide the complete pinout of the FPGA, which allows you to add a memory in a single click (running a board automation).
  • The MIG produces a ui_clk, a 100 Mhz clock that it uses to drive its AXI slave port. This means that the USB-to-AXI IP (60 Mhz) and MIG (100 Mhz) run at different speeds. An AXI clock converter is therefore needed.
  • In simple cases, reset is easy to handle, just use a switch (beware that most resets are active low, the switch must be set to 1 for the device to work). However, clock generators are a bit more difficult to handle as we have to wait for their clocks to become stable before releasing reset. In this design, there are two clock generators: the one that produces the 200 Mhz clock and one generator internal to the MIG. We therefore need two reset logics, one that releases its reset when the 200 Mhz clock is stable, one that releases it once the MIG can be used (when it has been calibrated).

  All in all, quite a number of components are required. Here is a large image (I recommend that you right-click on it and see it outside this page) that shows all the components I used and their connections. For my experiments, I've set the AXI bus widths to 128 bits, so that more data can be send/received for each address.

  

  In this image, I send a number of output signals to the LEDs available on the Nexys Video board for debugging purposes. If you do the same, you will be able to see that the MIG needs nearly half a second to complete its calibration, which may contribute significantly to the startup time of a microprocessor-based system.

  Software tool

  An upload/download tool can be downloaded here. Compile this file with:

  1	gcc -O2 -I/usr/include/libftdi1 -o ftdi-axi-master -lftdi1 main.c

  The program can be used to write to memory locations (ftdi-axi-master w filename.dat address) or to read from memory location (ftdi-axi-master r filename.dat address size). Addresses and sizes can be expressed in decimal, octal or hexadecimal form (with the 0x prefix).

  Upload

  When uploading or downloading, the program starts by initializing the FTDI chip in Synchronous FIFO mode, the mode expected by the FPGA and that provides the best performance. My previous blog post explains the process and tells you what to change if you own a board for which the FTDI chip has a different major-minor USB number.

  The actual uploading process consists of sending a list of addresses and data to the FPGA. The data width is configured at the beginning of the C file and must be adapted to your AXI bus width. Because sending packets over USB is slow, large packets should be sent. The program packs 1024 addresses, 1024 data words, then sends to whole over the USB bus. This allows data to be sent at over 14 MB/s (when addresses are counted, we should be over 17 MB/s, or 136 Mbps).

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

  struct element { unsigned int address; char data[WIDTH/8]; }; // Open file FILE *f = fopen(filename, "r"); // Send packets to the FPGA struct element elems[BURST_LEN]; unsigned int i = 0; while (!feof(f)) { // Produce element elems[i].address = address | 0x80000000; // Set "write" flag fread(&elems[i].data, WIDTH/8, 1, f); // If the list of elements is full, send them if (i == BURST_LEN - 1) { ftdi_write_data(ftdi, (unsigned char *)&elems, sizeof(elems)); i = 0; } else { ++i; } // Prepare for next address address += WIDTH / 8; } // Send last elements if (i != 0) { ftdi_write_data(ftdi, (unsigned char *)&elems, i * sizeof(struct element)); }

  Download

  The download process is comparable to the upload process, with the exception that the program only sends addresses (no data), and receives data. We need to be careful while doing that, as the FPGA answers to read addresses one at a time, and will not read following addresses until the answer has been transmitted. We must therefore ensure that the FTDI send buffer is always able to receive data, in order to prevent deadlocks from occuring (if the computer sends too many addresses and waits for them to be consumed, but the FPGA is waiting for its send buffer to become empty and does not read any address).

  Fortunately, in my experiments, it appears that using asynchronous USB transfer works and is not too slow. The program fills an address buffer, sends all of them asynchronously, reads all the data asynchronously, and libftdi ensures that information flows without deadlocks. Doing so allows libftdi to make the most out of the buffers available on the FTDI chip, which reduces USB transfers and allows data to be read at a bit under 1 MB/s. This is much slower than writes but still much faster than RS-232. A better protocol (send one read address and a size, then read everything) would make things much faster, though.

  Here is the code I use:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

  struct ftdi_transfer_control *wr_transfer, *rd_transfer; FILE *f = fopen(filename, "w"); // Send packets to the FPGA unsigned int addresses[BURST_LEN]; char buf[BURST_LEN * WIDTH/8]; unsigned int i = 0; for (int j=0; j<size; j += WIDTH/8) { // Produce address addresses[i] = address; if (i == BURST_LEN - 1) { // Asynchronously send addresses and read data wr_transfer = ftdi_write_data_submit( ftdi, (unsigned char *)&addresses, sizeof(addresses)); rd_transfer = ftdi_read_data_submit( ftdi, (unsigned char *)&buf, sizeof(buf)); // Wait for transfers to complete and write data to file if (wr_transfer && rd_transfer) { ftdi_transfer_data_done(wr_transfer); ftdi_transfer_data_done(rd_transfer); fwrite(&buf, sizeof(buf), 1, f); } i = 0; } else { ++i; } // Prepare for next address address += WIDTH / 8; } // Process last elements if (i != 0) { wr_transfer = ftdi_write_data_submit( ftdi, (unsigned char *)&addresses, i * sizeof(unsigned int)); rd_transfer = ftdi_read_data_submit( ftdi, (unsigned char *)&buf, i * WIDTH/8); if (wr_transfer && rd_transfer) { ftdi_transfer_data_done(wr_transfer); ftdi_transfer_data_done(rd_transfer); fwrite(&buf, i * WIDTH/8, 1, f); } }

  There is a bit of copy/paste in this code, which I don't like, but it works for its purpose.

  Note: Even though I'm able to transfer large files using this program (a 70 MB video file, archives, etc), it sometimes happen that the file read from memory starts with 16 random bytes. If that happens, resetting the board, re-writing and re-reading solves the problem. I don't know where it comes from, probably stray data in the FTDI buffer when writing or reading files.

• Fast USB connection on the Nexys Video using FT2232H

  Published on sam 25 juin 2016 in VHDL, (Comments)

  Over the past years, I developed a strong interest for logic circuit design and VHDL coding, in addition to computer programming and Artificial Intelligence. In this blog post, I present a small project that consists of exchanging data to and from a Digilent Nexys Video FPGA board and a computer using an USB 2.0 link. Instead of implementing a whole USB stack or relying on a slow RS-232 connection, this project uses the FT2232H chip available on the Nexys Video to exchange data at more than 14 MiB/s.

  • The board
  • The FT2232H chip
    • Libftdi
    • Configuring the FT2232H chip
    • Sending and receiving data
  • The FPGA
    • Implementation

  The board

  

  First, a bit of introduction. The Nexys Video is a development board featuring a Xilinx Artix 7 200T FPGA, the largest Artix FPGA (and therefore the largest no-too-expensive Xilinx FPGA). The FT2232H chip is available on many boards (especially from Digilent, but possibly also from other brands), which makes this project quite general. Moreover, I never use any digilent-specific tool or peripheral in this blog post.

  Plenty of other FPGA development boards exist, some of them much less expensive, but I needed the large 200T FPGA for a personal project. The Nexys Video also has very interesting features, like an HDMI sink (for video acquisition), gigabit Ethernet (so, you can acquire video and stream it on the network if you want), DDR3 memory and a nice amount of controllers (Ethernet MAC, audio chip, and a microcontroller that allows the FPGA to be programmed from a USB thumb drive or a SD card, in addition to JTAG-over-USB and QSPI).

  The FT2232H chip

  The FT2232H chip allows a device to send and receive data over a USB connection is many different ways, the chip being highly configurable by writing values in a small EEPROM memory or by configuring it directly over the USB connection. It is most often used as an UART-to-USB bridge, that allows microcontrollers and FPGA to very easily communicate with a computer. However, serial communication with the FT2232H is quite slow, with a top speed at around 1 MiB/s when special care is taken.

  Fortunately, the FT2232H also has a very high-speed mode, that also happens to be quite simple to use. The FPGA or microcontroller communicates with the chip using a 8-bit bus, and does so at 60 Mhz using a clock produced by the device (in Synchronous FIFO mode). It is also possible to have the device provide the clock to the FTDI chip (Asynchronous FIFO mode, as the FTDI chip samples data on fall edges of specific signals), but this is a bit slower as timing is more conservative in this mode.

  Enabling Synchronous FIFO mode is bit difficult as it is a two-step process. First, the FTDI chip must be configured in Asynchronous FIFO mode, using either a special USB command or the EEPROM memory. Fortunately, the Digilent Nexys Video board configures its FTDI chip in Asynchronous FIFO mode by default. The Synchronous mode must be enabled afterwards, and has to be so using a USB command.

  Libftdi

  FTDI chips present themselves on the USB bus as modem devices, using the USB CDC class. On Linux, the FT2232H chip exposes two devices, /dev/ttyUSB0 and /dev/ttyUSB1, one for each channel (the chip allows two devices to be connected to it, a JTAG programmer and a general-purpose connection on the Nexys Video). On Windows, COM1 and COM2 are available once a driver is installed.

  Opening /dev/ttyUSB0 and writing to it allows to communicate with the device connected to the first port of the FT2232H chip using its default mode, Asynchronous FIFO in the case of the Digilent Nexys Video. However, there is now way to enable Synchronous FIFO using only this ttyUSB0 device, we must use libftdi.

  libftdi is a pure user-space library, built on libusb, that allows to detect FTDI chips, configure their ports, send and receive data. This library is very easy to use but has one major problem: it unregisters the USB CDC driver. This means that once a libftdi call is made, the /dev/ttyUSB devices disappear and cannot be used anymore. All the communication with the chip needs to be done using libftdi.

  Configuring the FT2232H chip

  In this section, I present a very small program that opens a FT2232H USB device, configures it in Synchronous FIFO mode, receives a whole bunch of data (for benchmarking purposes), then allows the user to type numbers that will be sent to the device. This is not very useful but is a nice experiment, and I used it to turn on or off LEDs on the Nexys Video.

  First, some includes, that require that libftdi1-devel or libftdi-devel is installed. This library is available in all major Linux distributions and Mac OS ports. It is also available on Windows but I don't have any Windows machine on which to test that.

  1 2 3 4

  #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <ftdi.h>

  Opening an FTDI device is quite easy and requires only a couple of function calls. The ftdi variable points to a context that allows all these functions to store information on the device being used

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

  int ret; struct ftdi_context *ftdi; // Initialize FTDI library ftdi = ftdi_new(); if (!ftdi) { fprintf(stderr, "Cannot initialize libftdi\n"); return EXIT_FAILURE; } // Open device. You will have to change 0x6010 ! ret = ftdi_usb_open(ftdi, 0x0403, 0x6010); if (ret < 0) { fprintf( stderr, "Cannot open FTDI device 0x0403:0x6010: %d (%s)\n", ret, ftdi_get_error_string(ftdi)); ftdi_free(ftdi); return EXIT_FAILURE; }

  The USB device major-minor numbers can be found using lsusb. Find the line talking about your FTDI device, and look at the numbers that look like 0403:6010 or 0403:6001. Each FTDI device has a different number, and some board manufacturers change the number in order to use their own (if they have their own driver for instance).

  Now that the device is open, we can configure it in Synchronous FIFO mode. Because the Digilent board is already in Asynchronous FIFO mode, this is very simple as a simple flag has to be toggled. On other boards, more libftdi calls may be required (one more, I think, the one that puts the chip in Asynchronous FIFO mode). I invite you to consult the very nice libftdi documentation for that.

  1 2 3 4 5

  ret = ftdi_set_bitmode(ftdi, 0xFF, BITMODE_SYNCFF); if (ret != 0) { fprintf(stderr, "Failed to set mode to Synchronous FIFO"); }

  Sending and receiving data

  When the first libftdi call is made, the USB CDC driver is unregistered and the /dev/ttyUSB devices disappear. This means that they cannot be used anymore to easily exchange data with the device, by using cat or dd for instance. From now on, all the communication with the device must go through libftdi. Fortunately, the library exposes file-like functions that are quite simple to use. Here is how I benchmark the read speed of the device:

  1 2 3 4 5 6 7 8 9 10

  // Read 1024 bytes of data unsigned char buf[1024]; for (int i=0; i<32*1024; ++i) { ftdi_read_data(ftdi, buf, sizeof(buf)); } for (int i=0; i<1024; ++i) { printf("%i ", (int)buf[i]); }

  The first for reads 32 MiB of data. The second one displays the last 1024 bytes in integer format, so that I can check that the data is as expected (see next section for how to generate this data on the FPGA). In my experiments, data is read at around 14 MiB/s, which means that the FTDI chip is able to use 112 Mbps of the 480 Mbps USB connection. Not so fast but still much faster than RS-232. The only way to go faster is to use the Gigabit Ethernet port of the board, but this is much more complex to put in place (on the FPGA side). By the way, I highly recommend you to visit hamsterworks.co.nz if you don't know this website yet, this is very interesting especially if you own a Nexys Video or Arty FPGA board.

  In my project, I allow the user to send 10 bytes of data to the FPGA, that will use these bytes to turn LEDs on or off:

  1 2 3 4 5 6 7 8

  unsigned int v; for (int i=0; i<10; ++i) { printf("8-bit value to send to the LEDs (hexadecimal): "); scanf("%x", &v); ftdi_write_data(ftdi, (unsigned char *)&v, 1); }

  That's it for the computer part. Now, the FPGA must be able to communicate with the FTDI chip.

  The FPGA

  The FPGA side of the project is simple and complex at the same time. Simple as the FTDI chip is very easy to use (parallel port, the clock is provided, not too many signals to handle, easy timing). Complex because getting accurate timing information is quite challenging: the FTDI FT2232H datasheet lists timing information on page 41, but the way it is presented is quite misleading.

  On page 41, we see (when writing) that the clock period of the FTDI chip is 16.67 ns, for a clock frequency of 60 Mhz. We also see that the data that we want to write must be on the parallel bus two clock cycles after the chip told us that it accepts data. However, the timing table tells us that the setup time for the data is 16.67ns, a full clock cycle, and no hold time is needed.

  This is a quite uncommon way of expressing time. However, we can understand this table and schema in another way: on clock cycle N, the FPGA can sample TXE, the signal that tells us that data can be sent to the chip. On clock cycle N+1, the FPGA must place the data on the parallel bus and deassert WR, the signal that tells the FTDI chip that data has to be written. On clock cycle N+2, the data can be removed from the bus. Seen from clock cycle N+2, we have a one-clock-cycle setup time and no hold time.

  From what I can see in the datasheet, it appears that the FTDI chip samples and produces signals on falling clock edges, while the FPGA does that on rising edges. This seems very natural but requires that we are careful: the data that the FPGA produces will be sampled 8.335 ns after the rising clock edge, not 16.67 ns. So, don't forget to add an output delay constraint in order to ensure that the FPGA produces data on time.

  Implementation

  The datasheet coupled with the above paragraphs should allow you understand a simple example. In this example, the FPGA sends data whenever possible, producing an ever-repeating sequence of numbers from 0 to 255. This allows to easily check on a computer that no byte is missing or duplicated. When data arrives, the FPGA copies it to the 8 LEDs available on the Nexys Video.

  The interface of the VHDL module is quite simple. The FTDI chip provides the 60 Mhz clock, an input/output parallel bus, and a number of signals. These signals allow to know when data is available or data can be written. The FPGA can say whether it wants to read or write, and what is the direction of the parallel bus. There is also a very nice signal that allows the FPGA to send a wake-up USB packet, so that you FPGA can wake-up your computer if need be.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

  library ieee; use ieee.std_logic_1164.all; use ieee.numeric_std.all; entity ftdi_serial is port ( -- Clock from the FTDI chip prog_clko : in std_logic; -- Wired to a switch in my case. Resets -- are usually not used in FPGAs. reset : in std_logic; -- Data sent to the LEDs and received from the -- FTDI chip led : out std_logic_vector(7 downto 0); prog_d : inout std_logic_vector(7 downto 0); -- Signals, OEN gives the direction of the bus, -- SIWU allows to wake-up the computer. prog_rxen : in std_logic; prog_txen : in std_logic; prog_rdn : out std_logic; prog_wrn : out std_logic; prog_siwun : out std_logic; prog_oen : out std_logic -- '0' when the FTDI chip can write ); end ftdi_serial;

  The implementation is a simple state-machine. Reading needs two states, one that actually reads the data (and tells the FTDI chip that data has been read), and one that gives one clock cycle to the FTDI chip to update prog_rxen. Writing is a bit more complex as three clock cycles are needed: the first one changes the direction of the data bus, the second one places data on the bus now that the FPGA can drive it, and the third cycle simply waits so that the FTDI chip can process the data on the bus and update prog_txen.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

  architecture Behavioral of ftdi_serial is type state_t is (idle, read_wait, do_write, write_wait, write_wait2); -- Counter allows to produce the sequence of bytes -- sent to the computer signal counter : unsigned(7 downto 0); signal state : state_t; begin process(prog_clko) begin if rising_edge(prog_clko) then if reset = '1' then counter <= (others => '0'); prog_d <= (others => 'Z'); led <= "10101010"; state <= idle; prog_rdn <= '1'; prog_wrn <= '1'; prog_siwun <= '1'; prog_oen <= '0'; else prog_rdn <= '1'; prog_wrn <= '1'; prog_oen <= '0'; -- By default, let the FTDI chip -- drive the bus prog_d <= (others => 'Z'); case state is when idle => -- Give priority to reads over writes, so that -- the FPGA immediately responds to user queries. if prog_rxen = '0' then -- Data avilable on the bus (OEN is '0' by default) led <= prog_d; prog_rdn <= '0'; state <= read_wait; elsif prog_txen = '0' then -- Write possible, tell the FTDI chip that we -- will drive the bus prog_oen <= '0'; state <= do_write; end if; when read_wait => -- One cycle delay to let the FTDI chip process -- the write and update prog_rxen state <= idle; when do_write => -- Place data on the bus prog_d <= std_logic_vector(counter); prog_wrn <= '0'; prog_oen <= '0'; counter <= counter + 1; state <= write_wait; when write_wait => -- Delay so that the FTDI chip can process prog_wrn -- and update prog_txen state <= idle; end case; end if; end if; end process; end Behavioral;

  When dealing with an inout port in VHDL, we must not forget to assign it 'Z' before trying to read from it. On an electrical perspective, Z puts the port in high-impedence (input) mode. On a purely language side, the resolution function for std_logic_vector treats Z as unknown and gives priority to other drivers. If one end of a signal sets it to Z and the other end sets it to 1, for instance, then the signal takes a value of 1.

  This implementation is very simple and cannot really be used for real-world things. In the next post, An USB to AXI master interface, I present a small IP (and accompanying program) that allows to read and write to arbitrary addresses over an AXI bus. I use it to store files in the 512 MB DDR3 RAM available on the Nexys Video.

Page 1 / 1