Enabling real-time object detection on low cost FPGAs

Object detection using convolutional neural networks (CNNs) has garnered a lot of interest due to their high performance capability. Yet, the large number of operations and memory fetches to both on-chip and external memory needed for such CNNs result in high latency and power dissipation on resource constrained edge devices, hence impeding their real-time operation from a battery supply. In this paper, a resource and cost efficient hardware accelerator for CNN is implemented on an FPGA. Using an existing metric $$\mathrm{DSP}_\mathrm{efficiency}$$ and a new metric $$\mathrm{Cost}_\mathrm{efficiency}$$ as the primary optimization variables, exploration of algorithms and hardware using a design space exploration tool, called ZigZag, is undertaken. An optimized architecture is implemented on a Xilinx XC7Z035 FPGA and tiny-YOLOv2 is mapped to demonstrate the real-time object detection application. Compared to the state-of-the-art (SotA), the implementation results shows that the hardware achieves the best $$\mathrm{DSP}_\mathrm{efficiency}$$ at 90% and $$\mathrm{Cost}_\mathrm{efficiency}$$ at 0.146.