What is the application of eFPGA in embedded 360-degree visual field vision system?

Embedded FPGA (eFPGA) plays an important role in the chip. In order to meet the acquisition and processing of 360-degree video surveillance data outside the vehicle mentioned in Article 6, the use of eFPGA to design related functional chips has obvious advantages. As a company that provides independent FPGA chips and eFPGA IP products at the same time, Achronix can help smart car SoC designers develop and debug related functions on FPGA chips first, and do not need to significantly modify the design after the market enters batch applications. SoC equipped with eFPGA.

Embedded 360 ° field-of-view vision systems with multiple high-resolution cameras have entered various applications, such as automotive sensor fusion, video surveillance, target detection, and motion analysis. In such systems, the video streams of multiple real-time cameras (up to 6) are aggregated frame by frame, corrected for distortion and other image artifacts, adjusted for exposure and white balance, and then dynamically stitched into a 360 ° panoramic view , Output with 4K resolution and 60 fps frame rate, and finally projected onto a spherical coordinate space.

High-resolution fisheye camera lenses currently used for such applications usually have a wide-angle field of view (FOV). One of the biggest bottlenecks of the surround view camera system is: storing / reading and accessing multi-channel camera input data to or from external storage in real time, and then processing it as a single frame. The hardware needs to complete the processing operation between the original sensor data input by the camera and the stitched output video within a frame delay.

High-performance computing platforms have been developing in conjunction with CPUs to use FPGAs to provide specialized hardware acceleration for real-time image processing tasks. This configuration allows the CPU to focus on particularly complex algorithms, where they can quickly switch threads and context, and assign repetitive tasks to an FPGA to act as a configurable hardware accelerator / coprocessor / offload engine. Even if FPGAs and CPUs are used as discrete devices, the system can improve overall efficiency, because these technologies do not conflict, but cooperate like wearing gloves on hands.

For example, the image obtained from a fisheye lens suffers from severe distortion, so the stitching operation generated based on multiple camera videos is a highly computationally intensive task because it is a pixel operation. This stitching requires a lot of real-time image processing and a highly parallel architecture. However, this next-generation application exceeds the performance that FPGAs can continue to achieve, mainly due to the delay in data throughput of the chip. This in turn will affect the overall latency, throughput speed, and performance of the entire system.

Add eFPGA semiconductor intellectual property (IP) that can be embedded with the CPU in one SoC. Compared with an independent FPGA chip plus CPU solution, the embedded FPGA array structure has unique advantages. The main advantage lies in its stronger performance. An eFPGA can be directly connected to an ASIC (without I / O buffers) through a wide parallel interface, providing significantly improved throughput and latency counted in single-digit clock cycles. Low latency is the key to complex real-time image processing, such as correcting fisheye lens distortions.

With Speedcore eFPGA IP, customers can define their logic, memory, and DSP resource requirements, and then Achronix can configure their IP to meet their needs. Lookup tables (LUTs), RAM cell blocks and DSP64 cell blocks can be combined like building blocks to create the best programmable structure for any given application.

In addition to standard logic, embedded memory and DSP unit modules, customers can also define their own function blocks in the Speedcore eFPGA structure. By integrating these custom function modules with traditional building blocks into the logic array structure, optimized functions can be added to reduce the area and improve the performance of the target application, so that the performance of eFPGA can be greatly improved, especially for embedded Vision and image processing algorithms are very effective.

Using custom cell blocks to successfully solve high-performance image processing is a good example. When implementing the You Only Look Once (YOLO), a state-of-the-art, real-time object detection algorithm that uses neural networks, you can optimize The earlier method greatly improves performance. The algorithm relies on a large number of matrix multipliers, and when implemented in FPGA, these matrix multipliers need to be constructed using DSP and RAM modules; the optimal configuration between the DSP and RAM modules required by YOLO, and a typical FPGA Problems will occur in the mismatches found in the array structure. For example, the FPGA array structure may provide an 18 × 27 multiply / accumulate unit block and a 32 × 128 RAM DSP unit block, and the best solution at this time may be a 16 × 8 DSP unit block with 48 × 1024 RAM. By creating custom cell blocks that achieve optimal DSP and RAM module configurations, the resulting Speedcore array structure will use 40% less chip area to achieve the same functionality and achieve higher levels of system performance.

Embedding the FPGA array structure in the SoC provides two additional system-level benefits: Lower power consumption—programmable I / O circuits account for half of the total power consumption of individual FPGA chips, while an eFPGA can communicate with the other in the master SoC The module is directly connected to the internal circuit and does not require a large programmable I / O buffer at all. Lower system cost-Since eFPGA only needs to implement specific functions, the die size of eFPGA is much smaller than that of an equivalent stand-alone FPGA chip. This is because eFPGA no longer requires programmable I / O buffers and unnecessary interface logic.

With the help of ultra-low latency and real-time processing functions, a vision system based on a 360 ° field of view can be effectively implemented. Speedcore eFPGA with customized unit blocks is used in conjunction with a CPU in the same main control SoC, which is very suitable for implementing special functions such as target detection And image recognition, distortion and distortion correction, and finally stitching together the final image. Embedded FPGA array structure in SoC is a natural development process of system integration in ultra-deep submicron era.

If you want to know more, our website has product specifications for embedded system, you can go to ALLICDATA ELECTRONICS LIMITED to get more information