# ORB_FPGA

**Repository Path**: fpgaboy/ORB_FPGA

## Basic Information

- **Project Name**: ORB_FPGA
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2020-07-16
- **Last Updated**: 2023-03-27

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

## Introduction
This project implements a ORB feature extractor accelerator on FPGA (on PYNQ-Z2 board). ORB ( **O**riented FAST and **R**otated **B**RIEF) feature is a kind of image feature based on oriented FAST feature and BRIEF descriptor. It's easy to calculate and robust to rotating so it's widely used in embedded computer vision application. For example, it's used in feature matching of some SLAM (**S**imultaneous **L**ocalization **A**nd **M**apping) system, such as [ORB-SLAM](https://github.com/raulmur/ORB_SLAM) and [ORB-SLAM2](https://github.com/raulmur/ORB_SLAM2). SLAM system help robot or other platform to locate themselves and build a map of their surroudings. Here is a brief explanation of how ORB Features are extracted.  
<div align=center>
<img src="./pynq_notebook/data/ORB.jpg" />
</div>  

The extractor will take a gray image and applys a FAST extractor on it. After feature points being found, it will calculate m01 and m10 moment of the feature points and figure out the angles of them. Calculating BRIEF descriptors needs those angles and a gray image which is the original gray image applied a 7x7 gaussian filter.  
![Accelerator](./pynq_notebook/data/accel.jpg)  
## Result  
### 1.Resource Utilizaton (on Pynq-Z2)
| Resource | Utilization | Available | Utilization |
| :------: | :---------: | :-------: | :---------: |
|   LUT    |    35807    |   53200   |    67.31    |
|  LUTRAM  |    1412     |   17400   |    8.11     |
|    FF    |    54895    |  106400   |    51.59    |
| BRAM18K  |     50      |    140    |    35.71    |
|   DSP    |     24      |    220    |    10.91    |

### 2.Performance (testing on images in 640*480)
|   Platform    | Average Delay | Throughoutput |
| :-----------: | :-----------: | :-----------: |
| PS 650mHz -O  |   291.7 ms    |   3.43 FPS    |
| PS 650mHz -O2 |    98.5 ms    |   10.20 FPS   |
|   PL 140mHz   |   17.56 ms    |   56.58 FPS   |

### 3. Result Picture
![](./pynq_notebook/data/res.png)  
Feature points are drawn on the picture.Descriptors can be read in the buffer filled by DMA.
## Quick Start
Run these command lines on your Pynq-Z2 Board (tested on v2.4):
```
sudo pip3 install git+https://github.com/Siudya/ORB_FPGA.git
```
## Contents of each folder
### ip
HLS sources files 
### pynq_arch
Vivado project
### pynq_notebook
Notebook run in Jupyter Notebook
### software_test
A .cpp file that test the same process on CPU 
### test_data
Images for testing
### hw
.bit and .hwh files for PYNQ  

## How to rebuild Vivado project
**vivado 2018.3** is required.  

### Step 1: rebuild HLS IP
Open **Vivado HLS** command terminal and run these commands :  
```
cd <path-to-proj>/ip
vivado_hls -f build_ip.tcl
```  
This should take about half an hour.
### Step 2: rebuild top project
Open **Vivado** command terminal and run these commands :  
```
cd <path-to-proj>/pynq_arch
source pynq_arch.tcl
```