Design space exploration for efficient hardware implementation of mobile vision transformer on FPGA

Type of work:

Master Thesis

Assignment:

The goal of this work is to implement a mobile vision transformer on FPGA using High-Level Synthesis (HLS). The task is divided into the four main phases:

First: Understanding the vision transformer architecture and optimizing it using quantization-aware training.
Second: Analyzing the model's structure, detecting bottlenecks, and identifying possible cross-layer optimizations that can be exploited for an efficient hardware implementation on FPGA.
Third: Designing a hardware architecture and implementing it using HLS.
Fourth: Implementing the bottleneck components of the hardware architecture on an FPGA board and collecting measurements.

Skills:

• CVivado / Vitis HLS
• C / C++
• Python
• Pytorch / TensorFlow

Supervisor:

V. Rybalkin, M. Ghaffar,  M. Moursi

Student:

Nabih Talaat Ibrahim Saleh

Year:

2023