Trevor Vannoy
Advisor: Ross Snider
MS Thesis Defense | April 8 2020
Enabling Rapid Prototyping of Audio Signal Processing Systems using System-on-Chip Field Programmable Gate Arrays
Outline
- FPGA background
- Problems with FPGA development
- Development framework
- Sound effects processor example
- Delay and sum beamformer example
System-on-Chip
Field Programmable
Gate Arrays
a.k.a SoC FPGAs
What's an FPGA?
What's an SoC FPGA?
Why use them?
Why use them?
Parallel
Low latency
Scalable performance
Real-time
signal processing
Why Doesn't Everybody use SoC FPGAs?
Development is Hard
- Low-level
- Hardware and software
- Unfamiliar programming paradigm
Development Framework
Overview
graphical programming, automation
easier, faster, more accessible
Development Flow
Simulink Models
Autogen Flow
Dataplane Configuration
{
"avalon_memorymapped": {
"register": [
{
"data_type": {
"fractional_bits": 7,
"signed": false,
"type": "ufix8_En7",
"width": 8
},
"default_value": 0.5,
"max_value": 1,
"min_value": 0,
"name": "wet_dry_mix",
"reg_num": 2
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"default_value": 0,
"max_value": 1,
"min_value": 0,
"name": "bypass",
"reg_num": 0
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix6",
"width": 6
},
"default_value": 32,
"max_value": 32,
"min_value": 0,
"name": "bits",
"reg_num": 1
}
]
},
"avalon_memorymapped_flag": 1,
"avalon_sink": {
"signal": [
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"name": "avalon_sink_valid"
},
{
"data_type": {
"fractional_bits": 28,
"signed": true,
"type": "sfix32_En28",
"width": 32
},
"name": "avalon_sink_data"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_sink_channel"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_sink_error"
}
]
},
"avalon_sink_flag": 1,
"avalon_source": {
"signal": [
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"name": "avalon_source_valid"
},
{
"data_type": {
"fractional_bits": 28,
"signed": true,
"type": "sfix32_En28",
"width": 32
},
"name": "avalon_source_data"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_source_channel"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_source_error"
}
]
},
"avalon_source_flag": 1,
"clocks": {
"sample_frequency_Hz": 48000,
"sample_period_seconds": 2.0833333333333333e-05,
"system_frequency_Hz": 98304000.0,
"system_period_seconds": 1.0172526041666666e-08
},
"conduit_input": [],
"conduit_input_flag": 0,
"conduit_output": [],
"conduit_output_flag": 0,
"entity": "BC_dataplane",
"linux_device_name": "bitcrusher",
"linux_device_version": "18.0",
"model_abbreviation": "BC",
"model_name": "bitcrusher"
}{
"avalon_memorymapped": {
"register": [
{
"data_type": {
"fractional_bits": 7,
"signed": false,
"type": "ufix8_En7",
"width": 8
},
"default_value": 0.5,
"max_value": 1,
"min_value": 0,
"name": "wet_dry_mix",
"reg_num": 2
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"default_value": 0,
"max_value": 1,
"min_value": 0,
"name": "bypass",
"reg_num": 0
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix6",
"width": 6
},
"default_value": 32,
"max_value": 32,
"min_value": 0,
"name": "bits",
"reg_num": 1
}
]
},
"avalon_memorymapped_flag": 1,
"avalon_sink": {
"signal": [
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"name": "avalon_sink_valid"
},
{
"data_type": {
"fractional_bits": 28,
"signed": true,
"type": "sfix32_En28",
"width": 32
},
"name": "avalon_sink_data"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_sink_channel"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_sink_error"
}
]
},
"avalon_sink_flag": 1,
"avalon_source": {
"signal": [
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"name": "avalon_source_valid"
},
{
"data_type": {
"fractional_bits": 28,
"signed": true,
"type": "sfix32_En28",
"width": 32
},
"name": "avalon_source_data"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_source_channel"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_source_error"
}
]
},
"avalon_source_flag": 1,
"clocks": {
"sample_frequency_Hz": 48000,
"sample_period_seconds": 2.0833333333333333e-05,
"system_frequency_Hz": 98304000.0,
"system_period_seconds": 1.0172526041666666e-08
},
"conduit_input": [],
"conduit_input_flag": 0,
"conduit_output": [],
"conduit_output_flag": 0,
"entity": "BC_dataplane",
"linux_device_name": "bitcrusher",
"linux_device_version": "18.0",
"model_abbreviation": "BC",
"model_name": "bitcrusher"
}{
"avalon_memorymapped": {
"register": [
{
"data_type": {
"fractional_bits": 7,
"signed": false,
"type": "ufix8_En7",
"width": 8
},
"default_value": 0.5,
"max_value": 1,
"min_value": 0,
"name": "wet_dry_mix",
"reg_num": 2
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"default_value": 0,
"max_value": 1,
"min_value": 0,
"name": "bypass",
"reg_num": 0
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix6",
"width": 6
},
"default_value": 32,
"max_value": 32,
"min_value": 0,
"name": "bits",
"reg_num": 1
}
]
},
"avalon_memorymapped_flag": 1,
"avalon_sink": {
"signal": [
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"name": "avalon_sink_valid"
},
{
"data_type": {
"fractional_bits": 28,
"signed": true,
"type": "sfix32_En28",
"width": 32
},
"name": "avalon_sink_data"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_sink_channel"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_sink_error"
}
]
},
"avalon_sink_flag": 1,
"avalon_source": {
"signal": [
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"name": "avalon_source_valid"
},
{
"data_type": {
"fractional_bits": 28,
"signed": true,
"type": "sfix32_En28",
"width": 32
},
"name": "avalon_source_data"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_source_channel"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_source_error"
}
]
},
"avalon_source_flag": 1,
"clocks": {
"sample_frequency_Hz": 48000,
"sample_period_seconds": 2.0833333333333333e-05,
"system_frequency_Hz": 98304000.0,
"system_period_seconds": 1.0172526041666666e-08
},
"conduit_input": [],
"conduit_input_flag": 0,
"conduit_output": [],
"conduit_output_flag": 0,
"entity": "BC_dataplane",
"linux_device_name": "bitcrusher",
"linux_device_version": "18.0",
"model_abbreviation": "BC",
"model_name": "bitcrusher"
}registers
streaming interfaces
clocks
{
"avalon_memorymapped": {
"register": [
{
"data_type": {
"fractional_bits": 7,
"signed": false,
"type": "ufix8_En7",
"width": 8
},
"default_value": 0.5,
"max_value": 1,
"min_value": 0,
"name": "wet_dry_mix",
"reg_num": 2
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"default_value": 0,
"max_value": 1,
"min_value": 0,
"name": "bypass",
"reg_num": 0
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix6",
"width": 6
},
"default_value": 32,
"max_value": 32,
"min_value": 0,
"name": "bits",
"reg_num": 1
}
]
},
"avalon_memorymapped_flag": 1,
"avalon_sink": {
"signal": [
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"name": "avalon_sink_valid"
},
{
"data_type": {
"fractional_bits": 28,
"signed": true,
"type": "sfix32_En28",
"width": 32
},
"name": "avalon_sink_data"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_sink_channel"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_sink_error"
}
]
},
"avalon_sink_flag": 1,
"avalon_source": {
"signal": [
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "boolean",
"width": 1
},
"name": "avalon_source_valid"
},
{
"data_type": {
"fractional_bits": 28,
"signed": true,
"type": "sfix32_En28",
"width": 32
},
"name": "avalon_source_data"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_source_channel"
},
{
"data_type": {
"fractional_bits": 0,
"signed": false,
"type": "ufix2",
"width": 2
},
"name": "avalon_source_error"
}
]
},
"avalon_source_flag": 1,
"clocks": {
"sample_frequency_Hz": 48000,
"sample_period_seconds": 2.0833333333333333e-05,
"system_frequency_Hz": 98304000.0,
"system_period_seconds": 1.0172526041666666e-08
},
"conduit_input": [],
"conduit_input_flag": 0,
"conduit_output": [],
"conduit_output_flag": 0,
"entity": "BC_dataplane",
"linux_device_name": "bitcrusher",
"linux_device_version": "18.0",
"model_abbreviation": "BC",
"model_name": "bitcrusher"
}external signals
GUI Configuration
{
"name": "bitcrusher",
"pages": [
{
"name": "Main",
"panels": [
{
"controls": [
{
"dataType": "u6,0",
"defaultValue": 32,
"linkerName": "slider1bitcrusher",
"max": 32,
"min": 0,
"module": "bitcrusher",
"style": "default",
"title": "Bits",
"type": "slider",
"units": "bits"
},
{
"dataType": "boolean",
"defaultValue": 0,
"linkerName": "toggle1bitcrusher",
"max": 1,
"min": 0,
"module": "bitcrusher",
"style": "default",
"title": "Bypass",
"type": "toggle",
"units": "bypass"
},
{
"dataType": "u8,7",
"defaultValue": 0.5,
"linkerName": "slider2bitcrusher",
"max": 1,
"min": 0,
"module": "bitcrusher",
"style": "default",
"title": "Wet Dry Mix",
"type": "slider",
"units": "ratio"
}
],
"name": "Bitcrusher"
}
]
}
]
}{
"name": "bitcrusher",
"pages": [
{
"name": "Main",
"panels": [
{
"controls": [
{
"dataType": "u6,0",
"defaultValue": 32,
"linkerName": "slider1bitcrusher",
"max": 32,
"min": 0,
"module": "bitcrusher",
"style": "default",
"title": "Bits",
"type": "slider",
"units": "bits"
},
{
"dataType": "boolean",
"defaultValue": 0,
"linkerName": "toggle1bitcrusher",
"max": 1,
"min": 0,
"module": "bitcrusher",
"style": "default",
"title": "Bypass",
"type": "toggle",
"units": "bypass"
},
{
"dataType": "u8,7",
"defaultValue": 0.5,
"linkerName": "slider2bitcrusher",
"max": 1,
"min": 0,
"module": "bitcrusher",
"style": "default",
"title": "Wet Dry Mix",
"type": "slider",
"units": "ratio"
}
],
"name": "Bitcrusher"
}
]
}
]
}{
"name": "bitcrusher",
"pages": [
{
"name": "Main",
"panels": [
{
"controls": [
{
"dataType": "u6,0",
"defaultValue": 32,
"linkerName": "slider1bitcrusher",
"max": 32,
"min": 0,
"module": "bitcrusher",
"style": "default",
"title": "Bits",
"type": "slider",
"units": "bits"
},
{
"dataType": "boolean",
"defaultValue": 0,
"linkerName": "toggle1bitcrusher",
"max": 1,
"min": 0,
"module": "bitcrusher",
"style": "default",
"title": "Bypass",
"type": "toggle",
"units": "bypass"
},
{
"dataType": "u8,7",
"defaultValue": 0.5,
"linkerName": "slider2bitcrusher",
"max": 1,
"min": 0,
"module": "bitcrusher",
"style": "default",
"title": "Wet Dry Mix",
"type": "slider",
"units": "ratio"
}
],
"name": "Bitcrusher"
}
]
}
]
}pages
panels
controls
App Communication
GUI Autogen
Sound Effects Processor
- Bitcrusher
- Echo
- Flanger
Sound Effects
Bitcrusher
original
bitcrusher
Echo
original
echo
Circular Buffer
Flanger
original
flanger
Creating Sine Waves
Delay and Sum Beamformer
Theory
Spatial Filtering
Theory
\[\sum_{m=0}^{M-1} y_m(t - \Delta_m) = M \times y(t) \]
Linear Array
\[ \Delta_m = \frac{\left(m - \frac{M - 1}{2} \right) d}{c} \sin{(az)} \]
Rectangular Array
$$ \Delta_m = \frac{d}{c} \bigg( \left(m_y - \frac{M - 1}{2} \right) \sin{(az)} \cos{(el)} + \left(\frac{M - 1}{2} - m_z \right) \sin{(el)} \bigg) $$
Delay Interpolation
- Sampling \(\rightarrow\) quantized delays
- Interpolate between samples
Implementation
Implementation Overview
Supporting Hardware
System Overview
Mic Array
Daughter Card
Beamforming Test
Test Setup
- 1 kHz and 10 kHz test tones
- Steered array to 0° and 45°
Test Setup
Results — 1 kHz
no difference in directivity \(\rightarrow\) no change in amplitude
Results — 10 kHz
large difference in directivity and amplitude
Conclusion
Acknowledgments
Committee
- Ross Snider
- Rob Maher
- Brock LaMeres
- Brad Whitaker
Colleagues
- Tyler Davis
- Connor Dack
- Dustin Sobrero
Interns
- Aaron Koenigsberg
- Alex Salois
- Bailey Galacci
- Justin Williams
- Josh Harthan
- Josh Freund
- Matt Blunt
- Hezekiah Austin
- James Eaton
Funding
- National Institutes of Health
- College of Engineering Benjamin Fellowship
Questions?
Supplement
Dataplane
