Scenario 17 - 22
License
If you want to use the dataset or scripts on this page, please use the link below to generate the final list of papers that need to be cited.
Overview
The data collection setup of Scenarios 17–22 (outdoor). The left subfigure shows the exact locations of Unit 1 and Unit 2; the middle subfigure shows the full view of the street and surroundings; the right subfigure shows the traffic and street view from Unit 1 perspective.
Scenarios 17-22 are collected in an outdoor wireless environment representing a two-way city street. The DeepSense Testbed 3 is deployed in this environment where the two units are placed on the opposite sides of the street. The transmitter (Unit 2) constantly transmits using one antenna element of the phased array to realize omnidirectional transmission. The receiver continuously scans the surrounding environment using a receive mmWave beam steering codebook of 64 beams and by measuring the receive power with each beam. The testbed collects data samples at a dynamic rate ranging from 6 to 12 samples/s. The average sampling rate is 10 samples/s. Each data sample includes an RGB image and a 64-element receive power vector, both collected by Unit 1. Please refer to the detailed description of the testbed presented here.
McAllister Ave: It is a two-way street with 2 lanes, a width of 10.6 meters, and a vehicle speed limit of 25mph (40.6 km per hour. It is worth mentioning here that since this is a city street, vehicles of various sizes and travel speeds pass in both directions. This creates a diverse pool of blockages for the LOS link between the transmitter and receiver, which results in diverse received power maps (i.e., diverse power fluctuations across all 64 beam over time instances) and a diverse dataset.
Scenarios 17-22 are outdoor wireless environments representing a two-way city street. The DeepSense Testbed 3 is deployed in this environment where the two units are placed on the opposite sides of the street. The transmitter (Unit 2) constantly transmits using one antenna element of the phased array to realize omnidirectional transmission. The receiver continuously scans the surrounding environment using a receive mmWave beam steering codebook of 64 beams and by measuring the receive power with each beam. The testbed collects data samples at a dynamic rate ranging from 6 to 12 samples/s. The average sampling rate is 10 samples/s. Each data sample includes an RGB image and a 64-element receive power vector, both collected by Unit 1. Please refer to the detailed description of the testbed presented here.
McAllister Ave: It is a two-way street with 2 lanes, a width of 10.6 meters, and a vehicle speed limit of 25mph (40.6 km per hour. It is worth mentioning here that since this is a city street, vehicles of various sizes and travel speeds pass in both directions. This creates a diverse pool of blockages for the LOS link between the transmitter and receiver, which results in diverse received power maps (i.e., diverse power fluctuations across all 64 beam over time instances) and a diverse dataset.
Collected Data
Overview
Number of Data Collection Units: 2 (using DeepSense Testbed #3)
Total Number of Data Samples: 460837
Scenario-wise Data Samples: Scenario 17: 70000 || Scenario 18: 120000 || Scenario 19: 160000 || Scenario 20: 70837 || Scenario 21: 20000 || Scenario 22: 20000
Data Modalities: RGB images, 64-dimensional received power vector, GPS locations
Sensors at Unit 1: (Stationary Receiver)
- Wireless Sensor [Phased Array]: A 16-element antenna array operating in the 60 GHz frequency band and receives the transmitted signal using an over-sampled codebook of 64 pre-defined beams
- Visual Sensor [Camera]: The main visual perception element in the testbed is an RGB-D camera. The camera is used to capture RGB images of 960×540 resolution at a base frame rate of 30 frames per second (fps)
- Position Sensor [GPS Receiver]: A GPS-RTK receiver for capturing accurate real-time locations for the stationary unit 1
Sensors at Unit 2: (Mobile Transmitter)
- Wireless Sensor [Phased Array]: A stationary 60 GHz mmWave transmitter. The transmitter (Unit 2) constantly transmits using one antenna element of the phased array to realize omnidirectional transmission
- Position Sensor [GPS Receiver]: A GPS-RTK receiver for capturing accurate real-time locations for the stationary unit 2
| Testbed | 3 |
|---|---|
| Instances | Scenario 17: 70000; Scenario 18: 120000; Scenario 19: 160000; Scenario 20: 70837; Scenario 21: 20000; Scenario 22: 20000 |
| Number of Units | 2 |
| Data Modalities | RGB images, 64-dimensional received power vector, GPS locations |
| Unit1 | |
| Type | Stationary |
| Hardware Elements | RGB camera, mmWave phased array receiver, GPS receiver |
| Data Modalities | RGB images, 64-dimensional received power vector, GPS locations |
| Unit2 | |
| Type | Stationary |
| Hardware Elements | mmWave omni-directional transmitter, GPS receiver |
| Data Modalities | GPS locations |
Data Visualization
Download
Please login to download the DeepSense datasets
How to Access Scenario X Data?
Step 1. Download Scenario Data
Step 2. Extract the scenarioX.zip file
Scenario X folder consists of three sub-folders:
- unit1: Includes the data captured by unit 1
- unit2: Includes the data captured by unit 2
- resources: Includes the scenario-specific annotated dataset, data labels, and other additional information. For more details, refer to the resources section below.
The scenario X folder also includes the “scenario X.csv” file with the paths to all the collected data. For each coherent time, we provide the corresponding visual, wireless, and GPS data.
Resources
What are the Additional Resources?
Resources consist of the following information:
- visual data annotations: For the visual data, we provide the coordinates of the 2D bounding box and attributes for each frame
- data labels: The labels comprises of the ground-truth link blockage status, where ‘0’ represent line-of-sight (LOS) connection and ‘1’ represent blocked connection
- additional information: Includes the scenario-specific additional data. Details of the information is provided below
Visual Data Annotations
After performing the post-processing steps presented here, we generate the annotations for the visual data. Using state-of-the-art machine learning algorithms and multiple validation steps, we achieve highly accurate annotations. In this particular scenario, we provide the coordinates of the 2D bounding box and attributes for each frame.
We, also, provide the ground-truth labels for ‘8’ object classes as follows: ‘1’: person || ‘2’: bicycle || ‘3’: car || ‘4’: motorcycle || ‘5’: airplane || ‘6’: bus || ‘7’: train || ‘8’: truck
We follow the YOLO format for the bounding-box information. In the YOLO format, each bounding box is described by the center coordinates of the box and its width and height. Each number is scaled by the dimensions of the image; therefore, they all range between 0 and 1. Instead of category names, we provide the corresponding integer categories.
Data Labels
The label comprises of the ground-truth link-status manually annotated from the RGB images
- Ground-Truth Blockage: We utilize the mmWave receive power and annotate the data samples with link-status labels. More specifically, a link-status label of ‘1’ is assigned to a time instance when the LOS link is blocked, and a label of ‘0’ otherwise.
Additional Information
We, further, provide additional information for each sample present in the scenario dataset. The contents of the additional data is listed below:
- index: It represents the sample number
- time_stamp[UTC]: This represents the time of data capture in “hr-mins-secs-ms” format
An example table comprising of the data labels and the additional information is shown below.
| index | unit1_rgb | unit1_pwr_60ghz | unit1_blockage | unit1_loc | unit2_loc | seq_index | time_stamp[UTC] |
|---|---|---|---|---|---|---|---|
| 1 | ./unit1/camera_data/image_BS2_1_18_34_31.jpg | ./unit1/mmWave_data/mmWave_power_1.txt | ./unit1/label_data/label_1.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-31-0'] |
| 2 | ./unit1/camera_data/image_BS2_2_18_34_31.jpg | ./unit1/mmWave_data/mmWave_power_2.txt | ./unit1/label_data/label_2.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-31-500'] |
| 3 | ./unit1/camera_data/image_BS2_3_18_34_32.jpg | ./unit1/mmWave_data/mmWave_power_3.txt | ./unit1/label_data/label_3.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-32-0'] |
| 4 | ./unit1/camera_data/image_BS2_4_18_34_32.jpg | ./unit1/mmWave_data/mmWave_power_4.txt | ./unit1/label_data/label_4.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-32-200'] |
| 5 | ./unit1/camera_data/image_BS2_5_18_34_32.jpg | ./unit1/mmWave_data/mmWave_power_5.txt | ./unit1/label_data/label_5.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-32-400'] |
| 6 | ./unit1/camera_data/image_BS2_6_18_34_32.jpg | ./unit1/mmWave_data/mmWave_power_6.txt | ./unit1/label_data/label_6.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-32-600'] |
| 7 | ./unit1/camera_data/image_BS2_7_18_34_32.jpg | ./unit1/mmWave_data/mmWave_power_7.txt | ./unit1/label_data/label_7.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-32-800'] |
| 8 | ./unit1/camera_data/image_BS2_8_18_34_33.jpg | ./unit1/mmWave_data/mmWave_power_8.txt | ./unit1/label_data/label_8.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-33-0'] |
| 9 | ./unit1/camera_data/image_BS2_9_18_34_33.jpg | ./unit1/mmWave_data/mmWave_power_9.txt | ./unit1/label_data/label_9.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-33-166'] |
| 10 | ./unit1/camera_data/image_BS2_10_18_34_33.jpg | ./unit1/mmWave_data/mmWave_power_10.txt | ./unit1/label_data/label_10.txt | ./unit1/GPS_data/gps_location.txt | ./unit2/GPS_data/gps_location.txt | 1 | ['18-34-33-332'] |
