The UT Campus Object Dataset (CODa) is a mobile robot egocentric perception dataset collected at the University of Texas at Austin campus designed for research and planning for autonomous navigation in urban environments. CODa provides benchmarks for 3D object detection and 3D semantic segmentation. At the moment of publication, CODa contains the largest diversity of ground truth object class annotations in any available 3D LiDAR dataset collected in human-centric urban environments, and over 196 million points annotated with semantic labels to indicate the terrain type of each point in the 3D point cloud.
- 8.5 hours of multimodal sensor data.
Synchronized 3D point clouds and stereo RGB video from a 128-channel 3D LiDAR and two 1.25MP RGB cameras at 10 fps.
- RGB-D videos from an additional 0.5MP sensor at 7 fps A 9-DOF IMU sensor at 40 Hz.
- 54 minutes of ground-truth annotations containing 1.3 million 3D bounding boxes with instance IDs for 50 semantic classes.
- 5000 frames of 3D semantic annotations for urban terrain, and pseudo-ground truth localization.
Dataset Characteristics
Robot operators repeatedly traversed 4 unique pre-defined paths - which we call trajectories - in both the forward and opposite directions to provide viewpoint diversity. Every unique trajectory was traversed at least once during cloudy, sunny, dark lighting and rainy conditions amounting to 23 "sequences". Of these sequences, 7 were collected during cloudy conditions, 4 during evening/dark conditions, 9 during sunny days, and 3 immediately before/after rainfall. We annotated 3D point clouds in 22 of the 23 sequences.
Spatial map of geographic locations contained in CODa.
Data Collection
The data collection team consisted of 7 robot operators. The sequences were traversed in teams of two; one person tele-operated the robot along the predefined trajectory and stopped the robot at designated waypoints - denoted on the map above - on the route. Each time a waypoint was reached, the robot was stopped and the operator noted both time and waypoint reached. The second person managed the crowds' questions and concerns.
Before each sequence, the robot operator manually commanded the robot to publish all sensor topics over the Robot Operating System (ROS) middleware and recorded these sensor messages to a rosbag. At the end of each sequence, the operator stopped the data recording manually and post-processed the recorded sensor data into individual files. We used the official CODa development kit to extract the raw images, point clouds, inertial, and GPS information to individual files. The development kit and documentation are publicly available on Github (https://github.com/ut-amrl/coda-devkit).
Robot
Top-down diagram view of robot used for CODa.
For all sequences, the data collection team tele-operated a Clearpath Husky, which is approximately 990mm x 670mm x 820mm (length, width, height) with the sensor suite included. The robot was operated between 0 to 1 meter per second and used 2D, 3D, stereo, inertial, and GPS sensors. More information about the sensors is included in the Data Report.
Human Subjects
This study was approved by the University of Texas at Austin Institutional Review Board (IRB) under the IRB ID: STUDY00003493. Anyone present in the recorded sensor data and their observed behavior was purely incidental. To protect the privacy of individuals recorded by the robots and present in the dataset, we did not collect any personal information on individuals. Furthermore, the operator managing the crowd was acting as a point of contact for anyone who wished not to be present in the dataset. Anyone who did not wish to participate and expressed so was noted and removed from the sensor data and from the annotations. Included in this data package are the IRB exempt determination and the Research Information Sheet distributed to the incidental participants.
Data Annotation
Deepen AI annotated the dataset. We instructed their labeling team on how to annotate the 3D bounding boxes and 3D terrain segmentation labels. The annotation document is part of the data report, which is included in this dataset.
Data Quality Control
The Deepen team conducted a two-stage internal review process during the labeling process. In the first stage, human annotators reviewed every frame and flagged issues for fixing. In the second stage, a separate team reviewed 20% of the annotated frames for missed issues. Their quality assurance (QA) team repeated this process until at least 95% of 3D bounding boxes and 90% of semantic segmentation labels met the labeling standards. The CODa data collection team also manually reviewed each completed frame. While it is possible to convert these annotations from 3D to 2D, we can only guarantee high data quality for annotations on the 3D point clouds.
Dataset Organization
CODa is organized in the following levels: the data modality (2D/3D), sensor type, and sequence number.
We recommend users first inspect the metadata under the metadata directory to understand which files should be used for their task. The metadata files contain the file paths to the ground truth annotations relative to the root location where CODa is downloaded on your file system. For instance, there are lists that define the annotation files for the train, validation, and test splits for the 3D object detection and 3D semantic segmentation tasks. For an in-depth explanation of the CODa file structure, we refer the reader to the data report included in this dataset.
Dataset File Structure for CODa.
CODa Distribution and Bulk Data Download
CODa is distributed in four sizes: tiny, small, medium, and full. At this time we offer the tiny dataset to download from the Texas Data Repository (TDR) and the remaining splits by url from Texas Advanced Computing Center (TACC). The tiny dataset is only meant for preview purposes and is roughly 5% of the total dataset. The small dataset can be used for preliminary experiments as contains 25% of the total dataset. The medium dataset is roughly 50% of the full dataset and can be used for model training and experiments. The full dataset is 4 terabytes and should be used for public CODa benchmarks. We currently only release the tiny dataset and will release the full dataset in the near future.
Downloading CODa from TACC
To download each split or full sequence from CODa, you may download the zip files from TACC. Each sequence is in the sequence directory with the name "{sequence#}.zip". Each split is in the split directory and it labelled "CODa_{splitname}_split.zip". We recommend downloading and unzipping the contents into a dedicated directory for CODa.
Trying CODa tiny from TDR
To download the tiny split, download all of the .tar.gz files in the CODa_tiny_parts directory from this data repository. We split up the dataset into multiple tar files due to the 5GB file size limit on the Texas Data Repository. The tar files follow the naming convention CODa_{SPLIT}.tar.gz.part000 - CODa_{SPLIT}.tar.gz.part999, where the last three digits indicate the file order. We recommend downloading these files to the same directory to preserve the original file structure after decoding. Then use the ccc_tar_combiner.sh script provided in this repo to combine and decode the split .tar.gz files. You can run the script using the following, replacing CODA_DIRECTORY_PATH with the directory where you downloaded the .tar.gz files to: “bash ccc_tar_combiner.bash -d CODA_DIRECTORY_PATH”
Models and Using CODa
We release the official CODa development kit on Github and provide a version available on the Python Package Index (PIP). The development kit provides tools for downloading the large and medium versions of the dataset programmatically and visualizing the data and annotations in 2D and 3D modalities.
The official model training code and configurations for CODa are at Github. We will release the pre-trained model weights in the future on Texas Data Repository as well. For an in-depth look at the number and type of model configurations, please refer to the paper cited in Related Materials in this publication.
Data Usage Examples
Using CODa annotations we trained multiple 3D object detection and 3D semantic segmentation models. For a summary of these results showing the performance of the CODa dataset we refer the reader to the paper cited in Related Publications.