MUT: Maritime Urban Tracking Dataset

Abstract: Maritime visual target tracking datasets are essential for developing and benchmarking algorithms that enable safe navigation of intelligent marine vessels in congested sea environments. However, unlike in the automotive domain, where benchmarks such as KITTI have accelerated progress, maritime datasets remain scarce. This paper addresses this gap by introducing a dataset focused on visual perception and tracking in urban waters for autonomous surface vessels, the Maritime Urban Tracking (MUT) dataset. Data were collected using an autonomous ferry prototype for tasks including stereo matching, optical flow, scene flow, SLAM, 2D and 3D object detection, water segmentation, and tracking. The ego-vessel is equipped with two stereo cameras (short and wide-baseline), a LiDAR, an RTK GNSS and IMU for the INS, and a polarized stereo camera rig. In addition, some of the targets are equipped with up to two GNSS receivers with PPK to have world-frame reference tracks in the target tracking evaluation. The dataset has 19 target tracking scenarios, 8 calibration sequences, one mapping scenario and three docking scenarios. The length of these vary, target tracking scenarios last about one minute each, recorded at 30 fps for cameras and 10 Hz for LiDAR, totaling over seven hundred gigabytes of data. Earlier versions of this dataset have already contributed to several maritime autonomy publications. By making the dataset publicly available, we aim to reduce entry barriers and enable broader participation in advancing the field.

We suggest reading the paper and the Readme document on GitHub.