img-responsive

Introduction

Long-term robustness of perception under various environments has been the bottleneck of lifelong trustworthy autonomy in the application of outdoor mobile robotics and autonomous driving. Although monocular depth prediction has been well studied as a typical perception task, there is little work on robust depth prediction across different environments, e.g. changing illumination and seasons, which is owing to the lack of a diverse real-world dataset containing various scenarios and the corresponding benchmark. To this end, we introduce the SeasonDepth Prediction Challenge as the first open-source challenge focusing on depth prediction performance under different environmental conditions.

The SeasonDepth Prediction Challenge is based on our new monocular depth prediction dataset, SeasonDepth, which contains multi-traverse outdoor images from changing environments. To quantitatively evaluate the accuracy and robustness of monocular depth prediction across dramatically changing environments, we set up two tracks with 7 slices of training set under 12 different environmental conditions, using both mean and variance of performance as evaluation metrics. We believe our competition will contribute to flourish the long-term robust perception research among the research community with our dataset and benchmark.

Challenge Tracks

For this IROS 2022 Competition, we propose to host two tracks --- one supervised learning track and one self-supervised learning track for both supervised learning and self-supervised learning-based methods. We also provide high-quality demonstrations as a tutorial for some baseline algorithms. Anyone can access to the leaderboard of each track and participants can submit the predicted depth maps to occupy the top spot. Different from ICRA 2022 Challenge, participants do not need to sign up but just send the predicted depth maps as a .zip file to seasondepth@outlook.com with cc'ing to hanjianghu@cmu.edu and yangbaoquan@sjtu.edu.cn to submit the results for evaluation. All the results from the final leaderboard of ICRA 2022 Challenge will be regarded as baselines in IROS 2022 Competition leaderboard.

SLT 

Supervised Learning Track

  • The participants can make full use of all depth maps released in the SeasonDepth dataset to train the model

SSLT

Self-Supervised Learning Track

  • Only monocular image sequences are used for model training without the supervision of depth maps

Evaluation

The RGB images and depth ground truth have been released for the training set and validation set of the challenge. For the test set of the challenge, only RGB images will be released, and the corresponding ground truth is retained and used to evaluate the submissions in the challenge. The training and validation set contain 7 multi-environment slices of images under 12 different environments, and we leave one additional slice as the test set for the challenge. Besides our released training and validation set, we set no limits on other third-party public datasets or pretrained models in the competition. Each individual participant will be graded based on 6 metrics in SeasonDepth benchmark of the test set. The evaluation code and instructions can be found on evaluation toolkit for the convenience of participants to evaluate the performance themselves before submission to our challenge website. Note that the grading metrics are scaleless for relative depth values, which are compatible for both supervised and self-supervised learning-based methods.

Awards

SLT 

Supervised Learning Track

  • First prize: 500
  • Second prize: 300
  • Third prize: 200

SSLT

Self-Supervised Learning Track

  • First prize: 500
  • Second prize: 300
  • Third prize: 200

Timeline

  • Training Dataset Released

    Training and validation set availabe here

  • Test Set Released

    Available now! Click to Download!

  • Submission Deadline

    Don't forget to include your code link in your submission 

  • Award Decision Announcement

Organizers

Hanjiang Hu

Ph.D. @ CMU

Jiacheng Zhu

Ph.D. @ CMU

Zuxin Liu

Ph.D. @ CMU

Wenhao Ding

Ph.D. @ CMU

Shuai Wang

Master @ CMU

Jiarun Wei

AutoX

Baoquan Yang

Undergraduate @ SJTU

Zhijian Qiao

PhD @ HKUST

Ding Zhao

Assistant Professor @ CMU

Bo Li

Assistant Professor @  UIUC

Hesheng Wang

Professor @  SJTU

We are still looking for more sponsors. If you are interested, please contact seasondepth@outlook.com.

Special acknowledgement to Shuai Wang due to building up the user maintenance system for our ICRA 2022 event.

Back to top

© The SeasonDepth Committee · All Rights Researved.