The goal of SafeBench is to systematically evaluate the safety and security of autonomous driving (AD) algorithms based on diverse testing scenarios and comprehensive evaluation metrics. Safebench is based on Carla, a high-fidelity and open-sourced AD Simulator. There are three main features that distinguish Safebench from other scenario-based benchmarks:

• Parallel Running. Safebench supports the parallel running of multiple scenarios in one map, which dramatically increases the efficiency of collecting training data and evaluation.
• Comprehensive Scenario. Safebench integrates lots of scenario-generation algorithms, including data-driven generation, rule-based design, and adversarial examples.
• Usage Coverage. Safebench supports three running modes, including training ego policy, training scenario policy, and evaluating ego in scenarios.

• We are hosting a challenge in CVPR 2023 using Safebench platform. Please check our website for more details.
• The new version of Safebench is avaliable now.
• The paper introducing Safebench is published on NeurIPS2022.