DARPA SubT Final Competition

The DARPA Subterranean (SubT) Challenge Competition Final Event took place last week from September 21 through September 24 at the Louisville Mega Cavern. This marked the culmination of SubT, which started in 2018 and consisted of multiple competitions across a variety of physical and virtual competition scenarios. We wanted to congratulate all of the competitors on their hard work and innovation; and recognize the winning teams, Team CERBERUS in the systems competition and Team Dynamo in the virtual competition.

The overarching goal of the SubT competition is to develop robotic solutions capable of mapping and searching complex underground environments. SubT is the latest incarnation of DARPA robotics challenges, with prior programs including the DARPA Robotics Challenge (humanoid robots), the DARPA Urban Challenge (autonomous city driving), and the DARPA Grand Challenge (autonomous ground vehicles). The challenge was designed to mimic the conditions found in dangerous subterranean emergencies such as collapsed mines or caves, damaged subways during a hurricane, or a nuclear power plant after a melt down. This final event brought together aspects of the three previous competions and featured aspects of urban, cave, and mining subterranean exploration.

A SubT competitor must deploy a team a robots to explore three different types of environments including tunnel, cave, and urban. Artifacts are spread throughout the environment. Successfully identifying the location of an artifact within 5 meters of its ground truth pose will earn a team 1 point. The team with the most points wins. While this competition sounds relatively straightforward, the task is actually quite difficult for most robots. The difficulty is due to a range of factors, including the lack of GPS reception, dark and awkward terrain, and the difficulty in establishing a wireless connection back to the robot's operator. DARPA has released this video of the course for the Tunnel Circuit event that gives a pretty good overview of the complex environments they would like to explore.

With only a couple of points between first place and third place the competition between the systems track teams for the SubT challenge was fierce; it was fascinating to see how the team strategies were dictated by the composition and capabilities of the team’s robots. If you're not familiar with the teams, we covered the urban competition back in February of 2020, and IEEE Spectrum wrote a detailed description of each team prior to the final competition. Due to the nature of or role in the competition we had to keep the teams at arms length; so we don’t know much about the software payloads that were used on the robots. We suspect that most if not all the teams used ROS in some capacity. Looking at the first and second place teams the legged platforms Anymal and Spot both support ROS interfaces and CSIRO has published a white paper from an earlier competition stating that they used ROS. Further down the field team MARBLE, CTU-CRAS-NORLAB, and CoSTAR all made heavy use of the Clearpath Husky robot that also uses ROS extensively. We eagerly await the team’s final white papers where we’ll find out more about their software stacks; and if we’re lucky some of the software they developed will make its way to the open source community.

The in-person competition is only one aspect of DARPA's SubT Challenge. SubT also had a virtual competition that took place in a high fidelity simulation. This virtual competition lowered the barrier to entry for SubT by obviating the need for the competing teams to have a physical robot; instead we created a detailed simulation in Gazebo for the competition. These simulations allowed anyone with a computer the opportunity to compete at SubT; and most of the virtual track competitors were research and academic groups not directly sponsored by DARPA. The Gazebo simulations we created for SubT mirror the real world competition in almost every detail, as they should, given that we cloned the competition arenas and robots using the latest photogrammetry and laser scanning techniques. To further enhance the realism of these simulations we developed Gazebo plugins to replicate various effects specific to the competition. For example, network connectivity is constrained in the SubT environment -- WiFi doesn’t penetrate solid rock -- so we had to develop Gazebo plugins to replicate the conditions found underground. Other plugins we created included a thermal camera for detecting injured humans and dynamic rock falls triggered by the motion of the robots. We made both the simulated robots and practice environments readily available to the public using the SubT portal and Ignition Fuel, our web-based model repository. To make it easy for competitors, we used ROS to create a number of examples and tutorials on how to get started creating their own virtual SubT robot. All told, over two hundred new models have been created for the SubT, all of which are freely available at the SubT Tech Repo. Source code for the SubT Challenge simulator, along with tutorials and documentation, can be found at https://github.com/osrf/subt.

Scaling Simulation

Three virtual circuit events were held leading up to the SubT Final Event. Each circuit covered a different subterranean domain with different operational considerations. In order, the circuit events were Tunnel, Urban, and Cave. The tunnel domain represents human made rough underground environments similar to mines. Urban environments are also human made, but resemble subways and lower levels of buildings. Cave environments are natural unstructred underground environments.

We ran the three Virtual Circuits and the Virtual Finals on Cloudsim, our open source platform for hosting Ignition Gazebo on cloud infrastructue. For the Virtual Finals, we used 1824 cloud machines to perform 7781 hours of computation to simulate 58 robots (37 aerial / 21 ground) deployed across the nine teams. More than 12,000 kms were traversed across 8 different worlds. Over the course of the SubT competition we adapted and improved Cloudsim to address the evolving needs of the program. For example, for the final completion we added simulation recording capabilities to Cloudsim to allow the virtual competition to be broadcast during the final competition. This segment from DARPAtv during the event does a great job describing the scope and scale of the simulated competition. The full results for the virtual competition can be found on the DARPA leaderboard.

Final Scores for the Virtual Circuit

Wrapping It All Up

A real-world systems track accompanied each virtual circuit event, except for the Cave Circuit due to COVID; this is to say the Cave Circuit was purely virtual and run exclusively on Cloudsim. Open Robotics directly assisted in conducting the virtual circuit competitions using Cloudsim and Gazebo; and ROS was used by a large number of the competitors. Our participation in the DARPA SubT competition has done much to further the capabilities found in both ROS and Gazebo; and created a new generation of talented roboticists that have taken their academic research and applied them to real-world scenarios. The impact of the event will be felt for years to come due to the large number of open-source contributions to ROS, Gazebo, and Cloudsim that were needed to make SubT a reality. Contributions like the SubT Virtual Testbed, the SubT Hello World Repo, and the SubT robots and models available on Ignition Fuel will have a lasting positive impact on our open source community. We’re proud of the positive impact DARPA SubT has had on both open source and the field of robotics and we’re excited to see what comes next.