Cognition and State Modeling

Hydra

Hydra is a real-time spatial perception system that incrementally builds 3D scene graphs for robots from sensor streams, semantics, and geometric mapping signals.

Tool Introduction

Core parameters, trigger timing, and visual before/after demo references.

Short Explanation

Feed synchronized robot sensor streams into Hydra and it incrementally builds a layered 3D scene graph of objects, places, rooms, and buildings.

InputSensor data + semantic/geometric cues

OutputLayered 3D scene graph

Trigger TimingTriggered when the ROS launch file receives synchronized sensor streams.

RuntimeUbuntu 24.04 / ROS2 Jazzy / Python bindings

BeforeSensor data + semantic/geometric cues

Prepare the scene, image, video, sensor stream, prompt, or configuration expected by the original project.

AfterLayered 3D scene graph

Read the produced visualization, prediction, map, trajectory, mask, grasp pose, or other documented artifact.

Preset Example

A quick-run style example for the documentation page.

Inputtools/hydra/examples/sample_sequence

PromptUse the default indoor mapping configuration

ExpectedA dynamic scene graph with geometry, semantic objects, places, room nodes, and optimized graph artifacts.

Parameters And Output

Readable controls and the meaning of each returned artifact.

Parameter Explanation

configpathtools/hydra/configs/default.yaml

Controls graph layers, front-end settings, semantic inputs, and back-end optimization behavior.

input_sequencepath

A sensor sequence or simulator output containing the geometric and semantic observations Hydra consumes.

semantic_sourceselectconfigured model or labels

Selects whether labels come from an existing semantic model, logged annotations, or a simulator.

output_dirpath

Destination for graph files, mesh outputs, logs, and visualizations.

Output Explanation

objects

Object nodes with poses, bounding boxes, labels, and relations to the surrounding scene.

places

Topological free-space nodes that support navigation and spatial reasoning.

rooms/building

Higher-level hierarchical nodes used to summarize indoor structure.

How To Use

Official resources, deployment steps, academic context, citation, and source-reported benchmark numbers.

Resources

GitHubhttps://github.com/MIT-SPARK/Hydra Code Downloadhttps://github.com/MIT-SPARK/Hydra/archive/refs/heads/main.zip RSS 2022 Paperhttps://arxiv.org/abs/2201.13360 Hydra Installation Guidehttps://github.com/MIT-SPARK/Hydra#installation-and-running Hydra Evaluation Docshttps://github.com/MIT-SPARK/Hydra#hydra-evaluation IJRR 2024 Foundations Paperhttps://arxiv.org/abs/2304.02246 Installation Docshttps://github.com/MIT-SPARK/Hydra?tab=readme-ov-file#installation

Deployment Notes

Use Ubuntu 24.04 with ROS2 Jazzy for the current default branch.
Install the dependencies from the official installation guide before building Hydra.
Prepare semantic/geometric sensor data or a simulator sequence in the expected repository format.
Run the example script or ROS2 launch flow, then inspect graph outputs with the visualizer.

Relative Path Example

# Relative-path local entry for the Hydra tool folder
python tools/hydra/examples/run_hydra.py   --config tools/hydra/configs/default.yaml   --input tools/hydra/examples/sample_sequence   --output tools/hydra/runs/scene_graph

# Suggested repository layout when adding local files:
# tools/hydra/README.md
# tools/hydra/configs/default.yaml
# tools/hydra/examples/sample_sequence/
# tools/hydra/runs/scene_graph/

# This page documents the path. It does not execute Hydra.

Expected Result Shape

{
  "tool": "hydra",
  "status": "ok",
  "scene_state": [
    {
      "label": "3D scene graph construction",
      "score": 0.87,
      "output": "Layered 3D scene graph"
    }
  ],
  "timing": {
    "runtime": "On NVIDIA Xavier NX for uHumans2 Office, Hydra reports objects 75+/-35 ms, places 33+/-6 ms, and rooms 55+/-41 ms, targeting a 5 Hz keyframe rate.",
    "device": "documented in source benchmark when available"
  },
  "artifacts": {
    "visualization": "tools/hydra/runs/visualization.png",
    "raw_predictions": "tools/hydra/runs/predictions.json"
  }
}

Paper figure

Academic Info

Paper identity and contribution summary.

TitleHydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization

AuthorsNathan Hughes, Yun Chang, Luca Carlone

VenueRSS 2022; Foundations of Spatial Perception for Robotics, IJRR 2024

ContributionBuilds and optimizes dynamic 3D scene graphs online, giving robotics systems a structured representation of places, objects, rooms, agents, and metric-semantic spatial context.

Citation

@misc{hydra2022,
  title={Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization},
  author={Nathan Hughes and Yun Chang and Luca Carlone},
  year={2022},
  note={RSS 2022; Foundations of Spatial Perception for Robotics, IJRR 2024},
  url={https://arxiv.org/abs/2201.13360}
}

Benchmark

Only compact, source-reported numbers are shown here.

Dataset	Metric	Value	Runtime	Source
uHumans2 Office	Component timing	Objects 24.1+/-12.8 ms, places 8.1+/-1.3 ms, rooms 19.0+/-12.3 ms	5 Hz keyframe target	Hydra paper
SidPac Floor 3-4	Component timing	Objects 75.3+/-37.0 ms, places 4.2+/-2.1 ms, rooms 15.0+/-14.6 ms	Online graph construction	Hydra paper

Artifacts

RSS 2022 paper, component timing table, room precision/recall evaluation, loop-closure ablation, scene graph outputs, config files, logs, and visualization GIFs.

Demo Images

Visual references from the original tool. Click any image to inspect the original size.