Cognition and State Modeling

query_3d_scene_graph

Queries a stored 3D scene graph to recover previously observed object locations and spatial relations.

Tool Introduction

Core parameters, trigger timing, and visual before/after demo references.

Short Explanation

Use this tool when the current view cannot see an object but the robot may have observed it earlier in the mapped scene.

InputScene graph + object query + relation query

OutputObject matches, 3D positions, neighbor relations, confidence

Trigger TimingTriggered on demand after the required input files and configuration are prepared.

RuntimeLocal / hybrid wrapper

BeforeScene graph + object query + relation query

Prepare the scene, image, video, sensor stream, prompt, or configuration expected by the original project.

AfterObject matches, 3D positions, neighbor relations, confidence

Read the produced visualization, prediction, map, trajectory, mask, grasp pose, or other documented artifact.

Preset Example

A quick-run style example for the documentation page.

Inputtools/query-3d-scene-graph/examples/request.json

Promptobject_query: table; relation_query: nearby chairs

ExpectedMatched nodes with 3D centers, bounding boxes, relation edges, and confidence values.

Parameters And Output

Readable controls and the meaning of each returned artifact.

Parameter Explanation

dataset_rootpath/path/to/3RScan

Root folder containing the stored scans or scene graph artifacts.

target_scan_idtextf62fd5fd-9a3f-2f44-883a-1e5cf819608e

Scene identifier used to select the historical graph.

object_querytext

Object class, instance, or natural-language target to search for.

relation_querytext

Optional spatial relation such as near, on, lower than, or connected to.

Output Explanation

matches

Candidate graph nodes matching the object query.

position_3d

Stored 3D center and bounding-box size for each matched node.

confidence

Confidence or matching score produced by the local wrapper.

How To Use

Official resources, deployment steps, academic context, citation, and source-reported benchmark numbers.

Resources

3DSSG GitHubhttps://github.com/ShunChengWu/3DSSG Code Downloadhttps://github.com/ShunChengWu/3DSSG/archive/refs/heads/main.zip SceneGraphFusion Paperhttps://arxiv.org/abs/2010.05273 3RScan Datasethttps://waldjohannau.github.io/RIO/

Deployment Notes

Prepare a 3D scene graph dataset or converted scan graph under the tool folder.
Write the object and relation query as repository-relative JSON.
Run the local wrapper and save graph matches under tools/query-3d-scene-graph/runs/.
Use the returned 3D positions as memory hints for downstream perception or planning.

Relative Path Example

python tools/query-3d-scene-graph/run.py --request tools/query-3d-scene-graph/examples/request.json --output tools/query-3d-scene-graph/runs/result.json

Expected Result Shape

{
  "tool": "query-3d-scene-graph",
  "status": "ok",
  "results": [
    {
      "label": "Queryable 3D scene memory",
      "score": 0.87,
      "output": "Object matches, 3D positions, neighbor relations, confidence"
    }
  ],
  "timing": {
    "runtime": "Upstream proxy timing reports 35 Hz in the paper abstract, with detailed reconstruction at 28 ms and graph prediction at 133 ms.",
    "device": "documented in source benchmark when available"
  },
  "artifacts": {
    "visualization": "tools/query-3d-scene-graph/runs/visualization.png",
    "raw_predictions": "tools/query-3d-scene-graph/runs/predictions.json"
  }
}

Paper figure

Academic Info

Paper identity and contribution summary.

TitleSceneGraphFusion-style queryable memory tool

AuthorsAdd authors

VenueSubmitted tool sheet / local wrapper

ContributionTurns historical 3D scene graph observations into a query interface for objects that are no longer visible.

Citation

@misc{query3dscenegraphYEAR,
  title={SceneGraphFusion-style queryable memory tool},
  author={Author},
  year={YEAR},
  note={Submitted tool sheet / local wrapper},
  url={https://arxiv.org/abs/2010.05273}
}

Benchmark

Only compact, source-reported numbers are shown here.

Dataset	Metric	Value	Runtime	Source
3RScan / 3DSSG, full scene with GT instances	Relationship recall	R@50 0.85; R@100 0.87	Upstream graph prediction, not local wrapper timing	Upstream proxy: SceneGraphFusion CVPR 2021 paper, Table 1
3RScan / 3DSSG, full scene with GT instances	Object / predicate recall	Object R@5 0.70, R@10 0.80; predicate R@3 0.97, R@5 0.99	Upstream graph prediction, not local wrapper timing	Upstream proxy: SceneGraphFusion CVPR 2021 paper, Table 1
Incremental 3D scene graph prediction	Runtime	35 Hz system-level claim; 28 ms reconstruction + 133 ms graph prediction detail	RGB-D sequence processing	Upstream proxy: SceneGraphFusion CVPR 2021 paper

Artifacts

Upstream SceneGraphFusion paper, 3DSSG/3RScan links, input/output JSON examples, and local deployment notes from the submitted spreadsheet.

Demo Images

Visual references from the original tool. Click any image to inspect the original size.