Cognition and State Modeling

retrieve_past_visual_state_faiss

Retrieves similar past visual memories from an embedding index when current recognition is uncertain.

Tool Introduction

Core parameters, trigger timing, and visual before/after demo references.

Short Explanation

Search a visual memory bank for earlier snapshots that look similar to the current uncertain target.

InputImage embedding or cropped image

OutputMatched visual memories, memory ids, scores, snapshot paths

Trigger TimingTriggered on demand after the required input files and configuration are prepared.

RuntimeLocal / cloud wrapper

BeforeImage embedding or cropped image

Prepare the scene, image, video, sensor stream, prompt, or configuration expected by the original project.

AfterMatched visual memories, memory ids, scores, snapshot paths

Read the produced visualization, prediction, map, trajectory, mask, grasp pose, or other documented artifact.

Preset Example

A quick-run style example for the documentation page.

Inputtools/retrieve-past-visual-state-faiss/examples/query.npy

Prompttop_k: 3

ExpectedTop-k memory ids with distances or similarity scores and snapshot references.

Parameters And Output

Readable controls and the meaning of each returned artifact.

Parameter Explanation

query_feature_pathpathtools/retrieve-past-visual-state-faiss/examples/query.npy

Embedding file for the current crop or scene.

query_imagefile

Optional image crop that can be embedded before FAISS search.

top_knumber3

Number of nearest memories to return.

feature_dimnumber512

Embedding dimension expected by the index.

Output Explanation

results

Ranked visual memories returned by the FAISS search.

memory_id

Identifier of a stored past observation.

score

Distance or similarity score for the retrieved memory.

snapshot_path

Path to the stored image or state snapshot, when available.

How To Use

Official resources, deployment steps, academic context, citation, and source-reported benchmark numbers.

Resources

FAISS GitHubhttps://github.com/facebookresearch/faiss Code Downloadhttps://github.com/facebookresearch/faiss/archive/refs/heads/main.zip FAISS Paperhttps://arxiv.org/abs/2401.08281 FAISS Docshttps://faiss.ai/

Deployment Notes

Build or load a FAISS index over historical visual embeddings.
Prepare query embeddings or image crops under the examples folder.
Run the retrieval wrapper with a repository-relative query path and top-k value.
Save ranked memory results under tools/retrieve-past-visual-state-faiss/runs/.

Relative Path Example

python tools/retrieve-past-visual-state-faiss/run.py --query-feature tools/retrieve-past-visual-state-faiss/examples/query.npy --top-k 3 --output tools/retrieve-past-visual-state-faiss/runs/results.json

Expected Result Shape

{
  "tool": "retrieve-past-visual-state-faiss",
  "status": "ok",
  "results": [
    {
      "label": "Visual memory retrieval",
      "score": 0.87,
      "output": "Matched visual memories, memory ids, scores, snapshot paths"
    }
  ],
  "timing": {
    "runtime": "The submitted wrapper is described as interactive; actual latency depends on index type, vector count, GPU/CPU backend, and top-k setting.",
    "device": "documented in source benchmark when available"
  },
  "artifacts": {
    "visualization": "tools/retrieve-past-visual-state-faiss/runs/visualization.png",
    "raw_predictions": "tools/retrieve-past-visual-state-faiss/runs/predictions.json"
  }
}

Paper figure

Academic Info

Paper identity and contribution summary.

TitleFAISS-style visual memory retrieval

AuthorsAdd authors

VenueSubmitted tool sheet / local wrapper

ContributionUses vector search over stored visual embeddings to recover clear historical observations under occlusion or partial visibility.

Citation

@misc{retrievepastvisualstatefaissYEAR,
  title={FAISS-style visual memory retrieval},
  author={Author},
  year={YEAR},
  note={Submitted tool sheet / local wrapper},
  url={https://arxiv.org/abs/2401.08281}
}

Benchmark

Only compact, source-reported numbers are shown here.

Dataset	Metric	Value	Runtime	Source
Billion-scale similarity search	Nearest-neighbor search implementation speedup	8.5x faster than the previous reported state of the art	GPU nearest-neighbor search implementation	Upstream proxy: Meta Engineering FAISS blog
1B high-dimensional vectors	k-nearest-neighbor graph scale	First k-NN graph constructed on 1 billion high-dimensional vectors	GPU k-selection implementation	Upstream proxy: Meta Engineering FAISS blog

Artifacts

Upstream FAISS blog, official FAISS docs, query embedding example, top-k search result JSON, and local deployment notes from the submitted spreadsheet.

Demo Images

Visual references from the original tool. Click any image to inspect the original size.