fullseye

Using Fullseye as an AI assistant’s RAG (for Claude Code)

日本語 · English · 简体中文 · 繁體中文 · 한국어 · Deutsch · हिन्दी

The recommended way to run Fullseye is as the knowledge base (RAG) of an AI coding assistant. Because every op carries a machine-readable Markdown note (docs/ops, a single source of truth), no extra vector database or embedding service is needed. Any environment that can grep is already a RAG.

Three tiers of adoption are provided. Tiers 0/1 have zero external dependencies (self-contained in the Fullseye repository).

Use from PyPI: in an environment where you pip install fullseye, the console script fullseye-rag is available. A checkout (clone / pip install -e .) pins the full docs/ops corpus to the skill; a wheel-only install pins the bundled OP_CATALOG.md (the all-op catalog for AI) instead (if you later want the full per-op notes, just clone the repo and re-run). Update with py -3.11 tools/update_fullseye.py (refuses a dirty tree, --ff-only, updates the skill over a backup, and leaves Studio settings untouched — designed not to wreck your environment).

There is more than one way in (read this first)

Given only the documents, Codex and Copilot both failed to find the task-to-op index on the first try (2026-09-15), because this guide did not point to it. There are five ways in:

  1. By task → docs/CAPABILITIES.en.md. One file per task (docs/capabilities/) with the recommended pipeline (ordered ops) / alternatives / limits / real-world calibration; tests/test_capabilities.py checks that every op exists and that the types (in → out) connect from one step to the next.
  2. By what you want to do, in your own words → fullseye-rag search / fullseye.search_ops (next section).
  3. By op name or concept → docs/ops/INDEX.md → family index → op note.
  4. Programmatically → fs.op_find(query) (free text, stems) / fs.op_assist(op) (spec, presets, what to chain next) / fs.op_path(from_sort, to_sort) (op chain from type A to type B) / fs.op_producers(sort) (ops that produce a type) / fs.op_accepts(op) (measured accepted types); on the command line fullseye has <op> (HALCON names work too).
  5. Index files → docs/OP_INDEX.json (type contracts of every op) and the OP_CATALOG.md shipped in the wheel.

On Windows shells, read the notes as UTF-8 explicitly — they are UTF-8 without BOM; opened with the default code page the Japanese text turns to mojibake and an AI concludes “not there” when it simply could not read it (this happened with Codex). PowerShell: Get-Content -Encoding utf8; Python: PYTHONUTF8=1.

Start here: find ops by what you want to do (works with every tier)

fullseye-rag search "remove noise"          # command (works with just pip install fullseye)
fullseye-rag search "Rauschen entfernen" -k 5 --lang en

import fullseye
fullseye.search_ops("edge detection", k=5)   # Python; over MCP use fullseye_find_ops

Every op summary is indexed in seven languages (the Japanese original plus English, Simplified and Traditional Chinese, Korean, German and Hindi translations; the Hindi keeps technical terms in English, so search those in English) together with the op name, HALCON name and in/out types; results come back ranked, with the summary in your query’s language and the path of the note. The note bodies are Japanese, so a grep for “Rauschen”, “降噪” or “잡음” finds nothing — this search finds ops in your own language. No embeddings and no external service (BM25, standard library only). It does not know synonyms: if the hits look thin, rephrase or try English. If you already know the name, MCP fullseye_search_ops (substring match) is faster. Runnable example: py -3.11 examples/find_ops_by_meaning.py.

Tier 0: just open the repository (zero steps)

Open a checkout of the Fullseye repository in Claude Code and you can search and reference docs/ops/INDEX.md and the per-op notes directly. The corpus is repository content (not bundled in the wheel), so if you only pip-installed, clone the repository as well.

docs/ops/2d/<category>/<op>.md   # call form, type contract, HALCON alias, references, related ops
docs/ops/3d/<category>/<op>.md
docs/ops/INDEX.md                # whole table of contents, auto-generated by walking the folder tree
docs/ops/<family>/guides/<name>.md   # per-family how-to guides (math, diagrams, canonical citations)
docs/OP_INDEX.json               # machine-readable index of the registry

If you want to consult Fullseye while working in your own project, run the bundled setup script once:

py -3.11 tools/setup_claude_rag.py              # install (re-run = update)
py -3.11 tools/setup_claude_rag.py --uninstall  # remove

The bundled skill skills/fullseye-ops is copied to ~/.claude/skills/fullseye-ops, and the FULLSEYE_REPO = line in SKILL.md is automatically pinned to this checkout’s absolute path (so the AI knows where the corpus is no matter which project it is working in). It refuses to install on a checkout where the corpus (docs/ops) cannot be found (fail-closed).

From then on, on image-processing / geometric-vision topics Claude Code automatically invokes this skill and runs the flow: search (retrieve) docs/ops → pick ops whose types (sorts) connect and implement → verify with the bundled worked example. The skill body itself is the “how-to-use instructions for the AI”. To install it by hand, just copy skills/fullseye-ops into ~/.claude/skills/ (without the path pin, the AI looks up the repo location each time).

Tier 2 (optional): a clustered corpus — an advanced form with external tools

You can also build a “navigable corpus” that hierarchically clusters the 3,629 notes into topic clusters with an LLM summary per cluster. Internally we use the corpus2skill of a RAPTOR fork (TF-IDF + k-means + LLM summary), but this is an optional optimization, not a requirement. The only requirement is “take docs/ops as input and emit a per-cluster SKILL.md hierarchy”, so any equivalent tool can stand in.

Example of re-ingesting (after updating notes) — recorded honestly, exactly as we run it internally:

$env:RAPTOR_DIR="<path-to-raptor-checkout>"
py -3.11 raptor_corpus2skill.py --source <fullseye>/docs/ops --name fullseye_ops_corpus_v2 `
  --overwrite --max-depth 2 --max-clusters 6 --min-cluster-size 8   # needs ANTHROPIC_API_KEY

Caveat: a clustered corpus is a snapshot at ingest time. If you update docs/ops it goes stale unless you re-ingest (Tiers 0/1 never go stale because they always read the live notes).

Tier 3 (bundled): the literature layer — manufacturing knowledge folded into “which ops to use”

An op note answers “what does this op do”; it does not answer “what should I measure in this process, for this part”. docs/literature/ fills that gap: external literature corpora (mechanical design, mechatronic parts, manufacturing processes, driving simulation and evaluation, sensor simulation — about 8,900 OpenAlex metadata records; one theme → op table per field) summarised per cluster, with the ops to use in each cluster (a hand-written theme → op table whose op names are checked against the shipped notes at generation time) and the title / year / DOI of representative papers as provenance. Read it as: process, part or driving-evaluation topic → literature cluster → ops to use → op note (type contract, runnable example) → implementation. Abstracts are not copied, and ops are never picked by word overlap (that was measured to surface unrelated ops on generic words and dropped). Because the corpora live outside the repo, this layer alone is rebuilt with tools/gen_literature_notes.py --rad-root <RAD>, and tests/test_literature_notes.py guards its shape.

Optical metrology and inspection reliability (POD, laser speckle, shape from polarization, global light transport, acceptance testing and uncertainty, synthetic data: six axes) is a hand-curated layer: the papers were chosen by hand and each axis has an overview written by hand (manifest docs/literature/curated/).

What can actually reach it: the layer is retrievable by reading the repository files (the Claude Code skill fullseye-ops, checkout mode). The MCP server’s search (fullseye_search_ops / fullseye_find_ops) and fullseye-rag search do not return the layer’s clusters or papers. The hand-written context of the theme → op table (“theme: why this op”, Japanese with an English translation) is, however, part of the meaning search’s index, so a shop-floor phrasing (weld bead height, polishing removal) can find the op, and the result shows it in its contexts field. The wheel (pip install) does not ship docs/literature either.

Why this works (the design rationale)

  1. md = single source of truth: notes are deterministically auto-generated from the registry, and a CI drift test enforces “committed note == note generated from current code”. The document the AI reads and the actual code are always the same version (frontmatter version + fingerprint).
  2. Type (sort) contract: each note carries in:/out: and “related ops whose types connect”, so the AI can assemble a pipeline while type-checking it.
  3. Verifiable: every op has a ground-truth worked example (examples/ / examples_3d/), so the AI can run its own proposal and check it.
  4. End to end through to display: open Studio (py -3.11 studio.py) and a human can inspect what the AI assembled as image windows and 3-D views on the same screen (scripts can place multiple windows via dev_open_window, etc.).
This site is open source. Improve this page.