fullseye

Fullseye Unified Interface — Requirements Specification (v0.1, 2026-08-18)

日本語 · English

Requirements specification based on the direction from 2026-08-17–18. Canonical policy = raptor memory project_fullseye_mission_unified_vision_2026_08_18, gap analysis = EVIS_VISION_OSS_GAP.md. This document defines the what and why (requirements). The how to build it (implementation) comes from design/spike after this document is agreed.

1. Background and Purpose

Fullseye = a comprehensive library that holds every image-processing/vision algorithm as a “skill” and makes it immediately usable (= a dedicated HALCON). The goal is HALCON-class coverage (measured 979/2313 = 42.3% (2026-09-06), growing HALCON_COVERAGE.md). Currently, the algorithms are split across three separate interfaces, so they cannot be called consistently from the user side (humans, Studio, evis vision, agents). The purpose of this work is to unify the interface for using them, so that every algorithm can be discovered, called, introspected, and exposed in Studio in the same natural manner.

2. Target Users and Use Scenes

User Scene Implication
The user (human) Hand-writing in REPL / scripts, using it for work ★Sample code that looks natural to a human (top priority)
Fullseye Studio Grasp/test ops and tune parameters from a GUI Operate the same op consistently from the GUI too (needs introspection/meta)
evis vision pipeline stereo→cloud→6D pose→(MoveIt2)→muscle realization Assemble perception ops with a unified I/F
Agents/automation Enumerate and execute ops programmatically Discoverable (registry), types/meta machine-readable

3. Scope

In: Put image-processing ops (currently 654 in the registry) + vision/perception ops (currently 116 in the facade) onto a unified I/F. Both hand-written numpy implementations and OSS adapters use the same I/F. Studio exposure. Unification of introspection/meta/honest gate. Out: General-purpose CS (algo-c: sort/CRC/Huffman/palindrome = 39 ops. Not image/vision knowledge = out of scope, frozen). Re-implementing OSS internals (PCL/grid_map/OpenCV/MoveIt2 sit behind thin adapters; don’t reinvent).

4. Current State and Issues (measured 2026-08-18)

Split across 3 layers and 3 conventions:

Layer Count Current call Naturalness
Image registry 654 apply(image, "gaussian", a=0.5, b=0.5) = string name + generic 2 knobs a/b ✗ Most unnatural (encoding for evolution)
algo (out of scope) 39 run_algo("name", seq) = string dispatch ✗ (but off-mission)
Perception facade 116 fs.disparity_sgm(left, right, max_disp=16, ...) = named arguments △ Relatively natural but no registry/introspection

5. Functional Requirements

6. Non-Functional Requirements

7. API Design Policy (Qt-style, natural for a human to write)

Bad example (current): apply(image, "gaussian", a=0.5, b=0.5) / run_algo("name", seq) = machine-oriented string-dispatch.

The naturalness we aim for (proposal; needs user judgment in §9):

import fullseye as fs
# Chain on a core object (image ops): reads like a sentence
edges = fs.Image.load("scene.png").to_gray().gaussian(sigma=1.4).sobel()
# Namespace module + config object + verb method (perception ops): Qt-widget style
depth = fs.stereo.SGM(max_disp=128, window=5).compute(left, right)
cloud = fs.camera.Pinhole(K).backproject(depth)
plane = fs.pcseg.PlaneRANSAC(thresh=0.01).fit(cloud)

Borrowed from Qt: namespace modules (fs.stereo/fs.camera = QtWidgets-style), config object + verb method (.compute()/.fit()/.apply()), chaining on a core object (Image), sensible defaults, and discoverable. String names / raw registry / evolution’s a/b are hidden behind the scenes.

8. Constraints and Premises

9. Needs User Judgment (want to settle before implementation)

  1. Image op manner: Chaining with fs.Image(...).sobel(), or unify on the same config-object approach as perception?
  2. Execution model: eager (immediate compute; suited to REPL/Studio), or lazy pipeline (finalized with .run())?
  3. Naming: HALCON-leaning vocabulary (dyn_threshold, etc.), or general vocabulary (adaptive_threshold)?
  4. Studio exposure: Include it in the first spike, or lead with the Python API?

10. Acceptance Criteria (the unified I/F’s “done”)

11. Phased Plan (proposal)

  1. Agree on this requirements spec (the 4 judgments in §9).
  2. Design + a small spike: Image chain + one perception module (e.g., fs.stereo) as thin wrappers over existing implementations. Additive, 0 regressions.
  3. Unify meta/registry (F2/F3) → put existing ops on it in stages.
  4. Studio exposure (F6).
  5. Prove the OSS adapter contract (F4) in one domain (stereo=image_pipeline / pcseg=PCL).
  6. Grow coverage (N4; fill gaps in the HALCON/ROS2 map with an honest gate attached).

12. Decision Log

3DGS (3D Gaussian Splatting) support — first half (data acquisition) complete 2026-08-19

Question: Can 3DGS be done with Fullseye/sim-source?

Measured environment: RTX 5090 (32GB) present / torch=2.11.0+cpu (CPU build) / CUDA toolkit · gsplat · nerfstudio not installed. → Training (GPU) is currently impossible; the stack needs setting up.

sim-source’s advantage: The biggest front stage of 3DGS = camera-pose estimation from multi-view images (COLMAP) is unnecessary in sim, because the ground-truth pose is available directly.

Implementation (GPU-free, verified on CPU): sim_source.MuJoCo

Remaining (GPU half, needs judgment): setting up the training stack. On Windows there is friction with gsplat’s CUDA build (VS build tools+CUDA toolkit). Candidates = (A) torch cu128+gsplat in a dedicated venv / (B) via WSL2 / (C) keep only the exporter and hand transforms.json to an external trainer (ns-train splatfacto). A dedicated environment is recommended to avoid affecting the shared py -3.11 env.

Second half (GPU training) measured 2026-08-19 — end-to-end success with pure-torch 3DGS

gsplat native verdict: GPU works with torch 2.11.0+cu128 (RTX 5090 / capability(12,0)=sm_120). gsplat 1.5.3 can be imported, but the CUDA kernel, on its first JIT build, gives “No CUDA toolkit found. gsplat will be disabled”. nvcc/cl absent, and no prebuilt Windows wheel for cu128 either (pt2.7/2.8/2.9/2.11 all fail). → gsplat native requires CUDA Toolkit 12.8+VS Build Tools (not installed).

Workaround = pure-PyTorch 3DGS (gsplat_torch.py): a compiler-free reference splatter (quat→R / 3D covariance→2D Jacobian projection / global-depth-sorted alpha compositing). OpenGL c2w to a CV camera via F=diag(1,-1,-1).

Remaining: making gsplat a fast backend (install CUDA Toolkit+VS Build Tools or WSL2) / densify · prune / SH color / SSIM loss. Pure torch is for PoC (non-tiled, slow at large scale).

3DGS productionization batch 2026-08-19 — trainer/real scene/CLI/Studio/article

This site is open source. Improve this page.