Developer guide¶
For contributors: building the package, changing a binding, releasing wheels. A user of the package needs none of this; start at Getting started. This page covers setup, the pixi tasks, tests and the repository layout; the pages below go deeper.
| Page | For | Covers |
|---|---|---|
| The native stack build | anyone building from source | the one script that builds the C++ stack, for CI and development |
| How the binaries are built | release engineers | every binary from upstream source to the PyPI wheel |
| Wheel build caching | CI maintainers | the cache layers of the wheel workflow |
| Building against upstream main | binding authors tracking upstream | the upstream-main branch and the API changes it absorbs |
| Linux, macOS, Windows wheels | release engineers | platform traps met while shipping wheels |
| Binding conventions | binding authors | names, errors, lifetimes and the GIL, rule by rule |
| Binding API audit | binding authors | generated diff of each binding against its C++ API |
| Binding audit coverage | binding authors | which modules the audit covers and who owns each open row |
| Threading and the GIL | binding authors, users of threads | which calls release the GIL and why |
| TrajOpt-Ifopt constraints | binding authors | what the constraint bindings do beyond a one-to-one wrap |
| TrajOpt collision memory | anyone planning long trajectories | the per-waypoint contact manager copies and their memory cost |
| SWIG to nanobind | users of the SWIG tesseract_python |
what changed from the SWIG bindings |
Pixi Workspace¶
This project uses pixi exclusively for package management. Pixi manages the Python packages, build tools and platform-specific dependencies through a single lockfile (pixi.lock); the C++ stack comes from scripts/native_stack.py. No pip, conda, poetry, or venv.
The tesseract C++ stack is built by scripts/native_stack.py (pixi run native-build), the script the wheel workflow runs, so a development build and a wheel compile the same sources. The native stack describes it; How the binaries are built traces every binary from the upstream source to the PyPI wheel.
First-Time Setup¶
Install pixi (docs), then clone and build:
curl -fsSL https://pixi.sh/install.sh | bash
git clone --recurse-submodules https://github.com/tesseract-robotics/tesseract_nanobind.git
cd tesseract_nanobind
pixi run build
pixi run build resolves the environment, builds the C++ stack into .native/<triplet>/ (skipped when its inputs are unchanged), then compiles the nanobind extensions against it.
Available Tasks¶
| Task | Description | What it does |
|---|---|---|
native-build |
Build the C++ stack | scripts/native_stack.py all into .native/<triplet>/; no-op when unchanged |
build |
Build bindings | alias for install |
install |
Install bindings | native-build, then editable pip install -e . --no-build-isolation against .native/<triplet>/ws/install |
test |
Run tests | pytest with xdist parallel, excluding benchmarks (depends on install) |
test-benchmark |
Run benchmarks | benchmark tests, serial |
typecheck |
Type check | ty on src/tesseract_robotics/ |
lint |
Lint | ruff check |
fmt |
Format | ruff format |
stubs |
Regenerate stubs | the committed .pyi stubs; see Stub Generation |
stubs-check |
Check stubs | fails if committed stubs differ from the built modules |
docs (-e docs) |
Live docs | zensical serve with auto-reload |
docs-build (-e docs) |
Build docs | Static site to site/ (zensical build --strict) |
demo-record, demo-posters, demo-check |
Demo recordings | see Demo Recordings |
demo-players, demo-fetch, demo-check-site (-e docs) |
Demo files in the docs | see Demo Recordings |
docs-dotgraphs |
Pipeline graphs | regenerates the pipeline DOT graph SVGs (requires graphviz) |
check-changelog |
Check changelog | CHANGELOG.md release structure (the Docs workflow runs it too) |
audit-gil (-e audit) |
GIL audit | Every GIL release in the bindings against scripts/nogil_entry_points.yaml; see Threading and the GIL |
audit-test (-e audit) |
Audit contract tests | tests/audit, which needs libclang |
gil-hotspots |
GIL hotspots | Records benchmarks and tests with py-spy --gil, ranks the bindings that hold the GIL (Finding candidates) |
Running Tests¶
pixi run test # full suite, parallel
pixi shell && pytest tests -x --testmon # incremental (only changed)
Every example is a test¶
A full test session fails when it leaves an example unexecuted, and the failure names the file (#445). Examples are what a new user runs first; an example no test runs can break unnoticed.
Nothing needs registering. The gate, tests/examples/example_coverage.py, finds every module
under tesseract_robotics.examples (subpackages included) at the end of the session and checks
which of their functions ran:
- A module that defines
mainis an example: itsmain()orrun()must have run. - Any other module (
online_planning_scene,lowlevel/example_support) is a helper: at least one of its functions must have run. Importing a module does not count.
To add an example, put the file in src/tesseract_robotics/examples/ and write a test in
tests/examples/ that calls its run() and asserts on what it returns: a valid trajectory
(tests/planning_validity.py), contact distances, FK poses. Without the test, the next full run
ends with:
=========================== example coverage (#445) ============================
1 example module(s) not executed by the test suite. ...
.../src/tesseract_robotics/examples/my_new_example.py: main() / run() never ran
The gate judges only a session that collected every test_*.py under tests/examples and
deselected none of its tests. pixi run test and CI's test-artifacts job are judged; a single
file, a node id, -k, -m, --lf or --testmon is not, so targeted runs don't fail on examples
they did not ask for. To run the gate locally:
A passing judged run prints example coverage: all N example modules executed. Recording uses
sys.monitoring on a free tool id, so it runs alongside coverage tools and debuggers. Under xdist
each worker sends what it recorded to the controller, which judges. With
TESSERACT_TESTS_WITHOUT_TWC=1 the TWC viewer test skips, so the gate names
twc_workcell_positioner_viewer.py; that is the opt-out's cost, shown on purpose.
Documentation snippets¶
tests/docs/test_getting_started_snippets.py runs every python fence of the README, the docs
home and the getting-started, concepts, user-guide, API and examples pages, one page per
subprocess, the blocks of a page in order. A broken snippet fails CI.
- The getting-started and concepts pages, README and home run bare, as a reader pasting them
would. The home page shows the README's Quick Start block;
test_readme_structure.pykeeps the two identical, and the README's copy is the one that runs. - A reference page starts with
npand thelocator,robotandenvof the IRB2400. Names a page shows in isolation (program,problem,factory) come from an invisible<!-- snippet: ambient ... -->comment at the top of the page; helpers that build them live intests/docs/doc_assets.py, which also writes the small mesh, texture and VDB files the snippets read, into a scratch directory. - A block that only includes a shipped example (
--8<--) is the example's source, which the example tests run. <!-- snippet: skip (reason) -->on the line above a fence skips it; the reason is required. Use it for a browser viewer, a robot controller or a path only the reader has.
tests/docs/test_examples_gallery.py checks that the examples page lists every example console
script and shows a card for every published demo recording.
Daily Workflow¶
# First time setup (builds the native C++ stack, compiles the bindings)
pixi run build
# Run tests
pixi run test
# Run a single test file
pixi run python -m pytest tests/trajopt_sqp/test_trajopt_sqp_bindings.py -v
# Run only tests affected by recent changes
pixi run python -m pytest --testmon
# Run an example
pixi run tesseract_freespace_ompl_example
# or
pixi run python -m tesseract_robotics.examples.freespace_ompl_example
# Interactive shell (all env vars set)
pixi shell
python -c "from tesseract_robotics.planning import Robot; print('ok')"
Rebuild After C++ Changes¶
If you modify a C++ binding file (src/bindings/<module>.cpp):
To work against another upstream C++ commit, change dependencies.rosinstall and run
pixi run build (The native stack build).
Task Dependency Chain¶
Tasks use depends-on for ordered execution:
flowchart LR
test --> install
build --> install
Running pixi run test reinstalls the bindings first, since it depends on install.
Environments¶
The default environment is for local development; py312 runs the suite on the 3.12 floor
(CI builds the wheels without pixi, see binaries):
# Default environment (Python from pixi.lock)
pixi run test
# The Python 3.12 environment
pixi run -e py312 test
Environments defined in pyproject.toml:
| Environment | Python | Usage |
|---|---|---|
default |
from lockfile | Local development |
py312 |
3.12.x | Suite on the 3.12 floor |
py315 |
3.15.x | Suite on the newest Python |
How the binaries are built lists where each environment gets its C++.
Dependency Management¶
All dependencies live in pyproject.toml under [tool.pixi.dependencies] (conda packages) and [tool.pixi.pypi-dependencies] (PyPI packages).
# Add a conda dependency
pixi add some-package
# Add a PyPI dependency
pixi add --pypi some-package
# Update lockfile after editing pyproject.toml manually
pixi install
# See what's installed
pixi list
The C++ stack is not a pixi dependency: it is pinned by the commits in dependencies.rosinstall and the vcpkg revision and packages in scripts/native_stack.toml.
Worktrees¶
Each git worktree needs its own .native/<triplet>/ (a full native build) and its own pixi environment. Run builds through pixi run, which uses the worktree's environment; a raw pip install from a shell whose CONDA_PREFIX points at another checkout's environment builds against the wrong toolchain.
Building Portable Wheels¶
Editable installs bake absolute paths, so they aren't portable. The PyPI wheels are
built without conda by .github/workflows/wheels.yml: the tesseract C++ stack is built
from source with scripts/native_stack.py (colcon against vcpkg dependencies), and auditwheel, delvewheel or
a batched Mach-O repair bundles it. See binaries.
Architecture¶
graph TD
subgraph "C++ Libraries"
A[tesseract_environment]
B[tesseract_kinematics]
C[tesseract_collision]
D[tesseract_planning]
end
subgraph "src/bindings/ (nanobind sources)"
E[environment.cpp]
F[kinematics.cpp]
G[collision.cpp]
H[motion_planners.cpp]
end
subgraph "tesseract_robotics (Python modules)"
E2[environment]
F2[kinematics]
G2[collision]
H2[motion_planners]
end
subgraph "High-level API"
I[tesseract_robotics.planning]
J[Robot, MotionProgram, TaskComposer,<br/>plan_freespace / plan_ompl / plan_cartesian]
end
A --> E --> E2
B --> F --> F2
C --> G --> G2
D --> H --> H2
E2 --> I
F2 --> I
G2 --> I
H2 --> I
I --> J
The C++ libraries keep their upstream names (tesseract_collision); the Python modules drop the
tesseract_ prefix (tesseract_robotics.collision, issue #42). The old names are gone; there are
no aliases (module names).
Module Layout¶
Each src/bindings/<module>.cpp builds one nanobind extension. Two install shapes:
graph LR
subgraph direct["Direct install (16 modules)"]
D1["src/bindings/collision.cpp"] --> D2["tesseract_robotics/collision.abi3.so"]
D3["src/tesseract_robotics/collision.pyi"] -.stub.-> D2
end
subgraph wrapped["Wrapped package (6 modules)"]
W1["src/bindings/common.cpp"] --> W2["tesseract_robotics/common/_common.abi3.so"]
W3["tesseract_robotics/common/__init__.py"] --> W2
end
- Direct install:
collision,command_language,environment,geometry,kinematics,motion_planners_ompl,motion_planners_trajopt,serialization,srdf,state_solver,task_composer,task_composer_planning,time_parameterization,urdf,trajopt_ifopt,trajopt_sqp. The extension is the module;<module>.pyibeside it is its stub. There is no__init__.pyre-export wrapper. - Wrapped package: a Python
__init__.pyaround a compiled_<module>, because the module holds Python code.common(axis constants),scene_graph(lazySceneStatere-export) andmotion_planners,motion_planners_simple,motion_planners_descartes,motion_planners_trajopt_ifopt(profile-override guards). - Sources that belong to a module but are not its entry point sit beside it in
src/bindings/, prefixed with the module name (serialization_planner_cereal_registration.cpp).
Project Layout¶
graph LR
root["tesseract_nanobind/"]
root --> cfg["pyproject.toml, pixi.lock, CMakeLists.txt"]
root --> src["src/"]
root --> tests["tests/"]
root --> scripts["scripts/ (stubs, GIL audit, native stack build)"]
root --> up["upstream/ (tesseract, trajopt, tesseract_planning submodules)"]
root --> pk["packaging/ (the support-data wheel)"]
src --> pkg["tesseract_robotics/ (Python package, .pyi stubs, direct-install modules land here)"]
src --> bind["bindings/ (one <module>.cpp per extension)"]
src --> hdr["tesseract_nb.h (shared precompiled header)"]
pkg --> pl["planning/, viewer/, emitters/ (pure Python)"]
pkg --> wr["common/, scene_graph/, motion_planners*/ (wrapped packages)"]
Cross-Module Type Resolution¶
nanobind maintains separate type registries per module. When a function returns a type from another module, that module must be imported first:
NB_MODULE(_my_module, m) {
// Import module that defines the type BEFORE using it
nb::module_::import_("tesseract_robotics.collision");
// Now can return DiscreteContactManager from functions
.def("getContactManager", [...] { return self.getDiscreteContactManager(); })
}
See Migration Notes for details.
Type Checking¶
ty is the type checker of record: pixi run typecheck runs ty check on src/tesseract_robotics, the pre-push hook runs it, and CI runs it on Linux x86_64 against the built wheel. The whole package, examples and the generated .pyi stubs included, reports no diagnostics; a stub that ty rejects is fixed at the binding (nb::sig, named arguments, type registration), not ignored.
When adding new bindings, declare inheritance in nanobind to get correct stubs:
// Good - generates: class Derived(Base)
nb::class_<Derived, Base>(m, "Derived")
// Bad - generates: class Derived (no inheritance)
nb::class_<Derived>(m, "Derived")
Stub Generation¶
After modifying C++ bindings, regenerate the .pyi stubs:
This finds every nanobind extension module in the installed package and writes each
stub to src/tesseract_robotics/<module>.pyi (<module>/_<module>.pyi for the wrapped packages). Each module is rendered by
nanobind.stubgen in a fresh interpreter, because which cross-module types resolve
depends on what is already imported. Stubs are committed for IDE support and type
checking.
Stubs are generated, never hand-edited. Edits made by hand are lost at the next regeneration. Put documentation in the binding's docstring instead.
Drift gate (gh-159). pixi run stubs-check exits 1 when a committed stub differs
from what the build renders. CI runs it in every build job, on all platforms. Stubgen output depends on the nanobind version, so nanobind is pinned
exactly in pyproject.toml. Bumping nanobind means regenerating the stubs in the same
commit.
No std:: in stubs. stubgen writes a C++ type it can't map to Python as a quoted
C++ name. Such a type has no binding or converter, so it also fails at runtime. Spellings
of std:: types differ by platform (libc++ std::__1::, libstdc++ std::__cxx11::),
so such a leak also makes the stub platform-dependent. tests/scripts/test_generate_stubs.py
rejects them. Quoted tesseract::… names are the same kind of gap but
platform-independent. Each one means the type isn't bound, or the module doesn't
nb::module_::import_() the module that binds it.
nanobind's std::unordered_map caster matches default-allocator maps only.
src/tesseract_nb.h adds a caster for Tesseract's Eigen::aligned_allocator maps
(TransformMap and friends).
Pre-commit Hooks¶
| Hook | Stage | What it does |
|---|---|---|
| ruff check --fix | pre-commit | Auto-fix lint issues |
| ruff format | pre-commit | Format code |
| stage-formatted | pre-commit | Auto-stage ruff changes |
typecheck (pixi run typecheck, ty) |
pre-push | Type check |
Skip when needed: git commit --no-verify / git push --no-verify
Demo Recordings¶
The players on the example pages are static files, with no Python server behind them. car_seat leads the README and the docs home. Status: local playback, scrubbing and orbiting are confirmed by demo-check in headless Chrome on a Mac; the deployed site is under verification until demo-check --url has run against it. Phones and other browsers are untested.
scripts/demos/manifest.toml holds one entry per viewer example, published or excluded with a reason. published means a person looked at the recording; a test fails when a viewer example has no entry.
pixi run demo-record --only MODULE # docs/demos/{recordings,client}; updates the lock, drops that poster
pixi run demo-posters # a poster for each recording without one; --only MODULE re-captures
pixi run -e docs demo-players --preview
env PYTHONPATH=src pixi run -e docs zensical build --strict
pixi run demo-check --preview # drive every player in Chrome
To add a demo: give it a camera in the manifest and a builder in scripts/demos/scenes.py, record it, capture its poster, and add --8<-- "docs/demos/players/MODULE.html" under its heading on its page. After review, set status = "published" and delete reason.
Hosting. Recordings stay out of git, since each re-record would add its full size to history. The demo-assets release holds each file named by its SHA-256, and scripts/demos/demos.lock.json pins the hashes, so a re-record adds assets and older commits still build. To publish: pixi run demo-fetch --stage DIR, gh release upload demo-assets DIR/* --clobber, which is safe because the names are hashes. The tag has no digits on purpose: the wheel workflow and setuptools_scm act only on numeric tags. Upload before pushing: a file the release does not hold, or a published entry the lock has no recording or poster for, is a ::warning:: and the page builds without that player or poster (#446). Only a download whose bytes differ from the lock fails the build. Pages serves its own copies, which the build places in the site; nothing links to the release.
Docs figures. pixi run docs-figures renders the SVGs under docs/assets/figures/ and records their inputs, the upstream SHAs and the output hashes in figures.lock.json. --check (and tests/docs/test_figures_reproducible.py) fails only on structure: the lock and the committed SVGs name the same figures, and every SVG, committed or freshly rendered, is well-formed and draws. Drift from the lock (an edited script or figure, an upstream bump, other bytes) is printed as stale and never fails CI (#446); re-run pixi run docs-figures when you want the record current.
Things that are not obvious:
- A re-record is a new recording, because the planners are randomized. It needs a new review.
- A floor's position and size are baked in at record time. A camera is only a query parameter, so tuning one needs a new poster and no re-record.
- The docs builder rewrites
<img src>relative to the source.mdand leavesdata-srcalone, which is whyplayers.pykeeps two prefixes. Neither is checked at build time;demo-check-siteis the guard. demo-checkserves from an in-process server with a large listen backlog.http.server's default of 5 dropped 12 requests in 30 loads ofplanning.md, which asks for 10 posters at once.- glTF draws a material opaque whatever its alpha, unless
alphaModesaysBLEND. The viewer sets it for any alpha below 1. - Once a player is open, the scroll wheel zooms the scene and the page stays put. A reader scrolling past a closed player is unaffected.
Changelog at Release Time¶
CHANGELOG.md follows Keep a Changelog and is
published on the docs site. scripts/check_changelog.py checks its release
structure; the Docs workflow runs it before building, so a mis-cut release fails
CI instead of publishing a wrong changelog (0.35.0.7 and 0.35.0.8 were both cut
by hand, and both broke it). To cut <version>:
- Rename
## [Unreleased]to## [<version>] - <YYYY-MM-DD>and add a new, empty## [Unreleased]above it. - Add
[<version>]: https://github.com/tesseract-robotics/tesseract_nanobind/compare/<previous>...<version>above the previous release's link. - Point
[Unreleased]:atcompare/<version>...HEAD. - Run
pixi run check-changelogbefore committing.
Anything merged after the tag goes under ## [Unreleased], not into the
release's section.
Contributing¶
- Fork the repository
- Create a feature branch (or use a git worktree)
pixi run build(first time)- Make changes
pixi run testpixi run typecheck- Submit a pull request