Windows wheels: packaging¶
Notes captured during the GH-40 Windows-wheels-to-PyPI work. Recorded here so future maintainers don't re-walk the same multi-day diagnosis paths.
The Windows build differs from Linux/macOS in three ways that interact badly with anything resembling a plugin architecture (and tesseract is a plugin architecture top-to-bottom). Each one fails silently at packaging time and surfaces only at runtime, with an error message that points away from the real cause.
Plugin DLL transitive dependencies¶
delvewheel needs --analyze-existing to patch plugin DLLs
delvewheel repair walks the import tables of .pyd extension modules
by default — nothing else. Plugin factory DLLs (tesseract_*_factories.dll,
tesseract_kinematics_opw_factory.dll, …) that the build pipeline copies
into the wheel's package dir before the repair step do get included
in the wheel, but their import tables stay unpatched and any deps they
pull in beyond what the .pyd files already need go un-bundled.
The fix is delvewheel repair --analyze-existing … — that flag tells
delvewheel to scan and rewrite every DLL in the wheel, not just .pyd.
Without it, the wheel builds, installs, smoke-imports, and then fails at
first plugin instantiation. The misleading part: every test that touches
a plugin (kinematics, collision, task-composer pipelines) fails with
"Failed to load symbol", which sends the diagnosis chasing
__declspec(dllexport) and CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS. Those
aren't the bug.
This applied up to 0.35.0.x, when the pipeline copied the plugin DLLs into
the package. The collision, kinematics and task composer modules now link
the factory DLLs (see binaries), so
delvewheel repair reaches them through the .pyd import tables and
wheels.yml runs it without --analyze-existing.
Runtime DLL search path¶
os.add_dll_directory does not cover plain LoadLibrary from C++
os.add_dll_directory() (which delvewheel auto-injects into __init__.py
for the <project-name>.libs/ directory) only affects LoadLibrary calls
that pass LOAD_LIBRARY_SEARCH_USER_DIRS. CPython's .pyd loader passes
that flag, so add_dll_directory "works" for module imports — which
builds false confidence that it covers the whole process.
Once a .pyd is loaded and its C++ code calls LoadLibrary("plugin.dll")
itself — which is exactly what boost::dll::shared_library does, and
therefore what boost_plugin_loader does — that C++ call uses the
legacy Windows DLL search order, which respects PATH but ignores
add_dll_directory. Up to 0.35.0.x the fix was to prepend the package dir
and <project-name>.libs/ to PATH in __init__.py before any submodule
import. That block is gone: the plugin anchors make boost_plugin_loader open
each linked factory DLL by its resolved path (see
binaries), and importing tesseract_robotics
no longer changes PATH.
Note the directory naming asymmetry that bites here: delvewheel uses the
project's metadata name (tesseract-robotics-nanobind) for the libs
dir, not the Python package name (tesseract_robotics). It is named
tesseract_robotics_nanobind.libs/ on disk.
Diagnosing plugin-load failures¶
boost_plugin_loader's 'Failed to load symbol' is two errors in a trench coat
boost::dll-backed plugin loaders log
Failed to load symbol 'X' in library: Y in two distinct cases that look
identical from Python:
LoadLibrarysucceeded butGetProcAddressreturnedNULL— the symbol genuinely isn't in the DLL's export table.LoadLibraryreturnedNULLbecause a transitive dependency couldn't be resolved — the loader catches the boost exception and re-reports it as "Failed to load symbol".
On Windows, case 2 is far more common than case 1 and points the
investigation in a completely different direction (DLL search path,
bundled deps, delvewheel --analyze-existing) than case 1 would
(__declspec(dllexport), symbol visibility flags). Always confirm with
dumpbin /EXPORTS plugin.dll whether the symbol is actually present
before assuming an export issue. CI does not run this check; run it by
hand from a Visual Studio developer prompt on the DLL inside the wheel.
Cereal polymorphic registration¶
Cereal's polymorphic registry is per-DLL on Windows MSVC — consumers must re-register types in their own TU
cereal stores its polymorphic-type registry in class-template statics
(StaticObject<InputBindingMap<Archive>>,
StaticObject<OutputBindingMap<Archive>>, ...). Linux and macOS linkers
dedupe these template instantiations across shared libraries, giving the
process one shared registry. MSVC does not — each DLL gets its own
copy of every template static, with no automatic dllexport/dllimport
plumbing in cereal headers to share them.
Consequence: an upstream library (e.g. tesseract_command_language.dll)
that calls CEREAL_REGISTER_TYPE populates its own registry only.
A downstream .pyd that does serialization reads from its own empty
registry and fails at runtime with
RuntimeError: Trying to save an unregistered polymorphic type. The
type really is registered upstream — just not in the registry the
consumer actually consults.
CEREAL_FORCE_DYNAMIC_INIT is not enough on its own. It anchors
upstream's registration TU to keep the linker from stripping it, but
doesn't bridge the registry gap on the consumer side.
The actual Windows fix is to mirror upstream's registration block in
the consumer TU. For tesseract_command_language types
(MoveInstructionPoly, CompositeInstruction, etc.) that means copying
every CEREAL_REGISTER_TYPE and CEREAL_REGISTER_POLYMORPHIC_RELATION
line from upstream
tesseract_command_language/src/cereal_serialization.cpp into our
binding. Re-registration is a no-op on Linux/macOS (cereal tolerates
duplicate registrations of the same {type, archive} pair) and
populates the consumer's per-DLL registry on Windows. See
src/bindings/serialization.cpp for
the canonical placement; keep CEREAL_FORCE_DYNAMIC_INIT(LibName) as
belt-and-suspenders on the producer side.
Shared OMPL state¶
vcpkg's standard OMPL 1.7 port builds a static library with MSVC. Linking that
library into both the binding and the native planner duplicates OMPL's global
RNG seed generator. RNG_setSeed then seeds the binding's copy while the native
planner continues using a separate seed. A repeatable seed is not effective
across that DLL boundary.
The Windows-only overlay keeps OMPL's planner library static and moves its RNG
implementation into ompl_runtime.dll, together with its logging, hyperspheroid
and geometric-equation dependencies. Keeping these together avoids a circular
link dependency. Automatically exporting all of OMPL exceeds
MSVC's 65,535-symbol limit; automatic exports cover only this small runtime.
Linux/macOS retain their existing port. The wheel workflow compiles
two small DLL consumers and checks that setting the seed through one is visible
through the other; the ordinary planning tests still run on the repaired wheel.
The source-download checksum in the overlay is the upstream vcpkg port's required
download-verification field.