Skip to content

Unify pybind11 and nanobind into single-source bindings#5254

Open
soswow wants to merge 2 commits into
AcademySoftwareFoundation:mainfrom
soswow:single-file-nanobind-spike
Open

Unify pybind11 and nanobind into single-source bindings#5254
soswow wants to merge 2 commits into
AcademySoftwareFoundation:mainfrom
soswow:single-file-nanobind-spike

Conversation

@soswow

@soswow soswow commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Exploring an idea Anton had about unifiying two python backends into single source since they are so similar.

Summary

This PR unifies the early nanobind migration with the existing pybind11 Python bindings by compiling shared sources from src/python/ for both backends, instead of maintaining parallel copies under src/python-nanobind/.

Migrated modules so far: ROI, TypeDesc, ImageSpec, ParamValue / ParamValueList, plus the core OpenImageIO module helpers in py_oiio.cpp. Backend differences are isolated in py_backend.h (type aliases, binding API macros like .OIIO_PY_RW(...), and small oiio_py:: helpers). Shared Python↔C++ conversion logic lives in py_oiio.h with #if only where the backends genuinely differ.

CMake is updated so that when -DOIIO_PYTHON_BINDINGS_BACKEND=nanobind is selected, the module is built from the unified sources and installed as the main OpenImageIO package in the expected location. With both, pybind11 remains the primary module and nanobind builds as PyOpenImageIONanobind for side-by-side testing.

Duplicate nanobind-only .cpp files are removed (~1.8k lines deleted net). Testsuite coverage is extended with explicit roundtrip checks for shared helper paths (attributes, buffers, tuples).


Pros

  • Single source of truth for migrated bindings — no more keeping two copies of py_roi.cpp, py_imagespec.cpp, etc. in sync by hand.
  • Lower risk of backend drift — a fix or API tweak in a migrated module automatically applies to both pybind11 and nanobind.
  • Net code reduction — removes a large duplicated tree under src/python-nanobind/ while centralizing shared logic in one place.
  • Clearer migration pathMIGRATION_STATUS.md documents what is fully migrated, partially migrated, and still pybind-only.
  • Better nanobind-only build layout — on main, choosing nanobind does not install into the same package layout as pybind11; this PR fixes that for the migrated surface.
  • Side-by-side validationOIIO_PYTHON_BINDINGS_BACKEND=both builds both modules so existing testsuite/python-* and *.nanobind variants can be run against the same C++ sources.
  • Explicit parity tests — new/extended tests cover shared conversion paths (typed attributes, buffer→vector, tuple roundtrips) rather than relying on import smoke tests alone.
  • Maintainable binding style — method-name macros (.OIIO_PY_RW(...)) keep clang-format happy without clang-format off regions.

Cons

  • Added abstraction in the pybind11 path — even pybind-only builds now go through py_backend.h, macros, and shared helpers; slightly more indirection than plain .def_readwrite(...).
  • Regression risk for pybind11 — adapting existing pybind code to work for both backends could subtly change pybind11 behavior at the edges (init, casting, error paths, ownership).
  • Incomplete migration — most of the Python API (ImageBuf, ImageInput/ImageOutput, ImageBufAlgo, texture system, etc.) is still pybind-only; this PR is a foundation, not full nanobind coverage.
  • Known nanobind gaps remain — module-level exports and helpers listed in MIGRATION_STATUS.md are still missing or intentionally different vs pybind11.
  • Higher review and test burden — reviewers need to think about two backends; CI/local testing should cover both pybind and .nanobind variants for touched code.
  • Conditional compilation complexity#if defined(OIIO_PY_BACKEND_NANOBIND) remains in a few places (e.g. ParamValue constructors, buffer dispatch); easy to get wrong if extended carelessly.
  • Slightly less obvious binding code.OIIO_PY_RW("x", ...) reads less directly than .def_readwrite("x", ...) until you know the macro map.

@soswow soswow force-pushed the single-file-nanobind-spike branch 5 times, most recently from eb2308d to 4bf64a7 Compare June 26, 2026 23:45
soswow added 2 commits June 27, 2026 10:54
Added support for nanobind as a backend for Python bindings alongside pybind11.
Updated CMake configuration to conditionally include Python modules based on the selected backend.
Refactored existing Python binding declarations to utilize a unified interface for both backends.
Introduced a new MIGRATION_STATUS.md file to document the migration process and current status of nanobind integration.
Enhanced installation paths for nanobind modules to ensure proper isolation and organization within the build structure.
This change aims to improve flexibility in Python bindings and facilitate future development with nanobind.

Assisted-by: Cursor / auto
Signed-off-by: Aleksandr Motsjonov <soswow@gmail.com>
Refactored Python binding macros in the py_backend.h and related files to use a more consistent syntax with dot notation. This change enhances readability and maintains uniformity across the codebase. Adjusted the ImageSpec, ParamValue, ROI, and TypeDesc classes to utilize the new macro definitions. Minor formatting improvements were also made for better code clarity.

Signed-off-by: Aleksandr Motsjonov <soswow@gmail.com>
@soswow soswow force-pushed the single-file-nanobind-spike branch from 5cf873d to a78db1c Compare June 27, 2026 01:07
@soswow soswow changed the title [WIP] unify pybind11 and nanobind into single-source bindings Unify pybind11 and nanobind into single-source bindings Jun 27, 2026
@antond-weta

Copy link
Copy Markdown
Contributor

Thanks for doing this. I prefer this version to the previous one.

@soswow

soswow commented Jun 27, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for doing this. I prefer this version to the previous one.

100% I like it much more. I am still learning and I had wrong impression that they are very different

@soswow soswow marked this pull request as ready for review June 27, 2026 08:03
@soswow

soswow commented Jun 27, 2026

Copy link
Copy Markdown
Contributor Author

Btw, I've tried converting rest of the modules to nanobind in this style to see how this would go and it is surprisingly small diff:
I've created a PR in my own work against this branch just for demonstration purposes:

https://github.com/soswow/OpenImageIO/pull/1/changes

@soswow

soswow commented Jun 27, 2026

Copy link
Copy Markdown
Contributor Author

Here is a draft PR for adding more tests to improve chances of catching problems and minimize risk of regression in production pybind11 -
#5260 Please see description in it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants