feat: FFI / C-ABI surface + Python bindings (#302)

jqueuniet commented

2026-09-04 08:00:41 +02:00

Owner

Closes #302 — a C ABI over the parser crates plus a pip-installable Python
consumer. The native counterpart to the dm2-wasm browser bindings (#301):
same thin-marshalling-layer philosophy, different boundary.

What's here

Three layers, four commits:

dm2-convert (new, pure, wasm-free) — the 14 conversion functions and
the unified error, extracted out of dm2-wasm so both binding crates can
share them without either pulling in the other's toolchain. dm2-wasm is
now a re-export shim over it; its public API (dm2_wasm::core::*,
dm2_wasm::WasmError) is unchanged and it still builds for
wasm32-unknown-unknown. Two new functions land here: resfork_list_json
and resfork_extract — exposing the Mac resource-fork walker the issue
flagged as useful beyond DM2.
dm2-ffi (new) — a cdylib/staticlib/rlib with a single stateless
return convention: Dm2Buffer { data, len, error } returned by value,
error == NULL ⇒ success, one dm2_buffer_free per buffer. 16 conversion
exports + dm2_buffer_free + dm2_version. Every export is wrapped in
catch_unwind so no panic can unwind across the C boundary. The C header
(crates/dm2-ffi/include/dm2_tools.h) is generated by cbindgen and
committed, with a drift-check test that fails if it goes stale.
bindings/python/ (new) — a Pythonic ctypes wrapper: JSON functions
take/return dict, binary functions return bytes, failures raise
Dm2Error. A hatchling build hook builds and bundles the cdylib so
pip install ./bindings/python yields a working wheel on the builder's
platform.

API surface

Stateless throughout — every function is bytes-in → bytes/JSON/bool-out, no
handles cross the boundary.

Group	C exports (Python names mirror these)
Saves	`dm2_save_to_json` / `dm2_save_from_json` (byte-exact round-trip), `dm2_save_info_json`
Dungeon	`dm2_dungeon_to_json` / `dm2_dungeon_from_json` (byte-exact round-trip), `dm2_dungeon_skproject_json`
Graphics	`dm2_graphics_header_json`, `dm2_graphics_decode_png`
Music	`dm2_music_detect`, `dm2_music_hmp_to_smf`, `dm2_music_render_mod_wav`, `dm2_music_snd_to_wav`
FTL	`dm2_ftl_info_json`, `dm2_ftl_roundtrip_ok`
Resource fork	`dm2_resfork_list_json`, `dm2_resfork_extract`

Byte order for saves/dungeon is auto-detected in dm2-convert by reusing the
crates' own try-both-endians predicates.

Proof

dm2-convert: 9 native tests (save + dungeon byte-exact round-trips through
the JSON text; graphics/music/FTL read smokes; resfork list + extract +
bad-id error).
dm2-ffi: round-trip byte-exact through the FFI buffers, the error path
returns a non-null error with null data, dm2_version non-empty, and the
committed header matches cbindgen output (drift test).
dm2-wasm still builds for wasm32-unknown-unknown unchanged (the
extraction is inert to the browser side).
Python: pytest round-trips (including an edited cur_hp surviving
reconstruct) plus the graphics/FTL/resfork reads, all against present
fixtures.

Scope / follow-ups

PyPI publishing and multi-platform prebuilt wheels are intentionally out of
scope — the repo has no CI; pip install builds the cdylib locally.
Graphics stays read-only decode (the serialization policy, #307, is a
separate effort).
The C header enables bindings for any FFI-capable language; only Python is
built here.

The .idea/dm2-tools.iml source-folder entries the IDE auto-generates for the
new crates were intentionally left out of the commits.

https://claude.ai/code/session_01NvHsv9fmmZ7EmFnk7tBZLT

Closes #302 — a C ABI over the parser crates plus a pip-installable Python consumer. The native counterpart to the `dm2-wasm` browser bindings (#301): same thin-marshalling-layer philosophy, different boundary. ## What's here Three layers, four commits: - **`dm2-convert`** (new, pure, wasm-free) — the 14 conversion functions and the unified error, extracted out of `dm2-wasm` so both binding crates can share them without either pulling in the other's toolchain. `dm2-wasm` is now a re-export shim over it; its public API (`dm2_wasm::core::*`, `dm2_wasm::WasmError`) is unchanged and it still builds for `wasm32-unknown-unknown`. Two new functions land here: `resfork_list_json` and `resfork_extract` — exposing the Mac resource-fork walker the issue flagged as useful beyond DM2. - **`dm2-ffi`** (new) — a `cdylib`/`staticlib`/`rlib` with a single stateless return convention: `Dm2Buffer { data, len, error }` returned by value, `error == NULL` ⇒ success, one `dm2_buffer_free` per buffer. 16 conversion exports + `dm2_buffer_free` + `dm2_version`. Every export is wrapped in `catch_unwind` so no panic can unwind across the C boundary. The C header (`crates/dm2-ffi/include/dm2_tools.h`) is generated by cbindgen and committed, with a drift-check test that fails if it goes stale. - **`bindings/python/`** (new) — a Pythonic ctypes wrapper: JSON functions take/return `dict`, binary functions return `bytes`, failures raise `Dm2Error`. A hatchling build hook builds and bundles the cdylib so `pip install ./bindings/python` yields a working wheel on the builder's platform. ## API surface Stateless throughout — every function is bytes-in → bytes/JSON/bool-out, no handles cross the boundary. | Group | C exports (Python names mirror these) | | --- | --- | | Saves | `dm2_save_to_json` / `dm2_save_from_json` (byte-exact round-trip), `dm2_save_info_json` | | Dungeon | `dm2_dungeon_to_json` / `dm2_dungeon_from_json` (byte-exact round-trip), `dm2_dungeon_skproject_json` | | Graphics | `dm2_graphics_header_json`, `dm2_graphics_decode_png` | | Music | `dm2_music_detect`, `dm2_music_hmp_to_smf`, `dm2_music_render_mod_wav`, `dm2_music_snd_to_wav` | | FTL | `dm2_ftl_info_json`, `dm2_ftl_roundtrip_ok` | | Resource fork | `dm2_resfork_list_json`, `dm2_resfork_extract` | Byte order for saves/dungeon is auto-detected in `dm2-convert` by reusing the crates' own try-both-endians predicates. ## Proof - `dm2-convert`: 9 native tests (save + dungeon byte-exact round-trips through the JSON text; graphics/music/FTL read smokes; resfork list + extract + bad-id error). - `dm2-ffi`: round-trip byte-exact through the FFI buffers, the error path returns a non-null `error` with null `data`, `dm2_version` non-empty, and the committed header matches cbindgen output (drift test). - `dm2-wasm` still builds for `wasm32-unknown-unknown` unchanged (the extraction is inert to the browser side). - Python: pytest round-trips (including an edited `cur_hp` surviving reconstruct) plus the graphics/FTL/resfork reads, all against present fixtures. ## Scope / follow-ups - **PyPI publishing and multi-platform prebuilt wheels are intentionally out of scope** — the repo has no CI; `pip install` builds the cdylib locally. - Graphics stays read-only decode (the serialization policy, #307, is a separate effort). - The C header enables bindings for any FFI-capable language; only Python is built here. The `.idea/dm2-tools.iml` source-folder entries the IDE auto-generates for the new crates were intentionally left out of the commits. https://claude.ai/code/session_01NvHsv9fmmZ7EmFnk7tBZLT

jqueuniet added 4 commits

2026-09-04 08:00:41 +02:00

refactor(convert): extract pure conversion layer into dm2-convert 0b7e0d443d

Moves the 14 conversion functions and the unified error out of dm2-wasm
into a new wasm-free dm2-convert crate, adds resfork_list_json /
resfork_extract, and reduces dm2-wasm to a re-export + wasm-bindgen shim.
dm2-wasm still builds for wasm32-unknown-unknown unchanged.

Refs #302

feat(ffi): dm2-ffi C ABI over the parser crates db5485c7ab

Stateless Dm2Buffer{data,len,error} return convention with one
dm2_buffer_free, 16 panic-guarded exports plus dm2_version, a
cbindgen-generated committed header, and a drift-check test.

Refs #302

feat(python): pip-installable ctypes bindings over dm2-ffi f105cc33f0

Pythonic dict/bytes API with Dm2Error, a hatchling build hook that builds
and bundles the dm2-ffi cdylib, and a pytest suite mirroring the ABI
round-trip + read proofs.

Refs #302

chore(ffi): polish from final review — wheel tag, exception uniformity, cbindgen tidy

CI / msrv (1.90) (pull_request) Successful in 46s

Details

CI / audit (pull_request) Successful in 23s

Details

CI / stable (pull_request) Successful in 1m19s

Details

c23e5347e3

- hatch_build: infer_tag so the wheel carries its platform tag
- python: wrap _pack_type encode error as Dm2Error; rename resfork_extract id param
- python: drop unused test import
- cbindgen: remove no-op [export] prefix

Refs #302

Claude-Session: https://claude.ai/code/session_01NvHsv9fmmZ7EmFnk7tBZLT

jqueuniet added 1 commit

2026-09-04 08:09:23 +02:00

chore(ide): register dm2-convert and dm2-ffi source folders in .idea module

CI / msrv (1.90) (pull_request) Successful in 47s

Details

CI / audit (pull_request) Successful in 22s

Details

CI / stable (pull_request) Successful in 1m17s

Details

CI / msrv (1.90) (push) Successful in 47s

Details

CI / audit (push) Successful in 23s

Details

CI / stable (push) Successful in 1m18s

Details

1386379ec3

Refs #302

Claude-Session: https://claude.ai/code/session_01NvHsv9fmmZ7EmFnk7tBZLT

jqueuniet merged commit 1386379ec3 into main

2026-09-04 12:49:30 +02:00

jqueuniet deleted branch feat/ffi-bindings

2026-09-04 12:49:30 +02:00

Rows
Columns

feat: FFI / C-ABI surface + Python bindings (#302) #316

What's here

API surface

Proof

Scope / follow-ups