jqueuniet/dm2-tools

Fork 0

feat(dungeon): lossless round-trippable JSON for dungeon.dat (#309) #313

Merged

jqueuniet merged 4 commits from feat/dungeon-json-roundtrip into main

2026-09-02 17:18:56 +02:00

jqueuniet commented

2026-09-02 16:40:45 +02:00

Owner

Closes #309 — a lossless, round-trippable JSON representation for dungeon.dat in dm2-dungeon, alongside (not replacing) the existing lossy json::skproject interop dump. Follows the owned-DTO + base64-seed + byte-exact-gate pattern established for saves (#308).

Design — self-contained, no external files

Dungeon::to_bytes reads the original bytes in exactly one place: self.bytes[block_start..] (the tile-data-block tail). Everything before block_start re-emits purely from typed owned fields. So the lossless DungeonDump embeds:

every typed section (header, maps, columns, object list, text database, per-type object records, map bodies) — individually editable, and
the tile-data-block tail as a single base64 seed (preserves inter-body padding, e.g. the FM-Towns JP map_data_size=12436 +1 byte).

reconstruct() rebuilds a synthetic vec![0; block_start] ++ seed buffer, constructs a Dungeon, and calls to_bytes(). Because to_bytes reads only [block_start..] == seed and re-emits the rest from the (round-tripped-identical) typed fields, the output is byte-exact. Unlike the saves field-driven variant, reconstruction needs no external files — the whole dungeon lives in the one JSON document.

What this adds

serde on the dungeon section graph — direct derives on all 22 owned types (FileHeader, Map, ByteOrder, Columns, ObjectList, TextDatabase, Tile, Oid, MapBody, ObjectTables, and the 13 record types), #[serde(transparent)] on the newtypes. New base64 dep.
json::lossless — DungeonDump with from_dungeon, to_pretty_json, and reconstruct. json::skproject is untouched.
Hardened reconstruct — accepts untrusted / hand-edited JSON (the web save editor #301 is a downstream): format + schema guards, a 16 MiB allocation cap on block_start, and map/body length + body-fit checks. Every failure mode returns a typed JsonError; no reachable panic or over-allocation.
Byte-exact gate (tests/json_lossless_roundtrip.rs) — the #309 acceptance criterion, over real fixtures in both byte orders.
CLI — dm2 dump <dungeon.dat> --format dungeon-lossless, a distinct JsonFormat variant (does not overload the save-only --self-contained/--full/--gdat/--dungeon flags).

Proof

Byte-exact round trip through the JSON text over every present fixture: LE 10/10 (dos-en, dos-fr, dos-de, dos-demo, dos-en-beta, ibm-psv-jp, fm-towns-jp, pc9801-jp, pc9801-jp-demo, pc9821-jp) + BE 6/6 (mac-us, mac-jp, mac-demo, amiga-eu, sega-cd-us, sega-cd-jp), plus the FM-Towns padding regression. dm2-dungeon suite 114 lib + all integration tests green; dump_save unchanged (6/6). fmt clean; no new clippy warning on changed files; no new workspace dep beyond base64.

Notes / follow-ups (non-blocking)

The seed tail duplicates the map-body bytes that typed map_bodies re-emit — a documented size tradeoff (the typed body deterministically wins via the splice), matching the self-contained precedent; a pad-only optimization is possible later but risks missing an undiscovered padding gap.
Editing a map body's dimensions (changing body.to_bytes() length) is not supported through this path without also adjusting the Map size word — a limitation inherited from to_bytes, not introduced here.

Out of scope (per #309): the #307 web serialization scheme and graphics.

Closes #309 — a lossless, round-trippable JSON representation for `dungeon.dat` in `dm2-dungeon`, **alongside** (not replacing) the existing lossy `json::skproject` interop dump. Follows the owned-DTO + base64-seed + byte-exact-gate pattern established for saves (#308). ## Design — self-contained, no external files `Dungeon::to_bytes` reads the original bytes in exactly one place: `self.bytes[block_start..]` (the tile-data-block tail). Everything before `block_start` re-emits purely from typed owned fields. So the lossless `DungeonDump` embeds: - every typed section (header, maps, columns, object list, text database, per-type object records, map bodies) — individually **editable**, and - the tile-data-block tail as a single base64 **seed** (preserves inter-body padding, e.g. the FM-Towns JP `map_data_size=12436` +1 byte). `reconstruct()` rebuilds a synthetic `vec![0; block_start] ++ seed` buffer, constructs a `Dungeon`, and calls `to_bytes()`. Because `to_bytes` reads only `[block_start..] == seed` and re-emits the rest from the (round-tripped-identical) typed fields, the output is byte-exact. Unlike the saves field-driven variant, reconstruction needs **no external files** — the whole dungeon lives in the one JSON document. ## What this adds - **serde on the dungeon section graph** — direct derives on all 22 owned types (`FileHeader`, `Map`, `ByteOrder`, `Columns`, `ObjectList`, `TextDatabase`, `Tile`, `Oid`, `MapBody`, `ObjectTables`, and the 13 record types), `#[serde(transparent)]` on the newtypes. New `base64` dep. - **`json::lossless`** — `DungeonDump` with `from_dungeon`, `to_pretty_json`, and `reconstruct`. `json::skproject` is untouched. - **Hardened reconstruct** — accepts untrusted / hand-edited JSON (the web save editor #301 is a downstream): format + schema guards, a 16 MiB allocation cap on `block_start`, and map/body length + body-fit checks. Every failure mode returns a typed `JsonError`; no reachable panic or over-allocation. - **Byte-exact gate** (`tests/json_lossless_roundtrip.rs`) — the #309 acceptance criterion, over real fixtures in both byte orders. - **CLI** — `dm2 dump <dungeon.dat> --format dungeon-lossless`, a distinct `JsonFormat` variant (does not overload the save-only `--self-contained`/`--full`/`--gdat`/`--dungeon` flags). ## Proof Byte-exact round trip through the JSON **text** over every present fixture: **LE 10/10** (dos-en, dos-fr, dos-de, dos-demo, dos-en-beta, ibm-psv-jp, fm-towns-jp, pc9801-jp, pc9801-jp-demo, pc9821-jp) + **BE 6/6** (mac-us, mac-jp, mac-demo, amiga-eu, sega-cd-us, sega-cd-jp), plus the FM-Towns padding regression. `dm2-dungeon` suite 114 lib + all integration tests green; `dump_save` unchanged (6/6). fmt clean; no new clippy warning on changed files; no new workspace dep beyond `base64`. ## Notes / follow-ups (non-blocking) - The seed tail duplicates the map-body bytes that typed `map_bodies` re-emit — a documented size tradeoff (the typed body deterministically wins via the splice), matching the self-contained precedent; a pad-only optimization is possible later but risks missing an undiscovered padding gap. - Editing a map body's **dimensions** (changing `body.to_bytes()` length) is not supported through this path without also adjusting the `Map` size word — a limitation inherited from `to_bytes`, not introduced here. Out of scope (per #309): the #307 web serialization scheme and graphics.

jqueuniet added 4 commits

2026-09-02 16:40:45 +02:00

feat(dungeon): derive serde on the dungeon section graph fdfdcf77dd

Add serde::Serialize/Deserialize to every owned section type reachable
from Dungeon (FileHeader, Map, ByteOrder, Columns, ObjectList,
TextDatabase, Tile, Oid, MapBody, ObjectTables, and all 13 record
types), with #[serde(transparent)] on the newtype wrappers. Adds the
base64 workspace dep. Groundwork for the lossless DungeonDump JSON.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D3ec7SFejDHAwmCwbapMVV

feat(dungeon): lossless self-contained DungeonDump JSON + reconstruct b97c8a1c16

Add json::lossless alongside the lossy json::skproject dump. DungeonDump
carries every typed section (editable) plus the tile-data-block tail as
a base64 seed, so reconstruct() needs no external files: to_bytes reads
the original bytes only at [block_start..], which the seed supplies while
every earlier section re-emits from typed fields. parse -> DungeonDump ->
JSON -> DungeonDump -> reconstruct is byte-exact.

reconstruct is hardened against untrusted/hand-edited JSON: format and
schema guards, a 16 MiB allocation cap, map/body length and body-fit
checks — all failure modes return a typed JsonError, never panic or
over-allocate. skproject is untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D3ec7SFejDHAwmCwbapMVV

test(dungeon): byte-exact lossless JSON round-trip gate (LE+BE) 675cbb71ec

The #309 acceptance gate: parse -> DungeonDump -> JSON text ->
DungeonDump -> reconstruct == original bytes, byte-identical, over every
real dungeon fixture in both byte orders (LE 10/10, BE 6/6), plus the
FM-Towns JP off-by-one padding regression (the whole-tail seed preserves
the stray padding byte).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D3ec7SFejDHAwmCwbapMVV

feat(cli): dm2 dump --format dungeon-lossless

CI / msrv (1.90) (pull_request) Successful in 44s

Details

CI / stable (pull_request) Successful in 1m1s

Details

CI / audit (pull_request) Successful in 17s

Details

CI / msrv (1.90) (push) Successful in 46s

Details

CI / stable (push) Successful in 1m1s

Details

CI / audit (push) Successful in 15s

Details

ff81fbe91c

Wire the lossless dungeon JSON to the CLI as a distinct JsonFormat
variant (not overloading the save-only --self-contained/--full/--gdat/
--dungeon flags). Reuses name_hints_dungeon + detect_dungeon for
auto byte-order detection and bails cleanly on non-dungeon input.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D3ec7SFejDHAwmCwbapMVV

jqueuniet merged commit ff81fbe91c into main

2026-09-02 17:18:56 +02:00

jqueuniet deleted branch feat/dungeon-json-roundtrip

2026-09-02 17:18:56 +02:00

No reviewers

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

jqueuniet/dm2-tools!313

No description provided.

Rows
Columns