jqueuniet/dm2-tools

Fork 0

feat(saves): field-driven full save-JSON variant (#308 Plan 2) #312

Merged

jqueuniet merged 7 commits from feat/save-json-field-driven into main

2026-09-02 08:56:30 +02:00

jqueuniet commented

2026-08-31 16:16:01 +02:00

Owner

Completes issue #308 (lossless round-trippable save JSON) — the field-driven "full" variant (Plan 2), building on the merged self-contained variant (Plan 1, #311).

What this adds

serde on the decoded record_stream graph — direct derives on the 8-type graph (RecordStream, CheckcodeRecord, TileChain, …), spec-sanctioned.
bitstream_pad capture — the field-driven variant stores only the bitstream tail (from phase7_end_bit/8), not the whole bitstream; mutually exclusive with the self-contained bitstream.
SaveDump::full(&SaveFileFull, variant) — forward constructor for the field-driven dump.
SaveDump::reconstruct(map_info, gdat) — public reverse method: re-encodes phases 1–7 from decoded fields via to_bytes_from_fields, feeding a synthetic bitstream that carries only the pad.
byte-exact losslessness gate (tests/json_field_roundtrip.rs) — parse → decode_full → SaveDump::full → JSON → deserialize → reconstruct → assert byte-identical.
CLI: dm2 dump <save> --full --gdat <graphics.dat> --dungeon <dungeon.dat>, with flag validation (--full requires both; mutually exclusive with --self-contained).

Proof

Byte-exact round-trip verified over live fixtures: DOS slots 0–3 (4/4) and Mac saves (14/14) byte-identical, plus a gameplay-edit-reflected test. Full suite green: dm2-saves lib 183 passed, dm2-cli dump_save 6/6, json_field_roundtrip 3/3.

Coverage bound (by design)

Field-driven byte-exactness holds for saves whose record stream fully decodes (tile_chains non-empty ∧ champion_inventory.len() == heros_in_party) — the engine's own field-encode ceiling. Other saves remain covered by the self-contained variant.

Robustness

reconstruct deserializes untrusted JSON (web save editor #301/#307 downstream); hardened against malformed input — empty pad and oversized phase7_end_bit return JsonError (16 MB cap) rather than panicking or over-allocating.

Follow-up (documented, non-blocking): record_stream.raw_arrays round-trips but raw.raw_record_arrays is authoritative on reconstruct — candidate to skip-serialize for the web editor.

Completes issue #308 (lossless round-trippable save JSON) — the field-driven "full" variant (Plan 2), building on the merged self-contained variant (Plan 1, #311). ## What this adds - **serde on the decoded `record_stream` graph** — direct derives on the 8-type graph (`RecordStream`, `CheckcodeRecord`, `TileChain`, …), spec-sanctioned. - **`bitstream_pad` capture** — the field-driven variant stores only the bitstream tail (from `phase7_end_bit/8`), not the whole bitstream; mutually exclusive with the self-contained `bitstream`. - **`SaveDump::full(&SaveFileFull, variant)`** — forward constructor for the field-driven dump. - **`SaveDump::reconstruct(map_info, gdat)`** — public reverse method: re-encodes phases 1–7 from decoded fields via `to_bytes_from_fields`, feeding a synthetic bitstream that carries only the pad. - **byte-exact losslessness gate** (`tests/json_field_roundtrip.rs`) — parse → decode_full → SaveDump::full → JSON → deserialize → reconstruct → assert byte-identical. - **CLI**: `dm2 dump <save> --full --gdat <graphics.dat> --dungeon <dungeon.dat>`, with flag validation (`--full` requires both; mutually exclusive with `--self-contained`). ## Proof Byte-exact round-trip verified over live fixtures: **DOS slots 0–3 (4/4) and Mac saves (14/14) byte-identical**, plus a gameplay-edit-reflected test. Full suite green: dm2-saves lib 183 passed, dm2-cli dump_save 6/6, json_field_roundtrip 3/3. ## Coverage bound (by design) Field-driven byte-exactness holds for saves whose record stream fully decodes (`tile_chains` non-empty ∧ `champion_inventory.len() == heros_in_party`) — the engine's own field-encode ceiling. Other saves remain covered by the self-contained variant. ## Robustness `reconstruct` deserializes untrusted JSON (web save editor #301/#307 downstream); hardened against malformed input — empty pad and oversized `phase7_end_bit` return `JsonError` (16 MB cap) rather than panicking or over-allocating. Follow-up (documented, non-blocking): `record_stream.raw_arrays` round-trips but `raw.raw_record_arrays` is authoritative on reconstruct — candidate to skip-serialize for the web editor.

jqueuniet added 7 commits

2026-08-31 16:16:01 +02:00

feat(saves): derive serde on the RecordStream graph 9563183876

feat(saves): bitstream_pad helpers + field-driven RawSectionsJson::from_full e7760b3a17

Add bitstream_tail_pad/synthesize_bitstream (pure, symmetric inverse pair)
and RawSectionsJson::from_full, which captures every structural section
plus only the sub-byte bitstream tail pad instead of the full bitstream.
These are not yet wired into a public constructor (a later task adds the
field-driven SaveDump + reconstruction path), so they carry
#[allow(dead_code)] for now, following the existing convention in
dm2-cli/src/commands/save/mod.rs.

feat(saves): SaveDump record_stream field + full() constructor e5eacbe3e6

feat(saves): SaveDump::reconstruct + byte-exact field-driven gate 182cf656dc

Adds the reverse method for the field-driven full save-JSON variant:
reconstruct(map_info, gdat) rebuilds a SaveFileFull from JSON, feeding
a synthesized bitstream that carries only the stored bitstream_pad,
and calls SaveFileFull::to_bytes_from_fields to re-encode phases 1-7.

The new byte-exact gate (json_field_roundtrip.rs) proves this is
lossless: all 4 DOS retail save slots and all 14 Mac retail saves
round-trip byte-identical through dump -> JSON -> reconstruct.

Also removes the now-obsolete #[allow(dead_code)] on
synthesize_bitstream, which reconstruct is the first real caller of.

fix(saves): harden reconstruct against malformed JSON + Mac gate-honesty guard 486f415b61

synthesize_bitstream deserializes JSON that may be attacker-controlled
(the web save editor is an explicit downstream), so it must not panic
or over-allocate on malformed input:

- Empty bitstream_pad (with phase7_end_bit < 8) previously panicked
  via `.expect("pad is non-empty")`. synthesize_bitstream is now
  fallible and returns JsonError::BadLength instead.
- An unbounded phase7_end_bit previously drove an unbounded
  `vec![0u8; phase7_end_bit / 8]` allocation. Added MAX_BITSTREAM_BYTES
  (16 MiB, ~1000x headroom over real saves) and reject before
  allocating.

Also fixes the Mac field-driven round-trip test, which discarded its
per-save result and had no aggregate assertion: it could pass while
proving nothing if no Mac fixture took the field path. Mirrors the
DOS test's `asserted >= 1` guard.

Verified the byte-exact gate still round-trips all 4 DOS slots and
all 14 Mac saves byte-identical after these changes.

feat(cli): dm2 dump --full field-driven save JSON 8235c2fce2

fix(saves): final-review nits — robust Mac gate, symmetric WrongVariant test, raw-array doc

CI / msrv (1.90) (pull_request) Successful in 41s

Details

CI / audit (pull_request) Successful in 19s

Details

CI / stable (pull_request) Successful in 1m3s

Details

CI / msrv (1.90) (push) Successful in 38s

Details

CI / audit (push) Successful in 22s

Details

CI / stable (push) Successful in 1m7s

Details

9fdafd64ab

jqueuniet merged commit 9fdafd64ab into main

2026-09-02 08:56:30 +02:00

jqueuniet deleted branch feat/save-json-field-driven

2026-09-02 08:56:30 +02:00

No reviewers

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

jqueuniet/dm2-tools!312

No description provided.

Rows
Columns