dungeon.dat: editable self-contained JSON for the web (#307) #390

Closed
opened 2026-10-28 06:23:40 +01:00 by jqueuniet · 3 comments
Owner

Dungeon slice of #307.

Findings

  • Scheme: self-contained JSON fits. dungeon.dat has no binary payloads; the lossless dump of a 39 KB retail file is 547 KB pretty-printed and ~270 KB compact. No sidecars needed.

  • tile_block_seed is redundant. Over all 16 fixtures:

    • The map bodies tile the tile-data block in map order, with no gaps or overlaps. The block size is map_data_size.
    • The file ends with a u16 trailer: the sum of all preceding bytes mod 0x10000, in file byte order. DMDC2's CSerializedWriterWithChkSum writes the same sum.

    The seed duplicates the bodies and freezes the trailer. Consequences:

    • Any typed edit reconstructs with a stale checksum.
    • Resizing a map fails with BodyOutOfRange.
  • Packed fields are opaque to clients. Map carries map_size_word, flags_word and num_graphs_word; tiles and records are raw words. The web editor's dungeon browser therefore uses the lossy dungeonSkprojectJson and re-decodes tile bytes itself (byte >> 5). Editing has no path.

Plan

  1. Schema v2: drop tile_block_seed. Reconstruct writes the bodies at map_offset, requires exact tiling, and recomputes the trailer. Parse exposes the stored checksum and whether it is valid.
  2. Decoded projection: emit decoded fields (map size, level, graphics counts, tile type and attributes, record fields) instead of packed words, and pack them back on reconstruct. Still byte-exact over the fixtures.
  3. Binding parity (wasm, FFI, Python), CLI, docs.

Open

  • Does the engine verify the trailer? Settle it in-game with a dungeon whose checksum is wrong.

Acceptance

parse -> JSON -> reconstruct is byte-identical over every fixture. An edited dump reloads in-game.

Dungeon slice of #307. ## Findings - **Scheme:** self-contained JSON fits. `dungeon.dat` has no binary payloads; the lossless dump of a 39 KB retail file is 547 KB pretty-printed and ~270 KB compact. No sidecars needed. - **`tile_block_seed` is redundant.** Over all 16 fixtures: - The map bodies tile the tile-data block in map order, with no gaps or overlaps. The block size is `map_data_size`. - The file ends with a u16 trailer: the sum of all preceding bytes mod 0x10000, in file byte order. DMDC2's `CSerializedWriterWithChkSum` writes the same sum. The seed duplicates the bodies and freezes the trailer. Consequences: - Any typed edit reconstructs with a stale checksum. - Resizing a map fails with `BodyOutOfRange`. - **Packed fields are opaque to clients.** `Map` carries `map_size_word`, `flags_word` and `num_graphs_word`; tiles and records are raw words. The web editor's dungeon browser therefore uses the lossy `dungeonSkprojectJson` and re-decodes tile bytes itself (`byte >> 5`). Editing has no path. ## Plan 1. Schema v2: drop `tile_block_seed`. Reconstruct writes the bodies at `map_offset`, requires exact tiling, and recomputes the trailer. Parse exposes the stored checksum and whether it is valid. 2. Decoded projection: emit decoded fields (map size, level, graphics counts, tile type and attributes, record fields) instead of packed words, and pack them back on reconstruct. Still byte-exact over the fixtures. 3. Binding parity (wasm, FFI, Python), CLI, docs. ## Open - Does the engine verify the trailer? Settle it in-game with a dungeon whose checksum is wrong. ## Acceptance `parse -> JSON -> reconstruct` is byte-identical over every fixture. An edited dump reloads in-game.
Author
Owner

In-game (DOS EN, new game): a dungeon with a wrong checksum loads and plays; the edited one (rebuilt checksum) shows the edit. The engine ignores the trailer.

In-game (DOS EN, new game): a dungeon with a wrong checksum loads and plays; the edited one (rebuilt checksum) shows the edit. The engine ignores the trailer.
Author
Owner

Schema v3 (decoded fields instead of packed words) is agreed. It waits on #392, which decodes the residual bits first.

Schema v3 (decoded fields instead of packed words) is agreed. It waits on #392, which decodes the residual bits first.
Author
Owner

In-game (DOS EN, new game): a dungeon edited only through the schema 3 JSON (dm2 dump --format dungeon-lossless, a JSON edit, dm2 build) loads and shows both edits:

  • tile (3,8) of map 0 turned from floor to wall;
  • the two west-wall champion mirrors facing (1,5) and (1,3) have their champion numbers swapped (actuator data 1 ↔ 3).

Together with the byte-identical round-trip over every fixture, both acceptance criteria are met. Delivered in #398 (maps and tiles), #399 (records), #400 (actuators and creatures) and #401 (dm2 build, binding tests).

In-game (DOS EN, new game): a dungeon edited only through the schema 3 JSON (`dm2 dump --format dungeon-lossless`, a JSON edit, `dm2 build`) loads and shows both edits: - tile (3,8) of map 0 turned from floor to wall; - the two west-wall champion mirrors facing (1,5) and (1,3) have their champion numbers swapped (actuator `data` 1 ↔ 3). Together with the byte-identical round-trip over every fixture, both acceptance criteria are met. Delivered in #398 (maps and tiles), #399 (records), #400 (actuators and creatures) and #401 (`dm2 build`, binding tests).
jqueuniet added this to the dm2-tools project 2026-10-28 23:21:28 +01:00
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
jqueuniet/dm2-tools#390
No description provided.