Web-facing serialization scheme: self-contained JSON vs filesystem + sidecars #307

New issue

Open

opened 2026-07-31 22:31:12 +02:00 by jqueuniet · 0 comments

jqueuniet commented

2026-07-31 22:31:12 +02:00

Owner

Question

The DM2 JSON formats designed so far — dm2-dungeon's json::skproject dump and the
planned graphics/asset exports — assume a filesystem layout: a structured JSON
document alongside sidecar binary files (PNG images, WAV/PCM sounds, raw blobs). A
browser has no filesystem to scatter sidecars across. What serialization scheme should
the web path use, and how does it reconcile with the desktop/filesystem scheme?

What we know

dm2-dungeon's existing json::skproject dump is one-way and lossy (it re-implements
the retired C# convertDungeonDatToJson interop format; e.g. start_map is hardcoded
to 0 with the comment "No header field carries the start map"). Not round-trippable.
The parser crates already round-trip at the binary level — byte-exact
to_bytes / write_to, tested against fixtures.
The lossless-JSON work starting with saves uses a self-contained document: raw
not-yet-decoded sections embedded as base64, so a single JSON round-trips byte-exact
with no sidecar files. That is inherently web-friendly.
Graphics is the hard case: embedding many decoded images as base64 inside one JSON is
heavy, whereas a filesystem dump would use PNG sidecars.

Options to weigh

Single self-contained JSON with embedded/base64 binaries (web-first; heavy for graphics).
Filesystem bundle: JSON + sidecar binaries (desktop-first; needs a virtual-FS/zip shim
on the web).
An in-memory container (zip/tar) the web layer unpacks; JSON references entries by name.
Two projections over one shared owned document model (self-contained for web, sidecar
for filesystem).

How we'd confirm

Pick a scheme that (a) round-trips byte-exact for dungeon/saves, (b) keeps graphics
payloads tractable in the browser, and (c) shares one owned document model with the
filesystem tooling. Prototype against a saves fixture and a graphics fixture.

Notes

Gates the web exposure in #301 and the graphics lossless-JSON work. The per-format
lossless serializers (saves, dungeon) can proceed in parallel since their self-contained
form is already web-compatible; this issue settles the cross-cutting policy, especially
for graphics.

Labels

type/research, area/tooling, priority/medium.

## Question The DM2 JSON formats designed so far — `dm2-dungeon`'s `json::skproject` dump and the planned graphics/asset exports — assume a **filesystem layout**: a structured JSON document alongside sidecar binary files (PNG images, WAV/PCM sounds, raw blobs). A browser has no filesystem to scatter sidecars across. What serialization scheme should the web path use, and how does it reconcile with the desktop/filesystem scheme? ## What we know - `dm2-dungeon`'s existing `json::skproject` dump is one-way and lossy (it re-implements the retired C# `convertDungeonDatToJson` interop format; e.g. `start_map` is hardcoded to 0 with the comment "No header field carries the start map"). Not round-trippable. - The parser crates already round-trip at the **binary** level — byte-exact `to_bytes` / `write_to`, tested against fixtures. - The lossless-JSON work starting with saves uses a **self-contained** document: raw not-yet-decoded sections embedded as base64, so a single JSON round-trips byte-exact with no sidecar files. That is inherently web-friendly. - Graphics is the hard case: embedding many decoded images as base64 inside one JSON is heavy, whereas a filesystem dump would use PNG sidecars. ## Options to weigh - Single self-contained JSON with embedded/base64 binaries (web-first; heavy for graphics). - Filesystem bundle: JSON + sidecar binaries (desktop-first; needs a virtual-FS/zip shim on the web). - An in-memory container (zip/tar) the web layer unpacks; JSON references entries by name. - Two projections over one shared owned document model (self-contained for web, sidecar for filesystem). ## How we'd confirm Pick a scheme that (a) round-trips byte-exact for dungeon/saves, (b) keeps graphics payloads tractable in the browser, and (c) shares one owned document model with the filesystem tooling. Prototype against a saves fixture and a graphics fixture. ## Notes Gates the web exposure in #301 and the graphics lossless-JSON work. The per-format lossless serializers (saves, dungeon) can proceed in parallel since their self-contained form is already web-compatible; this issue settles the cross-cutting policy, especially for graphics. ## Labels `type/research`, `area/tooling`, `priority/medium`.

jqueuniet added this to the Bindings & distribution milestone

2026-07-31 22:31:12 +02:00

jqueuniet added the

priority/medium

area/tooling

type/research

labels

2026-07-31 22:31:12 +02:00

jqueuniet referenced this issue

2026-07-31 22:31:54 +02:00

Lossless, round-trippable JSON for dm2-saves #308

jqueuniet referenced this issue