jqueuniet/dm2-tools
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

dm2-graphics: dtText should truncate at first NUL like the engine — JP ports pad to even length, crate emits trailing U+0000 #193

New issue
Closed
opened 2026-07-05 07:48:35 +02:00 by jqueuniet · 1 comment
jqueuniet commented 2026-07-05 07:48:35 +02:00 (Migrated from codeberg.org)
Copy link

Problem

Text::parse (crates/dm2-graphics/src/text.rs ~145) requires the last decoded byte to be NUL and tokenises everything before it. The engine stops at the first NUL: DM2_FORMAT_SKSTR terminates on '\0' wherever it appears ([SKPROJECT] v5/gfxstr.cpp:326-331).

This diverges on real data: 384 Mac JP and 382 FM-Towns Japanese dtText payloads decode to …text\0\0 — the string is NUL-terminated one byte early and padded to even length. The crate keeps the interior NUL as a literal byte, so:

  • segments() carries a spurious trailing 0x00 in the last Literal;
  • to_display_string() on those records ends with U+0000 (encoding_rs maps CP932 0x00 → NUL).

Probed: every affected payload has exactly one padding byte and it is always zero (no port hides content after the first NUL), so truncation is safe. dos-en / mac-us / amiga / sega-cd have no interior NULs at all.

Fix direction

Truncate the decoded body at the first NUL (and keep MissingNul for payloads with none anywhere), matching the engine. The even-length padding convention is worth a line in docs/formats/graphics-dat.md § dtText.

Probe: scratch/probe_text_nul.py (gitignored).

## Problem `Text::parse` (`crates/dm2-graphics/src/text.rs` ~145) requires the **last** decoded byte to be NUL and tokenises everything before it. The engine stops at the **first** NUL: `DM2_FORMAT_SKSTR` terminates on `'\0'` wherever it appears (`[SKPROJECT] v5/gfxstr.cpp:326-331`). This diverges on real data: **384 Mac JP** and **382 FM-Towns** Japanese dtText payloads decode to `…text\0\0` — the string is NUL-terminated one byte early and padded to even length. The crate keeps the interior NUL as a literal byte, so: - `segments()` carries a spurious trailing `0x00` in the last `Literal`; - `to_display_string()` on those records ends with U+0000 (encoding_rs maps CP932 `0x00` → NUL). Probed: every affected payload has exactly one padding byte and it is always zero (no port hides content after the first NUL), so truncation is safe. dos-en / mac-us / amiga / sega-cd have no interior NULs at all. ## Fix direction Truncate the decoded body at the first NUL (and keep `MissingNul` for payloads with none anywhere), matching the engine. The even-length padding convention is worth a line in `docs/formats/graphics-dat.md` § dtText. Probe: `scratch/probe_text_nul.py` (gitignored).
jqueuniet commented 2026-07-06 06:22:45 +02:00 (Migrated from codeberg.org)
Copy link

Fixed by #208 (merged into main).

Fixed by #208 (merged into `main`).
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
pages
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jqueuniet/dm2-tools#193
No description provided.