dm2-saves: Mac in-bitstream multi-byte fields decoded as little-endian — game_tick, GlobalWords, timer fields scrambled #168

New issue

Closed

opened 2026-07-04 15:14:38 +02:00 by jqueuniet · 1 comment

jqueuniet commented

2026-07-04 15:14:38 +02:00

(Migrated from codeberg.org)

Problem

Multi-byte fields inside the suppression bitstream are interpreted as little-endian on every platform. On Mac saves they are big-endian: the 68k engine memcpys its BE structs through the suppression writer, with per-field byte-swapped mask tables (the same model already implemented correctly for champions via MAC_CHAMPION_MASK and for the timer count in SaveFile::decode_full). Decoding them LE yields scrambled values.

Byte-exact round-trips cannot catch this: encode is the exact inverse of decode, so the wrong interpretation is invisible until a value is displayed, edited, or converted.

Evidence (Mac fixtures)

Probing fixtures/mac-us/saves/* with the current decoder:

Fixture	Field	LE (current)	byte-reversed
`early_game`	`game_tick`	6,226,176	351
`early_game`	`last_creature_tick`	`0x38ffffff`	`0xffffff38`
`endgame`	`game_tick`	14,204,419	245,464
`endgame`	`last_action_tick`	13,745,667	245,457

0xffffff38 is exactly the engine's documented new-game init constant for last_creature_tick (see savegamebuffer.rs field docs), and 351 is a plausible tick for a just-started game; endgame's reversed game_tick/last_action_tick are 7 ticks apart (a save moment). The LE readings are incoherent. DOS fixtures behave correctly under LE (control: sksave0 game_tick = 284,800 with last_creature_tick = 0x00045742 nearby).

Affected sites

savegamebuffer.rs from_decoded_bytes / as_bytes (LE always): game_tick, random_seed, last_creature_tick, last_action_tick, rain_next_special, timer_count_raw (the count itself is special-cased in decode_full, but the struct field remains LE-scrambled and is printed by dm2 info).
globals.rs decode_globals / GlobalWords::encode_to (u16::from_le_bytes / to_le_bytes unconditionally). Latent — all fixture global words are zero — but dm2 save global on a Mac save reads/writes values ×256 off.
timer.rs from_decoded_bytes / as_bytes: w_00 (LE always); val_a/val_b are exposed as byte pairs so interpretation is caller-side, but the same applies.
records.rs type-7 val_b (u16::from_le_bytes([data[2], data[3]]), no byte-order branch) around line 855.
10-bit cloud fields: checkcode.rs cloud_index decode/encode (~673/1007) and records.rs decode/encode_cloud_postfix (~933/957) use MASK_10BIT_CLOUD_INDEX = [0xff, 0x03] + LE on both platforms. On Mac the mask layout is [0x03, 0xff], so the decoded value is bit-permuted. Note the crate already implements the correct swapped model for the 9-bit actuator extra (records.rs ~775-782 ByteOrder::Big branch) — the 10-bit fields are inconsistent with it. cloud_index is a timer-array index in the engine.

Fields whose payload fits a single byte (player x/y/dir/map, party count, weather low bytes, etc.) are unaffected, which is why the existing Mac plausibility tests pass.

Fix direction

One byte-order-aware interpretation layer for in-bitstream multi-byte fields, mirrored exactly in encode (the champion code is the template: swapped mask + BE accessors). Doc claims that these fields are "always little-endian" need updating (tracked separately).

Reproduction

Decode any Mac fixture with SaveFile::parse(bytes, ByteOrder::Big) + decode_savegamebuffer() and compare game_tick against the byte-reversed reading; early_game's last_creature_tick reversal reproducing the 0xffffff38 new-game constant is the decisive check.

## Problem Multi-byte fields inside the suppression bitstream are interpreted as little-endian on every platform. On Mac saves they are big-endian: the 68k engine memcpys its BE structs through the suppression writer, with per-field byte-swapped mask tables (the same model already implemented correctly for champions via `MAC_CHAMPION_MASK` and for the timer count in `SaveFile::decode_full`). Decoding them LE yields scrambled values. Byte-exact round-trips cannot catch this: encode is the exact inverse of decode, so the wrong interpretation is invisible until a value is displayed, edited, or converted. ## Evidence (Mac fixtures) Probing `fixtures/mac-us/saves/*` with the current decoder: | Fixture | Field | LE (current) | byte-reversed | | --- | --- | --- | --- | | `early_game` | `game_tick` | 6,226,176 | **351** | | `early_game` | `last_creature_tick` | `0x38ffffff` | **`0xffffff38`** | | `endgame` | `game_tick` | 14,204,419 | **245,464** | | `endgame` | `last_action_tick` | 13,745,667 | **245,457** | `0xffffff38` is exactly the engine's documented new-game init constant for `last_creature_tick` (see `savegamebuffer.rs` field docs), and 351 is a plausible tick for a just-started game; `endgame`'s reversed `game_tick`/`last_action_tick` are 7 ticks apart (a save moment). The LE readings are incoherent. DOS fixtures behave correctly under LE (control: sksave0 `game_tick` = 284,800 with `last_creature_tick` = `0x00045742` nearby). ## Affected sites - `savegamebuffer.rs` `from_decoded_bytes` / `as_bytes` (LE always): `game_tick`, `random_seed`, `last_creature_tick`, `last_action_tick`, `rain_next_special`, `timer_count_raw` (the count itself is special-cased in `decode_full`, but the struct field remains LE-scrambled and is printed by `dm2 info`). - `globals.rs` `decode_globals` / `GlobalWords::encode_to` (`u16::from_le_bytes` / `to_le_bytes` unconditionally). Latent — all fixture global words are zero — but `dm2 save global` on a Mac save reads/writes values ×256 off. - `timer.rs` `from_decoded_bytes` / `as_bytes`: `w_00` (LE always); `val_a`/`val_b` are exposed as byte pairs so interpretation is caller-side, but the same applies. - `records.rs` type-7 `val_b` (`u16::from_le_bytes([data[2], data[3]])`, no byte-order branch) around line 855. - 10-bit cloud fields: `checkcode.rs` `cloud_index` decode/encode (~673/1007) and `records.rs` `decode/encode_cloud_postfix` (~933/957) use `MASK_10BIT_CLOUD_INDEX = [0xff, 0x03]` + LE on both platforms. On Mac the mask layout is `[0x03, 0xff]`, so the decoded value is bit-permuted. Note the crate already implements the correct swapped model for the 9-bit actuator extra (`records.rs` ~775-782 `ByteOrder::Big` branch) — the 10-bit fields are inconsistent with it. `cloud_index` is a timer-array index in the engine. Fields whose payload fits a single byte (player x/y/dir/map, party count, weather low bytes, etc.) are unaffected, which is why the existing Mac plausibility tests pass. ## Fix direction One byte-order-aware interpretation layer for in-bitstream multi-byte fields, mirrored exactly in encode (the champion code is the template: swapped mask + BE accessors). Doc claims that these fields are "always little-endian" need updating (tracked separately). ## Reproduction Decode any Mac fixture with `SaveFile::parse(bytes, ByteOrder::Big)` + `decode_savegamebuffer()` and compare `game_tick` against the byte-reversed reading; `early_game`'s `last_creature_tick` reversal reproducing the `0xffffff38` new-game constant is the decisive check.

jqueuniet commented

2026-07-06 08:47:19 +02:00

(Migrated from codeberg.org)

Fixed by #209 (merged into main).

Fixed by #209 (merged into `main`).

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

jqueuniet/dm2-tools#168

No description provided.

Rows
Columns