Tier-A Vocab-Hash Artifact in Cross-Architecture Hot-Dim Studies: Why Output-Layer Token-ID Projection Manufactures False Universality, and an Artifact-Aware Screening Protocol

A widely useful observation protocol for transformer LLMs reads the output-layer logits, maps each generated token id to a hidden-state index via d = tokid mod D, and aggregates per-index firing intensity into a cross-prompt heatmap. I call this Tier-A observation. Applied to a panel of 18 LLMs across 13 architectures, the Tier-A protocol identifies eleven anchor hidden-state indices that appear in the top-50 hottest list of as many as 11 of the 18 networks, including AllenAI's OLMo2 7B at 29.8x the hypergeometric random baseline.

I report a direct contradiction. On the same OLMo2-7B checkpoint with the same prompts, switching from Tier-A to Tier-B (per-block residual stream via HuggingFace output_hidden_states) returns zero of the eleven Tier-A anchors among OLMo2's actual top-magnitude residual indices. The genuinely hot residual indices are {514, 2273, 2594, 2983, 808, 1066}, with no agreement with the Tier-A anchor set.

Mechanism: the d = tokid mod D projection is a modular hash from token-id space (~100k entries) into hidden-state index space (2k-8k entries). High-frequency BPE entries shared across BPE schemes hash to overlapping indices regardless of underlying residual geometry. Tier-A confounds genuine residual sharing with BPE-induced index aliasing.

Corrective: a two-tier screening protocol. Tier-A as a cheap candidate-signal generator (good for narrowing the search space). Tier-B as the validation layer where index-level cross-architecture claims are made. The per-layer cross-architecture alignment finding survives this demotion at the layer level, reported in the companion paper.

Open data and code: https://github.com/norika1207-lab/mercury-paper-handoff