NORIKA ODA

06RESEARCH PAPERPreprint

Relation-State Continuity in Qwen2.5-7B Base: Evidence for an Attention-Topology Path

Chen, Ho Yiing

發表日期

DOI 10.5281/zenodo.20449372 Zenodo 紀錄頁 PDF(74 KB)

01

摘要

ABSTRACTZenodo 原文
I probe Qwen2.5-7B base across eight relation-tracking cases and report four aggregate scores summarising the 29-layer profile: residual carry AUC 0.55, attention anchor mass AUC 0.089, formation peak 0.94, cross-layer circuit score 0.39. The formation profile is bimodal with peaks at layer 1 and layer 28, and no single intermediate layer dominates. The work of routing anchor information to query position is spread across layers 19 through 27, where residual carry rises from 0.60 to 0.81 while attention to anchors stays in the 0.12 to 0.23 range. I label this pattern attention topology path. The classification is observational; causal verification via activation patching is the next experiment.
02

全文 PDF

FULL TEXT1 個檔

PDF 與 Zenodo 上的檔案相同(大小與 md5 已核對)。

03

論文資料

DETAILSZenodo 紀錄 20449372
作者
Chen, Ho Yiing
發表日期
資源類型
Preprint
DOI
10.5281/zenodo.20449372
Zenodo
zenodo.org/records/20449372
授權
Creative Commons Attribution 4.0 International
版本
v1

作者 Chen, Ho Yiing 是織田紀香的本名(陳禾穎)英文拼寫。標題、摘要、關鍵字與授權照 Zenodo 原文。

04

關鍵字

KEYWORDS14 個
  • LLM interpretability
  • mechanistic interpretability
  • relation tracking
  • Qwen2.5-7B
  • residual stream
  • attention anchors
  • topology path
  • layer attribution
  • activation probing
  • Mercury
  • consumer hardware
  • open source
  • independent research
  • reproducibility
05

其他研究論文

MORE