06RESEARCH PAPERPreprint
Relation-State Continuity in Qwen2.5-7B Base: Evidence for an Attention-Topology Path
發表日期
摘要
ABSTRACTZenodo 原文I probe Qwen2.5-7B base across eight relation-tracking cases and report four aggregate scores summarising the 29-layer profile: residual carry AUC 0.55, attention anchor mass AUC 0.089, formation peak 0.94, cross-layer circuit score 0.39. The formation profile is bimodal with peaks at layer 1 and layer 28, and no single intermediate layer dominates. The work of routing anchor information to query position is spread across layers 19 through 27, where residual carry rises from 0.60 to 0.81 while attention to anchors stays in the 0.12 to 0.23 range. I label this pattern attention topology path. The classification is observational; causal verification via activation patching is the next experiment.
全文 PDF
FULL TEXT1 個檔- 下載 PDFpaper-4-rsc-qwen.pdf・74 KB・本站
PDF 與 Zenodo 上的檔案相同(大小與 md5 已核對)。
論文資料
DETAILSZenodo 紀錄 20449372- 作者
- Chen, Ho Yiing
- 發表日期
- 資源類型
- Preprint
- DOI
- 10.5281/zenodo.20449372
- Zenodo
- zenodo.org/records/20449372
- 授權
- Creative Commons Attribution 4.0 International
- 版本
- v1
作者 Chen, Ho Yiing 是織田紀香的本名(陳禾穎)英文拼寫。標題、摘要、關鍵字與授權照 Zenodo 原文。
關鍵字
KEYWORDS14 個- LLM interpretability
- mechanistic interpretability
- relation tracking
- Qwen2.5-7B
- residual stream
- attention anchors
- topology path
- layer attribution
- activation probing
- Mercury
- consumer hardware
- open source
- independent research
- reproducibility