06RESEARCH PAPERPreprint
Naive Contrastive Function Vector Injection Does Not Transfer Reasoning Capability Within the Llama-3 Family
發表日期
摘要
ABSTRACTZenodo 原文We test whether a contrastive activation difference computed at a single layer of a reasoning-distilled donor can be injected into a generally instruction-tuned host of the same architecture to transfer mathematical reasoning capability. Donor: DeepSeek-R1-Distill-Llama-8B. Host: Llama-3.1-8B-Instruct. Both share Llama-3 architecture and tokenizer, removing common cross-family confounds. We inject the contrastive vector at layer 12 with alpha in {0, 0.1, 0.3, 0.5} and evaluate on 10 held-out GSM8K problems. Baseline accuracy is 65.0%. Injection produces a monotonically decreasing accuracy: 50.0% at alpha=0.1, 30.0% at alpha=0.3, and 10.0% at alpha=0.5 (Delta = -55 percentage points relative to baseline). No cell achieves raw p<0.05 under paired exact McNemar testing; closest is p=0.0625 at alpha=0.5, which does not survive Bonferroni correction. We conclude that naive last-token contrastive subtraction does not encode a transferable reasoning capability in this single-layer setting. Reasoning-marker rate stays stable (2.00 to 2.30) across all injection cells while accuracy collapses, indicating the contrastive vector captures surface mathematical vocabulary rather than transferable computational structure. We discuss three structural hypotheses (reasoning is not a single direction; wrong layer/geometry; donor-host capability gap too small) and propose four concrete follow-up protocols. All code, calibration prompts, test prompts, per-cell generation outputs, and aggregation results are released open-source for reproduction.
全文 PDF
FULL TEXT1 個檔- 下載 PDFpaper-e1-negative.pdf・139 KB・本站
PDF 與 Zenodo 上的檔案相同(大小與 md5 已核對)。
論文資料
DETAILSZenodo 紀錄 20394207- 作者
- Chen, Ho Yiing
- 發表日期
- 資源類型
- Preprint
- DOI
- 10.5281/zenodo.20394207
- Zenodo
- zenodo.org/records/20394207
- 授權
- Creative Commons Attribution 4.0 International
作者 Chen, Ho Yiing 是織田紀香的本名(陳禾穎)英文拼寫。標題、摘要、關鍵字與授權照 Zenodo 原文。
關鍵字
KEYWORDS10 個- mechanistic interpretability
- function vectors
- activation steering
- LLM composition
- cross-model capability transfer
- negative result
- GSM8K
- Llama-3
- DeepSeek-R1
- Frankenstein framework
其他研究論文
MORE- 較新一篇:A Reproducible Within-Family SLERP Merge of Llama-3.1-8B-Instruct and DeepSeek-R1-Distill-Llama-8B
- 較舊一篇:An Independent Billing Audit of Claude's Production API: 4 Experiments Across Haiku, Sonnet, and Opus Confirm Mathematical Honesty; Tool-Definition Overhead and Silent Desktop-App Routing Remain Consumer Concerns
- 織田紀香的研究論文(全部 66 筆)