NORIKA ODA

06RESEARCH PAPERPreprint

Naive Contrastive Function Vector Injection Does Not Transfer Reasoning Capability Within the Llama-3 Family

Chen, Ho Yiing

發表日期

DOI 10.5281/zenodo.20394207 Zenodo 紀錄頁 PDF(139 KB)

01

摘要

ABSTRACTZenodo 原文
We test whether a contrastive activation difference computed at a single layer of a reasoning-distilled donor can be injected into a generally instruction-tuned host of the same architecture to transfer mathematical reasoning capability. Donor: DeepSeek-R1-Distill-Llama-8B. Host: Llama-3.1-8B-Instruct. Both share Llama-3 architecture and tokenizer, removing common cross-family confounds. We inject the contrastive vector at layer 12 with alpha in {0, 0.1, 0.3, 0.5} and evaluate on 10 held-out GSM8K problems. Baseline accuracy is 65.0%. Injection produces a monotonically decreasing accuracy: 50.0% at alpha=0.1, 30.0% at alpha=0.3, and 10.0% at alpha=0.5 (Delta = -55 percentage points relative to baseline). No cell achieves raw p<0.05 under paired exact McNemar testing; closest is p=0.0625 at alpha=0.5, which does not survive Bonferroni correction. We conclude that naive last-token contrastive subtraction does not encode a transferable reasoning capability in this single-layer setting. Reasoning-marker rate stays stable (2.00 to 2.30) across all injection cells while accuracy collapses, indicating the contrastive vector captures surface mathematical vocabulary rather than transferable computational structure. We discuss three structural hypotheses (reasoning is not a single direction; wrong layer/geometry; donor-host capability gap too small) and propose four concrete follow-up protocols. All code, calibration prompts, test prompts, per-cell generation outputs, and aggregation results are released open-source for reproduction.
02

全文 PDF

FULL TEXT1 個檔

PDF 與 Zenodo 上的檔案相同(大小與 md5 已核對)。

03

論文資料

DETAILSZenodo 紀錄 20394207
作者
Chen, Ho Yiing
發表日期
資源類型
Preprint
DOI
10.5281/zenodo.20394207
Zenodo
zenodo.org/records/20394207
授權
Creative Commons Attribution 4.0 International

作者 Chen, Ho Yiing 是織田紀香的本名(陳禾穎)英文拼寫。標題、摘要、關鍵字與授權照 Zenodo 原文。

04

關鍵字

KEYWORDS10 個
  • mechanistic interpretability
  • function vectors
  • activation steering
  • LLM composition
  • cross-model capability transfer
  • negative result
  • GSM8K
  • Llama-3
  • DeepSeek-R1
  • Frankenstein framework
05

其他研究論文

MORE