When a function-calling gain is really an interface gain.
This project tests whether prompt-prepended skills improve tool-use procedures or simply teach the output contract expected by the scorer.
这个项目检验：prompt 注入的 skill 到底提升了工具调用程序，还是只是教会模型评测器要求的输出格式。
Figure from the paper: standard scores can move for reasons that canonicalized scoring exposes.
One score, two mechanisms.
A correct function call can still fail when its wrapper keys do not match the evaluator contract.
同一个分数可能混合了任务决策和接口合规两种机制。
What the diagnostics show.
The artifact is designed for auditability: each result is tied to a scorer, control, or portability check.
每个结论都对应一个评分器、控制实验或跨契约检查。
standard-score regressions become correct after canonicalized rescoring in the highlighted Qwen3-14B case.
format-only gain on Qwen3-8B, exceeding the +11.3 pp full-skill gain in the same cell.
maximum gap between target-native and length-matched generic procedural prompts in the reported API-Bank check.
| Diagnostic | What it isolates | How to read it |
|---|---|---|
| Canonicalized | Wrapper-key aliasing | Separates outer-format failures from call structure; not a complete semantic executor. |
| Format-only | Explicit interface contract | If it matches the full skill, the gain is not uniquely procedural. |
| Repaired / balanced | Induction-pool sensitivity | Tests whether the result survives cleaner extraction and demonstration selection. |
Small public method package.
The repository keeps the reusable scorer and hand-written controls public without bundling raw model outputs or generated skills.
仓库公开可复用的 scorer 和手写 controls，不捆绑原始模型输出或自动生成 skill。
Install
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtRescore
python src/rescore_canonical.py \
--results-dir /path/to/results \
--output /tmp/summary.jsonThe paper reports the final numerical matrices; this repository is the compact method release for scorer and control inspection.论文中给出最终数值矩阵；本仓库是用于检查 scorer 和 controls 的精简方法包。
Cite the paper.
Use the arXiv record for the current preprint.
当前预印本请使用 arXiv 记录。
@misc{chen2026attributingstructuredoutputgainsfunction,
title = {Attributing Structured-Output Gains in Function Calling: Interface Alignment versus Procedural Transfer},
author = {Wanyi Chen and Daoyuan Chen and Fang Kong},
year = {2026},
eprint = {2607.02595},
archivePrefix = {arXiv},
primaryClass = {cs.SE},
url = {https://arxiv.org/abs/2607.02595}
}