burtenshaw/tmax2-rawcmd-tmaxsft-lora-lr1e5-r8-s80

TMax2 Raw-Command TMax-SFT LoRA

This is a LoRA adapter for allenai/tmax-2b trained as a terminal-agent raw-command distillation rung.

Method

• Base model: allenai/tmax-2b
• Method: TRL SFT with PEFT LoRA, rank 8, learning rate 1e-5, 80 steps
• Training data: allenai/tmax-sft, config skill_tax_20260505_2.2k_combined_balanced_thinking_only_success
• Conversion: successful teacher traces were converted into prompt/completion rows where the completion is only the next bash command
• Data size: 384 train episodes, 3,581 train action rows; 64 held-out episodes, 619 held-out action rows
• Training job: SLURM 22303226, one hopper-prod node, qos=low
• Trackio: https://huggingface.co/spaces/burtenshaw/terminal-bench-loop-trackio

Proxy Evaluation

This is not a full Terminal-Bench result. It is a raw-command TBLite proxy over four held-out tasks: log-summary, jsonl-aggregator, protein-sequence, and schedule-vacation.

Model	Strict proxy success	Valid command rate	Avg turns	Notes
allenai/tmax-2b base	0/4	0.7083	6.0	Same evaluator run
This adapter	1/4	0.9000	5.0	Solved jsonl-aggregator

The adapter improved over the same-size base on this proxy gate, but it does not approach the final project goal of Terminal-Bench > 40. Treat it as a promotion candidate for the next controlled rung, not a benchmark win.

Artifacts

• Training root: /fsx/benjamin_burtenshaw/post-training-agent-experiments/2026-06-24-tmax2-rawcmd-trackio-sft/
• Eval summary in this repo: eval/tblite_rawcmd_proxy/eval_summary.json
• Combined base-vs-adapter summary: eval/tblite_rawcmd_proxy/combined_summary.json