Qwen3.8-27B harness adapters
Hugging Face says that freezing Qwen3.8-27B and adding a custom LoRA adapter trained with reinforcement learning inside the pi harness lifts its Terminal-Bench 2.1 pass@1 from 39% to 55%.1 Hugging Face says the adapter also cuts average token use per task from 2.9K to 1.5K, mostly because the model reasons less, calls fewer tools and does better on its first attempt.2 Latent Space's AINews newsletter also covered the pi adapter, citing an improvement from 15 to 32 solved problems among the 51 most difficult SWE-bench Pro tasks, along with 55% fewer tool invocations for each task.3
For each agent harness (pi, Hermes Agent and OpenCode), the team built a separate LoRA adapter on top of Qwen3.8-27B, applying reinforcement learning within the harness itself through TRL's AsyncGRPOTrainer, OpenEnv and harbor.45 According to Hugging Face, each of the three specialist adapters outperformed the base model on both benchmarks when run in its own harness. In the hard 51-task SWE-bench Pro subset, pi climbed from 39.2% to 64.7%, hermes went from 29.4% to 43.1%, and opencode moved from 35.3% to 51.0%.67 Across the entire Terminal-Bench 2.1, Hugging Face reports improvements of 16.3 points for pi and 14.1 for hermes. For opencode the overall improvement is 6.0 points, rising to 13.6 on the 66 tasks that had no role in selecting its checkpoint.89
Hugging Face adds that one merged "super-adapter" likewise outperformed the base model in all three harnesses, though with thinking turned off it retains a smaller share of each specialist's improvement, and the loss is greatest under pi.1011
Hugging Face reports that the adapters stayed close to the base model's 23.0% pass@1 on AutomationBench: pi at 27.0%, OpenCode at 25.3%, the merged adapter at 23.7% and Hermes lower at 17.7%.12 Hugging Face reads this as showing that harness-specific training neither transfers clearly to a very different setting nor breaks the model there, noting the differences are within noise except perhaps Hermes.13
Sources quoted
Take Qwen3.8-27B, its Terminal-Bench 2.1 score with the pi harness is 39% which uses on average 2.9K tokens per task. If we freeze those weights and add a custom RL-trained LoRA adapter we can get 55% pass@1, with average token usage of 1.5K.
— Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and costwith average token usage of 1.5K. Most of this is down to the model reasoning less, calling less tools, and performing better first time.
— Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and costA small adapter trained inside pi lifted Qwen3.8-27B from 15 to 32 solves on the 51 hardest SWE-bench Pro tasks. Across that subset it made 55% fewer tool calls per task.
— Latent Space, [AINews] GPT-6.2 Astra, the #2 Frontier Model is better and 20%-600% cheaper, but has mixed reviews; 6.2 Sol likely main workhorseWe trained one LoRA adapter for Qwen3.8-27B per agent harness (pi, Hermes Agent, OpenCode), with RL inside the harness itself.
— Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and costDatasets, training and agent rollouts all happen on the Hub in Jobs using TRL's AsyncGRPOTrainer, OpenEnv and harbor.
— Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and costAll three specialist adapters beat the base model under their own harness, on both benchmarks.
— Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost- Quote not reprinted: this page already repeats as much of this article as we allow. — Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost
- Quote not reprinted: this page already repeats as much of this article as we allow. — Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost
- Quote not reprinted: this page already repeats as much of this article as we allow. — Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost
- Quote not reprinted: this page already repeats as much of this article as we allow. — Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost
It keeps less of each specialist's gain with thinking off, most under pi
— Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost- Quote not reprinted: this page already repeats as much of this article as we allow. — Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost
- Quote not reprinted: this page already repeats as much of this article as we allow. — Hugging Face, Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost
Who reported it
- Hugging Faceannouncement: Publishes per-harness RL LoRA adapters for pi, Hermes Agent and OpenCode, a merged super-adapter, and a reusable training recipe with datasets.
- Latent Spacerepeats Hugging Face: A small adapter trained inside pi raised Qwen3.8-27B from 15 to 32 solves on 51 hard SWE-bench Pro tasks, with 55% fewer tool calls.
Facts and takes
Facts: 9 checked facts from 1 source
- Qwen team released Qwen 3.8 27B
- Qwen 3.8 27B: 39% on Terminal-Bench 2.1 (pi harness)
- Qwen 3.8 27B: 5.6% on Terminal-Bench 4.1 (pass@1)
- Qwen 3.8 27B: 73.0 on Terminal Bench 2.1
- Qwen 3.8 27B: 42.2 on DeepSWE v1.1
- Qwen 3.8 27B: 91.3 on SWE Bench Pro v2
- Qwen 3.8 27B: 33.9 on HLE (text-only)
- Qwen 3.8 27B: 79.5 on IFBench
- Qwen 3.8 27B: 89.2 on GPQA Diamond
Takes: 5 views from 1 holders · 1 positive · showing 1
- +Hugging Face: Quote not reprinted: this page already repeats as much of this article as we allow.
Timeline
- · Latent Space: [AINews] GPT-6.2 Astra, the #2 Frontier Model is better and 20%-600% cheaper, but has mixed reviews; 6.2 Sol likely main workhorse
- · Hugging Face: Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost