Inside RANA: refusal ablation and its tradeoffs

Qwen3.8-27B-RANA-abliterated is our open-weight release for red-teaming and refusal-behavior research. It changes Qwen3.8-27B's weights rather than asking the model to act differently through a jailbreak prompt.

Getting a model to stop refusing is only part of the job. The harder question is what the edit costs: reasoning and coding capability, vision, tool use, or answers that run out of room before they finish. RANA was evaluated against the base model and the orcarouter/Qwen3.8-27B-Uncensored reference, called orca below, to measure those tradeoffs.

The result: more held-out requests answered under both judges, fewer unfinished responses than orca at the same token budget, and less measured drift from the base model. BF16, FP8, GGUF, and MLX weights are available under Apache-2.0.

Start with the results

The release evaluation put all three models through the same harness with thinking enabled. Refusal testing used 200 HarmBench behaviors and 240 held-out harmful prompts, three sampling seeds, and a 16,384-token generation budget. Two LLM judges read the responses: gpt-oss-safeguard-20b and Mistral-Small-24B-Instruct-2501.

"Answered" means the judge found the harmful benchmark request substantively fulfilled. It is not an accuracy score. The judges disagree on hedged and partial answers, so their results stay separate. Both favor RANA on the held-out set; on HarmBench, they disagree about which abliteration answers more often.

The KL measurement compares first-token probability distributions on 100 harmless prompts. Lower means closer to the base model on that protocol, not better at every task. The base model's low budget-hit rate also needs context: it usually refuses these harmful requests early.

Both abliterations recorded zero hard refusals on HarmBench under both judges after the same mechanical re-judging procedure. Each had two raw gpt-oss hard-refusal labels across the three seeds; a second pass with a fixed auditor reminder relabeled them. Those raw results and the procedure are published together. Zero is a result under that evaluation policy, not a promise about every possible prompt.

What changed in the weights

RANA stands for Reasoning-Anchored, Norm-preserving Ablation. It removes a direction associated with refusal from weights that write to the residual stream, then rescales the edited weight vectors to their original norms. The approach builds on refusal-direction research by Arditi and colleagues and norm-preserving abliteration by Jim Lai, also known as grimjim.

The BF16 release changes 131 weight matrices: attention outputs, MLP outputs, and token embeddings, including two corresponding matrices in the multi-token prediction (MTP) head. All 333 vision-tower tensors remain byte-identical to the base model.

The published BF16 shards passed functional checks for vision input, three-turn tool calling, and MTP speculative decoding. These checks show that those paths work; unchanged vision weights alone would not prove unchanged behavior on every image task.

The weights and evaluation records are public. The exact intervention recipe and code are not, so this is an open-weight release, not a fully reproducible training or editing pipeline.

Capability did not come for free

The capability suite covered MMLU-Pro, GSM8K, IFEval, TruthfulQA, and HumanEval. The average in the table is absolute movement from base, not average capability loss; TruthfulQA's MC1 and MC2 scores are averaged into one task.

RANA's clearest loss against base was about 1.5 percentage points on TruthfulQA. Its HumanEval change was -0.4 ± 0.9 points, within the measured uncertainty. Against orca, RANA was worse at instruction following: IFEval fell by 2.2 ± 0.9 points. MMLU-Pro used a 700-question subset, not the full benchmark.

The release also had rejection criteria. An earlier candidate lost 2.24 points on HumanEval and was not released. The final build passed the published gates, including a mean absolute capability change of at most one point, no task below minus two points, and checks on the edited tensors and runtime functions.

Give long reasoning room to finish

A response cut off at the token limit is not automatically a refusal or a loop. Most of RANA's HarmBench budget hits were long reasoning traces, with no more repetition than normal output.

The FP8 follow-up re-ran all 59 prompts that hit the first seed's 16k budget, this time allowing 64k output tokens. Of those, 57 finished and the gpt-oss judge marked 55 as answering the request. That explains why the generation budget matters when comparing these models.

The standard evaluation used a 20,480-token serving limit. The BF16 configuration declares 262,144 positions, but those evaluations do not validate the full context window. For long technical requests, adjust both context capacity and the output budget using the guidance in the relevant model card.

Pick a build

These are weight-file sizes, not complete RAM or VRAM requirements. Allow for runtime memory and any separate vision or MTP files.

Evaluation coverage differs by build. FP8 has three-seed refusal testing with both judges. GGUF's functional and refusal checks cover Q4_K_M only, with one refusal-testing seed. MLX's recorded checks ran on Linux with the CUDA backend, not on a Mac, and did not re-measure refusal behavior.

For MLX, download one precision folder and load it by local path. Passing the whole repository ID to mlx-vlm can download every precision, roughly 160 GB. Each model card has the commands and file layout for its build.

Run it, inspect it, measure it

RANA is a safety-alignment-removed research model. If you put it in front of end users, moderation needs to live outside the model. Reduced refusal does not establish greater accuracy or safety, and the Apache-2.0 terms inherited from Qwen still apply.

Start with the model directory, the full model card, and the recorded results. Compare the released weights on your own workload, not just the headline numbers.

For the hosted experience, open Abliterant Chat. Model downloads and Chat are separate products: these benchmarks describe the released weights, not Chat's serving configuration.