GAPS: Dimension-Level Gates for Conditional Activation Steering

GAPS introduces dimension-level conditioning to selective activation steering, adding static separability and dynamic posterior gates to restrict interventions to neurons with reliable concept information. The method aims to reduce unnecessary steering by applying gates that decide which neurons to intervene on, and it plugs into existing conditional methods. Evaluations on toxicity mitigation and concept removal show improved or matching Pareto performance compared to token-level approaches.

GAPS (Gated Activation Steering via Posterior and Separability) adds two training-free gates to existing conditional activation steering methods: a static separability gate that activates steering only in neurons with statistically reliable concept information, and a dynamic posterior gate that activates steering only when a neuron's current activation is better explained by the undesired concept under a Gaussian model.

The gates incur O(D) per-token overhead and are designed to be compatible with current methods. Empirical results on RealToxicityPrompts and OneSeC datasets demonstrate overall Pareto-front improvements for toxicity mitigation and concept removal across Gemma-3 and Qwen-3 models, notably reducing toxicity rates compared to DSAS alone.

Source: cs.CL updates on arXiv.org — https://arxiv.org/abs/2609.01878