GAPS: Dimension-Level Gates for Conditional Activation Steering

GAPS: Dimension-Level Gates for Conditional Activation Steering

GAPS (Gated Activation Steering via Posterior and Separability) adds two training-free gates to existing conditional activation steering methods: a static separability gate that activates steering only in neurons with statistically reliable concept information, and a dynamic posterior gate that activates steering only when a neuron's current activation is better explained by the undesired concept under a Gaussian model.

The gates incur O(D) per-token overhead and are designed to be compatible with current methods. Empirical results on RealToxicityPrompts and OneSeC datasets demonstrate overall Pareto-front improvements for toxicity mitigation and concept removal across Gemma-3 and Qwen-3 models, notably reducing toxicity rates compared to DSAS alone.

Why it matters: By adding dimension-level selectivity, GAPS reduces unnecessary intervention, potentially lowering unintended alterations to benign behavior while preserving or enhancing safety gains. This approach can tighten the efficiency and effectiveness of activation steering under fixed capability budgets.

AI-assisted brief