LightAIR: Lightweight Action Inversion and Riemannian Rectification for Text-based Person Anomaly Search

Yulun Zhang¹, Zixu Li^2,†, Zhiwei Chen², Zhiheng Fu², Wenbo Wang², Zihang Qiu², Zhilin Wang², Ruxin Wang^1,✉, Yupeng Hu²

¹ Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences ² Shandong University

^† Project leader ^✉ Corresponding author

ACM Multimedia 2026

Paper PDF Code BibTeX

About the paper

Abstract

Traditional Text-based Person Search (TPS) is typically limited to matching static appearance attributes, severely neglecting dynamic action information. The Text-based Person Anomaly Search (TPAS) task bridges this gap, requiring models to locate micro-level specific abnormal behaviors while matching macro-level appearance of pedestrians. However, current TPAS methods face fundamental limitations: external explicit pose estimators are fragile in unconstrained surveillance scenarios, and implicit learning encounters visual decoupling failure under pixel-level entanglement, causing dominant appearance information to easily swallow and contaminate subtle action features. Furthermore, performing contrastive optimization on hard negative samples ("same appearance, different actions") in conventional Euclidean spaces induces severe shortcut learning. To address these, we propose the Lightweight Action Inversion and Riemannian rectification network (LightAIR). First, it introduces textual semantic priors as anchors via a lightweight action inversion operator to extract pure action features, thereby overcoming visual-inherent coupling. Subsequently, it employs orthogonal null-space projection to constrain appearance features within the orthogonal complement space of action features, guaranteeing strict forward decoupling. Finally, we designed a gradient rectification module that computes the Riemannian gradient to constrain the backpropagation trajectory, forcing the gradient flow to update strictly along the tangent space that preserves decoupling properties, thereby cutting off harmful shortcuts. Extensive experiments on the widely used TPAS and TIPR datasets demonstrate that LightAIR significantly outperforms existing state-of-the-art methods.

LightAIR motivation figure explaining the shift from TIPR to TPAS, visual decoupling failure, and shortcut learning — MotivationFigure 1

Approach

Method

LightAIR extracts action features from textual semantic priors, separates appearance through orthogonal projection, and preserves the separation during optimization with Riemannian gradient rectification.

LightAIR framework with Action Inversion Operator, Orthogonal Null-Space Projection, and Gradient Rectification — LightAIR frameworkFigure 2

Evaluation

Experiments

Results on TPAS and conventional text-based person retrieval benchmarks.

Full Multi-weather performance table comparing LightAIR with X-VLM and CMP — PAB multi-weather settingTable 1

Full PAB performance table under standard, 0.1M, and 1M training settings — PAB benchmarkTable 2

Full TIPR performance table across CUHK-PEDES, ICFG-PEDES, RSTPReid, and UFineBench — TIPR benchmarksTable 4

Full UCC out-of-distribution performance table comparing LightAIR with prior methods — Out-of-distribution evaluation on UCCTable 3

Qualitative results

Action-sensitive retrieval

Green boxes mark matched images, red boxes mark mismatches, and blue boxes indicate hard negatives.

Qualitative retrieval comparisons between LightAIR and CMP for four action-sensitive text queries

Resources

Paper and citation

PaperarXiv abstract PDF20-page manuscript CodePyTorch implementation DOI10.1145/3767308.3835461

BibTeX

@article{zhang2026lightair,
  title   = {LightAIR: Lightweight Action Inversion and Riemannian Rectification for Text-based Person Anomaly Search},
  author  = {Zhang, Yulun and Li, Zixu and Chen, Zhiwei and Fu, Zhiheng and Wang, Wenbo and Qiu, Zihang and Wang, Zhilin and Wang, Ruxin and Hu, Yupeng},
  journal = {arXiv preprint arXiv:2608.09152},
  year    = {2026},
  url     = {https://arxiv.org/abs/2608.09152}
}