Dynamically Instance-Guided Adaptation: A Backward-free Approach for Test-Time Domain Adaptive Semantic Segmentation
Abstract
Semantic segmentation models often fail when deployed in new target domains due to domain shifts. Test-Time Domain Adaptation for Semantic Segmentation (TTDA-Seg) aims to adapt models efficiently during inference without target labels, but existing methods struggle with efficiency (requiring backward optimization) or effectiveness (often inadequately addressing semantic shifts), and can suffer from error accumulation. This thesis proposes Dynamically Instance-Guided Adaptation (DIGA), a novel, efficient, and backward-free approach for TTDA-Seg. DIGA leverages information from each test instance to guide its adaptation without requiring backpropagation, thus avoiding costly optimization and error accumulation. The method consists of two main components: a Distribution Adaptation Module (DAM) that adaptively combines source and instance Batch Normalization (BN) statistics for robust feature representation, and a Semantic Adaptation Module (SAM) which constructs a dynamic non-parametric classifier using historical and instance-aware prototypes to refine semantic predictions. Evaluated on five diverse benchmark datasets, DIGA outperforms state-of-the-art TTDA-Seg methods in segmentation accuracy while maintaining high computational efficiency. This work presents DIGA as a robust and practical approach for Test-Time Domain Adaptation in semantic segmentation, highlighting the effectiveness of instance-guided adaptation for handling domain shifts during inference.