GLACIA: Instance-Aware Positional Reasoning for Glacial Lake Segmentation via Multimodal Large Language Model

Abstract

Glacial lake monitoring bears great significance in mitigating the anticipated risk of Glacial Lake Outburst Floods. However, existing segmentation methods based on convolutional neural networks (CNNs) and Vision Transformers (ViTs), remain constrained to pixel-level predictions, lacking high-level global scene semantics and human-interpretable reasoning. To address this, we introduce GLACIA (Glacial LAke segmentation with Contextual Instance Awareness), the first framework that integrates large language models with segmentation capabilities to produce both accurate segmentation masks and corresponding spatial reasoning outputs. We construct the Glacial Lake Position Reasoning (GLake-Pos) dataset pipeline, which provides diverse, spatially grounded question–answer pairs designed to overcome the lack of instance-aware positional reasoning data in remote sensing. Comparative evaluation demonstrate that GLACIA (mIoU: 87.30) surpasses state-of-the-art method based on CNNs (mIoU: 78.55 – 79.01), ViTs (mIoU: 69.27 – 81.75), Geo-foundation models (mIoU: 76.37 – 87.10), and reasoning based segmentation methods (mIoU: 60.12 – 75.66). Our approach enables intuitive disaster preparedness and informed policy-making in the context of rapidly changing glacial environments by facilitating natural language interaction, thereby supporting more efficient and interpretable decision-making.

Motivation

Conceptual shift from traditional segmentation (a) and VQA-based reasoning (b) to our reasoning-driven paradigm (c), which unifies accurate instance-specific masks with interpretable positional reasoning .

Our paradigm (Fig(c)) unifies both, producing interpretable positional reasoning and accurate, instance- specific masks. This design introduces two core capabilities:
(1) multimodal reasoning, which integrates LLM- derived lake counts, quadrant cues, and relative positioning (e.g., “near the center,” “top-right”) with visual evidence; and

(2) instance-aware spatial localization, which generates distinct segmentation masks for each lake based on reasoning-derived positions.

Method

The proposed architecture (Fig) integrates multispectral feature extraction with multimodal reasoning to enable precise glacial lake segmentation. The Prithvi-Res encoder combines a ResNet-34 stem adapted for six-channel input with selected transformer layers from Prithvi-EO v2, fusing local textures and global context into compact features (Fig (a)). In parallel, a multimodal LLM processes RGB imagery and textual instructions, generating positional segmentation-specific tokens adapted with LoRA (Fig. (b)). These tokens condition the Prompt Mask Decoder, which aligns semantic prompts with multispectral features through cross-attention, self-attention, and upsampling (Fig. (c)). Joint optimization with segmentation and text generation losses ensures accurate spatial masks while preserving semantic reasoning. .

Results

Our proposed GLACIA model outperforms all other deep learning approaches, including GFMs (Prithvi and DOFA), ViT-based model (TransNorm), the hybrid model (UViT), and U-Net in both single-lake and multi-lake scenarios

Qualitative comparison of glacial lake segmentation. Red boxes indicate false positives, blue boxes indicate false negatives. Our reasoning-enhanced model accurately captures small and irregular lakes, reducing both errors compared to baseline methods.

BibTeX

@article{maurya2025glacia,
  title={GLACIA: Instance-Aware Positional Reasoning for Glacial Lake Segmentation via Multimodal Large Language Model},
  author={Maurya, Lalit and Kaushik, Saurabh and Tellman, Beth},
  journal={arXiv preprint arXiv:2512.09251},
  year={2025}
}