CAFE · CC BY-SA 4.0
From Pixels to Concepts: Do Segmentation Models Understand What They Segment?
NeurIPS 2026 · Evaluations & Datasets Track (Poster)
A correct mask can still reflect the wrong concept. CAFE tests this with 2,146 paired examples that introduce misleading appearance, context, or material cues while preserving the target region and mask.
My contribution
I co-curated the benchmark and built CAFE-SAM3, an agent that inspects regions, refines prompts, and selects a matching mask or abstains. I also led follow-up evaluation and failure analysis, with mentoring from Profs. Xihui Liu and Han Wang.
BibTeX
@inproceedings{liang2026pixels,
title = {From Pixels to Concepts: Do Segmentation Models Understand What They Segment?},
author = {Liang, Shuang and Wang, Zeqing and Li, Yuxian and Liu, Xihui and Wang, Han},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS), Evaluations \& Datasets Track},
year = {2026},
url = {https://arxiv.org/abs/2605.09591}
}