无监督/半监督/对比学习

⚠️ 以下所有内容总结都来自于 大语言模型的能力，如有错误，仅供参考，谨慎使用
🔴 请注意：千万不要用于严肃的学术场景，只能用于论文阅读前的初筛！
💗 如果您觉得我们的项目对您有帮助 ChatPaperFree ，还请您给我们一些鼓励！⭐️ HuggingFace免费体验

2025-02-12 更新

Prototype Contrastive Consistency Learning for Semi-Supervised Medical Image Segmentation

Authors:Shihuan He, Zhihui Lai, Ruxin Wang, Heng Kong

Medical image segmentation is a crucial task in medical image analysis, but it can be very challenging especially when there are less labeled data but with large unlabeled data. Contrastive learning has proven to be effective for medical image segmentation in semi-supervised learning by constructing contrastive samples from partial pixels. However, although previous contrastive learning methods can mine semantic information from partial pixels within images, they ignore the whole context information of unlabeled images, which is very important to precise segmentation. In order to solve this problem, we propose a novel prototype contrastive learning method called Prototype Contrastive Consistency Segmentation (PCCS) for semi-supervised medical image segmentation. The core idea is to enforce the prototypes of the same semantic class to be closer and push the prototypes in different semantic classes far away from each other. Specifically, we construct a signed distance map and an uncertainty map from unlabeled images. The signed distance map is used to construct prototypes for contrastive learning, and then we estimate the prototype uncertainty from the uncertainty map as trade-off among prototypes. In order to obtain better prototypes, based on the student-teacher architecture, a new mechanism named prototype updating prototype is designed to assist in updating the prototypes for contrastive learning. In addition, we propose an uncertainty-consistency loss to mine more reliable information from unlabeled data. Extensive experiments on medical image segmentation demonstrate that PCCS achieves better segmentation performance than the state-of-the-art methods. The code is available at https://github.com/comphsh/PCCS.

医学图像分割是医学图像分析中的一项重要任务，但当标注数据较少而大量数据未标注时，这会变得非常具有挑战性。对比学习通过从部分像素构建对比样本，在医学图像分割的半监督学习中表现出良好的效果。然而，尽管之前的对比学习方法可以从图像内的部分像素中挖掘语义信息，但它们忽略了未标注图像的整体上下文信息，这对于精确分割非常重要。为了解决这个问题，我们提出了一种新型的半监督医学图像分割对比学习算法，称为原型对比一致性分割（PCCS）。其核心思想是将相同语义类的原型拉近，并将不同语义类的原型相互推远。具体来说，我们从未标注的图像中构建了一个带符号距离图和不确定性图。带符号距离图用于构建对比学习的原型，然后我们根据不确定性图估计原型的不确定性作为原型之间的权衡。为了获得更好的原型，我们基于学生-教师架构，设计了一种名为原型更新原型的新机制，以帮助更新对比学习的原型。此外，我们提出了一种不确定性一致性损失，以从未标注的数据中挖掘更可靠的信息。在医学图像分割方面的广泛实验表明，PCCS在分割性能上优于最先进的方法。代码可在https://github.com/comphsh/PCCS找到。

论文及项目相关链接

PDF 17 pages, 10 figures, 7 tables

Summary

本文提出了一种基于半监督学习的医学图像分割方法——原型对比一致性分割（PCCS）。该方法利用无标签图像的上下文信息，通过构建符号距离图和不确定性图来生成原型，并利用学生-教师架构更新原型，同时引入不确定性一致性损失，从无标签数据中挖掘更可靠的信息。实验证明，PCCS在医学图像分割上取得了更好的性能。

Key Takeaways

1. PCCS方法针对医学图像分割任务中的半监督学习环境设计。
2. 通过构建符号距离图和不确定性图，利用无标签图像的上下文信息。
3. 学生-教师架构用于更新对比学习的原型。
4. 引入原型对比一致性学习来拉近同类原型距离并推远不同类原型。
5. 提出不确定性一致性损失，以从无标签数据中获取更多可靠信息。
6. 实验结果显示，PCCS方法较现有技术取得了更好的医学图像分割性能。

Cool Papers

点此查看论文截图

Homeomorphism Prior for False Positive and Negative Problem in Medical Image Dense Contrastive Representation Learning

Authors:Yuting He, Boyu Wang, Rongjun Ge, Yang Chen, Guanyu Yang, Shuo Li

Dense contrastive representation learning (DCRL) has greatly improved the learning efficiency for image-dense prediction tasks, showing its great potential to reduce the large costs of medical image collection and dense annotation. However, the properties of medical images make unreliable correspondence discovery, bringing an open problem of large-scale false positive and negative (FP&N) pairs in DCRL. In this paper, we propose GEoMetric vIsual deNse sImilarity (GEMINI) learning which embeds the homeomorphism prior to DCRL and enables a reliable correspondence discovery for effective dense contrast. We propose a deformable homeomorphism learning (DHL) which models the homeomorphism of medical images and learns to estimate a deformable mapping to predict the pixels’ correspondence under topological preservation. It effectively reduces the searching space of pairing and drives an implicit and soft learning of negative pairs via a gradient. We also propose a geometric semantic similarity (GSS) which extracts semantic information in features to measure the alignment degree for the correspondence learning. It will promote the learning efficiency and performance of deformation, constructing positive pairs reliably. We implement two practical variants on two typical representation learning tasks in our experiments. Our promising results on seven datasets which outperform the existing methods show our great superiority. We will release our code on a companion link: https://github.com/YutingHe-list/GEMINI.

密集对比表示学习（DCRL）大大提高了图像密集预测任务的学习效率，显示出其在降低医学图像采集和密集标注的巨大成本方面的巨大潜力。然而，医学图像的特性导致了不可靠的对应关系发现，从而带来了DCRL中大规模正负（FP＆N）对开的难题。在本文中，我们提出了GEoMetric vIsual deNse sImilarity（GEMINI）学习，它将同胚先验知识嵌入到DCRL中，并为有效的密集对比提供了可靠的对应关系发现。我们提出了可变形同胚学习（DHL），对医学图像的同胚进行建模，并学习估计可变形映射以预测拓扑保持下的像素对应关系。这有效地减少了配对搜索空间，并通过梯度驱动了负对的隐式和软学习。我们还提出了几何语义相似性（GSS），用于提取特征中的语义信息来测量对应学习的对齐程度。它将提高变形的学习效率并提升性能，可靠地构建正对。我们在两个典型的表示学习任务上实现了两个实用变体，并在七个数据集上进行了实验验证，其表现优于现有方法的结果，显示了我们的巨大优势。我们将在相关链接上发布我们的代码：https://github.com/YutingHe-list/GEMINI。

论文及项目相关链接

PDF Accepted by T-PAMI 2025

Summary

本文提出了GEMINI学习，该学习在DCRL基础上结合了同胚先验知识，为高效的密集对比学习提供了一种可靠的对应关系发现方法。提出了变形同胚学习（DHL）来模拟医学图像的同胚性，学习预测在拓扑保护下的像素对应关系，有效减少了配对搜索空间，并通过梯度实现了隐性负对的学习。同时提出了几何语义相似性（GSS），用于提取特征中的语义信息来测量对应学习的对齐程度，提高了学习和变形的效率和性能，可靠地构建正对。实验在两个典型的表示学习任务上实施了两种实际应用变体，七个数据集上的有前景的结果表现出优越性能。相关代码将公开共享于https://github.com/YutingHe-list/GEMINI。

Key Takeaways

• DCRL对于图像密集预测任务提高了学习效率，但在医学图像中存在不可靠的对应关系发现问题。
• GEMINI学习结合了同胚先验知识到DCRL中，为有效的密集对比学习提供了可靠的对应关系发现方法。
• 变形同胚学习（DHL）模拟医学图像的同胚性，减少了配对搜索空间并实现了隐性负对的学习。
• 几何语义相似性（GSS）用于提取语义信息来测量对应学习的对齐程度，提高了学习和变形的效率和性能。
• 实验在两个典型的表示学习任务上实施了两种实际应用变体，并在七个数据集上表现出优越性能。

Cool Papers

点此查看论文截图

Detecting Backdoor Samples in Contrastive Language Image Pretraining

Authors:Hanxun Huang, Sarah Erfani, Yige Li, Xingjun Ma, James Bailey

Contrastive language-image pretraining (CLIP) has been found to be vulnerable to poisoning backdoor attacks where the adversary can achieve an almost perfect attack success rate on CLIP models by poisoning only 0.01% of the training dataset. This raises security concerns on the current practice of pretraining large-scale models on unscrutinized web data using CLIP. In this work, we analyze the representations of backdoor-poisoned samples learned by CLIP models and find that they exhibit unique characteristics in their local subspace, i.e., their local neighborhoods are far more sparse than that of clean samples. Based on this finding, we conduct a systematic study on detecting CLIP backdoor attacks and show that these attacks can be easily and efficiently detected by traditional density ratio-based local outlier detectors, whereas existing backdoor sample detection methods fail. Our experiments also reveal that an unintentional backdoor already exists in the original CC3M dataset and has been trained into a popular open-source model released by OpenCLIP. Based on our detector, one can clean up a million-scale web dataset (e.g., CC3M) efficiently within 15 minutes using 4 Nvidia A100 GPUs. The code is publicly available in our \href{https://github.com/HanxunH/Detect-CLIP-Backdoor-Samples}{GitHub repository}.

对比语言图像预训练（CLIP）容易受到后门攻击的影响，攻击者只需对训练数据集进行0.01%的污染，就能实现对CLIP模型的近乎完美的攻击成功率。这引发了人们对当前使用CLIP在未经严格审查的网页数据上预训练大规模模型的实践的担忧。在这项工作中，我们分析了被后门污染的样本在CLIP模型中学习的表示，发现它们在局部子空间具有独特特征，即它们的局部邻域比清洁样本更加稀疏。基于这一发现，我们对检测CLIP后门攻击进行了系统研究，并表明这些攻击可以通过传统的基于密度比的局部异常检测器轻松有效地检测出来，而现有的后门样本检测方法则失败了。我们的实验还表明，原始CC3M数据集中已经存在无意中的后门，并已训练成OpenCLIP发布的流行开源模型。基于我们的检测器，使用4个NVIDIA A100 GPU，可以在15分钟内高效地清理大规模的网页数据集（例如CC3M）。代码可在我们的GitHub仓库中找到：https://github.com/HanxunH/Detect-CLIP-Backdoor-Samples。

论文及项目相关链接

PDF ICLR2025

Summary
CLIP模型面临后门攻击威胁，仅需对训练集千分之一数据进行毒害即可实现近乎完美的攻击成功率。研究发现中毒样本在局部子空间具有独特特征，可利用基于密度比的局部异常检测器轻松高效检测这些攻击。原始CC3M数据集中存在无意中的后门，并已融入OpenCLIP发布的开源模型中。可使用我们的检测器在15分钟内高效清理大规模网络数据集（如CC3M）。

Key Takeaways

1. CLIP模型易受后门攻击，只需少量数据毒害即可实现高成功率攻击。
2. 中毒样本在局部子空间具有独特特征，表现为局部邻域更加稀疏。
3. 基于密度比的局部异常检测器可有效检测CLIP后门攻击。
4. 现有后门样本检测方法对CLIP模型效果不佳。
5. 原始CC3M数据集中存在无意中的后门。
6. 受影响的开源模型已融入OpenCLIP发布的模型中。

Cool Papers

点此查看论文截图