发布日期: 2025-04-29

更新日期: 2025-04-30

文章字数: 1.3k

阅读时长: 5 分

阅读次数:

⚠️ 以下所有内容总结都来自于大语言模型的能力，如有错误，仅供参考，谨慎使用
🔴 请注意：千万不要用于严肃的学术场景，只能用于论文阅读前的初筛！
💗 如果您觉得我们的项目对您有帮助 ChatPaperFree ，还请您给我们一些鼓励！⭐️ HuggingFace免费体验

2025-04-29 更新

DreamID: High-Fidelity and Fast diffusion-based Face Swapping via Triplet ID Group Learning

Authors:Fulong Ye, Miao Hua, Pengze Zhang, Xinghui Li, Qichao Sun, Songtao Zhao, Qian He, Xinglong Wu

In this paper, we introduce DreamID, a diffusion-based face swapping model that achieves high levels of ID similarity, attribute preservation, image fidelity, and fast inference speed. Unlike the typical face swapping training process, which often relies on implicit supervision and struggles to achieve satisfactory results. DreamID establishes explicit supervision for face swapping by constructing Triplet ID Group data, significantly enhancing identity similarity and attribute preservation. The iterative nature of diffusion models poses challenges for utilizing efficient image-space loss functions, as performing time-consuming multi-step sampling to obtain the generated image during training is impractical. To address this issue, we leverage the accelerated diffusion model SD Turbo, reducing the inference steps to a single iteration, enabling efficient pixel-level end-to-end training with explicit Triplet ID Group supervision. Additionally, we propose an improved diffusion-based model architecture comprising SwapNet, FaceNet, and ID Adapter. This robust architecture fully unlocks the power of the Triplet ID Group explicit supervision. Finally, to further extend our method, we explicitly modify the Triplet ID Group data during training to fine-tune and preserve specific attributes, such as glasses and face shape. Extensive experiments demonstrate that DreamID outperforms state-of-the-art methods in terms of identity similarity, pose and expression preservation, and image fidelity. Overall, DreamID achieves high-quality face swapping results at 512*512 resolution in just 0.6 seconds and performs exceptionally well in challenging scenarios such as complex lighting, large angles, and occlusions.

本文介绍了DreamID，这是一款基于扩散技术的脸交换模型，能够实现高水平的身份相似性、属性保留、图像保真度和快速的推理速度。与传统的脸交换训练过程不同，后者经常依赖隐式监督并且难以实现令人满意的结果。DreamID通过构建Triplet ID Group数据建立脸交换的显式监督，从而显著提高身份相似性和属性保留。扩散模型的迭代性质给利用高效的图像空间损失函数带来了挑战，因为在训练期间进行耗时的多步采样来获得生成的图像是不切实际的。为了解决这一问题，我们利用加速扩散模型SD Turbo，将推理步骤减少到单次迭代，实现了具有显式Triplet ID Group监督的高效像素级端到端训练。此外，我们提出了改进的基于扩散的模型架构，包括SwapNet、FaceNet和ID Adapter。这一稳健的架构充分释放了Triplet ID Group显式监督的威力。最后，为了进一步扩大我们的方法的应用范围，我们在训练过程中显式地修改Triplet ID Group数据，以微调并保留特定的属性，如眼镜、脸型等。大量实验表明，DreamID在身份相似性、姿势和表情保留以及图像保真度方面均优于现有先进技术。总体而言，DreamID在512*512分辨率下实现了高质量的脸部交换结果，仅耗时0.6秒，并且在复杂光照、大角度和遮挡等挑战场景下表现尤为出色。

摘要

本文介绍了DreamID，一种基于扩散的面貌换脸模型，实现了高水平的身份相似性、属性保留、图像保真度和快速推理速度。通过构建Triplet ID Group数据实现显式监督，提高了身份相似性和属性保留。借助加速扩散模型SD Turbo，减少了推理步骤至单次迭代，实现了具有显式Triplet ID Group监督的高效像素级端到端训练。此外，我们提出了改进型的基于扩散的模型架构，包括SwapNet、FaceNet和ID Adapter，充分发挥了Triplet ID Group显式监督的威力。最后，我们通过在训练过程中明确修改Triplet ID Group数据，进一步改进了方法，以微调并保留特定属性，如眼镜和脸型。实验表明，DreamID在身份相似性、姿势和表情保留以及图像保真度方面优于现有技术。总体而言，DreamID在512*512分辨率下实现了高质量的面貌换脸结果，仅需0.6秒，并在复杂光照、大角度和遮挡等挑战场景中表现优异。

关键见解