发布日期: 2025-05-02

更新日期: 2025-05-05

文章字数: 1.2k

阅读时长: 4 分

阅读次数:

⚠️ 以下所有内容总结都来自于大语言模型的能力，如有错误，仅供参考，谨慎使用
🔴 请注意：千万不要用于严肃的学术场景，只能用于论文阅读前的初筛！
💗 如果您觉得我们的项目对您有帮助 ChatPaperFree ，还请您给我们一些鼓励！⭐️ HuggingFace免费体验

2025-05-02 更新

MagicCraft: Natural Language-Driven Generation of Dynamic and Interactive 3D Objects for Commercial Metaverse Platforms

Authors:Ryutaro Kurai, Takefumi Hiraki, Yuichi Hiroi, Yutaro Hirao, Monica Perusquía-Hernández, Hideaki Uchiyama, Kiyoshi Kiyokawa

Metaverse platforms are rapidly evolving to provide immersive spaces for user interaction and content creation. However, the generation of dynamic and interactive 3D objects remains challenging due to the need for advanced 3D modeling and programming skills. To address this challenge, we present MagicCraft, a system that generates functional 3D objects from natural language prompts for metaverse platforms. MagicCraft uses generative AI models to manage the entire content creation pipeline: converting user text descriptions into images, transforming images into 3D models, predicting object behavior, and assigning necessary attributes and scripts. It also provides an interactive interface for users to refine generated objects by adjusting features such as orientation, scale, seating positions, and grip points. Implemented on Cluster, a commercial metaverse platform, MagicCraft was evaluated by 7 expert CG designers and 51 general users. Results show that MagicCraft significantly reduces the time and skill required to create 3D objects. Users with no prior experience in 3D modeling or programming successfully created complex, interactive objects and deployed them in the metaverse. Expert feedback highlighted the system’s potential to improve content creation workflows and support rapid prototyping. By integrating AI-generated content into metaverse platforms, MagicCraft makes 3D content creation more accessible.

元宇宙平台正在迅速演变，为用户提供沉浸式交互和内容创作空间。然而，生成动态且可交互的3D对象仍然是一个挑战，因为需要高级的3D建模和编程技能。为了应对这一挑战，我们推出了MagicCraft系统，该系统可根据自然语言提示为元宇宙平台生成功能性3D对象。MagicCraft使用生成式AI模型管理整个内容创作管道：将用户文本描述转换为图像，将图像转换为3D模型，预测对象行为，并分配必要的属性和脚本。它还提供了一个交互式界面，使用户可以通过调整方向、比例、座位位置和握持点等特征来完善生成的物体。MagicCraft在集群（一种商业元宇宙平台）上实施，经过7名专业CG设计师和51名普通用户的评估。结果表明，MagicCraft大大减少了创建3D对象所需的时间和技能。没有3D建模或编程经验的用户也能成功创建复杂的交互式对象并将其部署在元宇宙中。专家反馈突出了该系统改进内容创作工作流程和支持快速原型的潜力。通过将AI生成的内容集成到元宇宙平台中，MagicCraft使3D内容创作更加便捷。

论文及项目相关链接

PDF 16 pages. Preprint for submission to IEEE Access

Summary

MagicCraft系统通过自然语言提示为元宇宙平台生成功能性的3D对象，简化了内容创作流程。该系统使用生成式AI模型管理整个内容创作管道，包括将用户文本描述转化为图像，将图像转化为3D模型，预测对象行为，并分配必要的属性和脚本。同时，提供互动界面让用户调整生成物体的特性，如方向、大小、座位位置和抓握点等。在商用元宇宙平台Cluster上的实施效果评估显示，MagicCraft显著减少了创建3D对象所需的时间和技能，即使是初学者也能成功创建复杂的互动物体并在元宇宙中部署。专家反馈认为该系统有潜力改善内容创作流程并支持快速原型设计，将AI生成的内容整合到元宇宙平台，使3D内容创作更加便捷。

Key Takeaways