Jun 27, 2023
We're incredibly excited to introduce the first-ever production AI system that can turn complex 2D images into 3D models. Turning any flat picture of an object into a 3D model has been unsolved for decades. This is because a single image doesn't give us much information about depth or how things should look from different angles. Plus, creating a 3D model from scratch is a complex process that requires specific skills and can take even professional artists and game developers hours or even days to complete just one asset. This makes it impossible for an average consumer to generate 3D content and requires 1000s of dollars per production grade asset. We are solving this problem and making it widely available to everyone.
Turning images into game-engine ready 3D assets has massive implications for gaming, robotics, mixed reality, VFX and e-commerce. Moreover, this new technology opens up a world of possibilities for anyone with a spark of imagination.
Picture this: You have a brilliant idea for an object, say a piece of furniture or a fantastical creature. All you need to do is visualize it in your mind, maybe even sketch it out. You could also use tools like Midjourney, stable diffusion, or Dall-E to transform your concept into a tangible image form. No need for high-level technical knowledge, no need for advanced 3D modelling skills. These cutting-edge tools combined with our new AI system bridge the gap between imagination and reality. You simply visualize, and the AI takes care of the rest, crafting your ideas into detailed 3D models.
Capability
Our focus is on transforming images into 3D models because, compared to text, images retain more stylistic details and suffer less loss of information. An image-based approach is not only more intuitive but also aligns with the existing workflows of concept artists in real-world 3D content production. Plus, creating images from text with AI models is already an easy task, making this process more seamless.
In this blog post, we specifically highlight gaming applications but our approach is general-purpose and can scale to long-tail applications. Our outputs are better than existing systems and work on a wide range of objects and characters. Compared to the latest Shape-E model from openAI, our model does significantly better and is improving every day.
An artist can use these AI-generated models either as a 3D blockout to create a production-ready asset or directly use the outputs.
Rate of Progress
We develop our foundation AI models and control the entire process, allowing us to quickly enhance and refine our systems. 3D generative models, especially text to 3D models, are prone to hallucinations – such as extra heads or parts. Our solution greatly mitigates this problem and this keeps getting better with more compute, data and feedback loops.