OpenAI’s New Image AI Nails Newton’s Spectrum and Draws Triangular Wheels, Too

OpenAI is replacing its image-generation AI model DALL-E 3 with “ChatGPT 4o Image Generation” (hereafter referred to as ChatGPT Image).

ChatGPT Image is powered by OpenAI’s generative AI model GPT-4, combining advanced text comprehension with high-level image generation capabilities. As a result, it can generate more complex and accurate images that better match demanding user requests compared to DALL-E 3. OpenAI plans to make ChatGPT Image available through ChatGPT soon.

On Tuesday, OpenAI briefed Korean correspondents and announced the launch of ChatGPT Image. It is OpenAI’s first new image-generation AI model in a year and six months since the release of DALL-E 3 in September 2023.

Even with detailed prompts, DALL-E 3 often failed to fully meet user needs, leading to increasing calls for improvement. According to OpenAI, ChatGPT Image represents a significant advancement over DALL-E 3 in terms of capabilities.

OpenAI first launched DALL-E in January 2021 and released an improved version, DALL-E 2, in April 2022. Since then, the company has continued to upgrade its image-generation AI. OpenAI stated that ChatGPT Image would fully replace DALL-E 3. However, the DALL-E 3 service will be phased out gradually to accommodate users who are familiar with it.

Gabriel Goh, an OpenAI employee in charge of multimodal operations (handling text, images, and audio simultaneously), emphasized that ChatGPT Image is not an upgraded version of DALL-E 3 but an entirely different image-generation AI model. He explained that this represents a shift in OpenAI’s image generation AI from novelty to practicality.

One of its biggest strengths is its ability to follow complex, multi-step user instructions. For example, when a user enters a prompt like “a blue star and a red triangle” into ChatGPT, ChatGPT Image accurately reflects the properties of each object to generate the exact image the user wants.

OpenAI explained that testing showed ChatGPT Image can accurately generate up to 15 different user-specified objects. It added that this exceeds the previous model’s performance, DALL-E 3.

Unlike DALL-E 3, ChatGPT Image has no difficulty generating images beyond common sense.

Jackie Shannon, another OpenAI employee working on multimodal operations, explained that when given a prompt like “a bicycle with triangular wheels,” ChatGPT Image does not struggle like DALL-E 3 but immediately generates an accurate image with triangular wheels. It can also create four-panel comics using precise text and character placement and develop new images by creatively reusing existing visuals from physicist Newton’s prism experiment.

Because of its advanced capabilities, OpenAI expects the AI model to benefit educational and graphic materials. In one example, ChatGPT Image accurately generated eight types of whales, including a blue whale for an academic poster, with all colors correctly rendered.

Shannon predicted that ChatGPT Image would help AI evolve from creating fun visual effects to becoming a practical tool. She added, “We’re excited that OpenAI is opening the door for anyone to create detailed and accurate visual materials easily.”

ChatGPT Image is available for free to all OpenAI users through ChatGPT. It is also used in OpenAI’s video-generation AI model, Sora. The tool responds in Korean as well. However, whether “ChatGPT Image” will perform as well in Korean as in English remains uncertain. OpenAI plans to update the model to ensure accurate performance in Korean and other languages.