Three months after the release of DALL-E 2, OpenAI has implemented a new technique that is supposed to bring more diversity to the representation of people. The measure is intended to create images that do not follow prejudices or a majority presence, for example in relation to certain professional groups, but rather represent the diversity of the world population.
The technique works directly at the system level when DALL-E 2 receives a request to represent a person without gender or ethnicity, such as “Firefighter” or “CEO”. In the blog post examples, the system generated six white males before the adjustment for the former category. With the new technique, it spat out the portraits of four men and two women of different ethnic backgrounds.
Pride vs Prejudice
Bias is one of the biggest problems in artificial intelligence. Machine learning models are based on human-made training data, and there is a risk of the system adopting biases. In 2018, for example, Amazon scrapped its ML-based evaluation system for applications because it discriminated against female applicants compared to male applicants.
Before the new technology, all of the six images created with the keyword “Teacher” were female and know what will change with the new technology.
For generative models that create their own content such as text or images, there is also the risk that users can attempt to manipulate them or use them for content that glorifies violence or is sexist, for example. OpenAI only publicly released the GPT-3 language model after a relatively long closed phase. One reason was that the company initially wanted to reduce bias. An analysis in early 2021 showed that the model had adopted deeply rooted prejudices against Muslims from the training data. In February, OpenAI published InstructGPT, an adapted variant of GPT-3 that uses human feedback to exclude certain topics such as sexual content or violence.
For DALL-E 2, OpenAI has taken a number of precautionary measures since it was first introduced to prevent the generation of inappropriate or prejudiced images. Among other things, the company has filtered the training set so that it does not offer templates for violent, hateful or sexual content. In addition, the company’s content policy prohibits the generation of such content. On the technical side, a filter for text input and uploaded images is intended to ward off abuse of the system. According to OpenAI, the measures have been expanded in recent months.
DALL-E 2 creates images as instructed
DALL-E is an AI system that creates images based on descriptions. OpenAI released the first version in January 2021. It uses the GPT-3 language model, which also comes from OpenAI. While the latter draws its basic knowledge from a large collection of texts, OpenAI has trained DALL-E and its successors with numerous images and associated descriptions. The system uses the basis to generate, for example, an Andy Warhol-style astronaut on a horse. The name DALL-E is a portmanteau of Spanish artist Salvador Dali’s surname and the title of the Pixar film “WALL-E”.
The successor DALL-E 2, released in April 2022, combines two techniques that OpenAI has developed since the release of the first variant: CLIP (Contrastive Language-Image Pre-training), an artificial neural network that converts visual concepts into categories, and GLIDE (Guided Language to Image Diffusion for Generation and Editing), a text-guided diffusion model that, according to a paper, outperformed DALL-E primarily in the areas of photorealism and apt description.
Unlike its predecessor, DALL-E 2 can subsequently change images and add contextual content. In addition, the system can redesign existing images. The project page shows variations of well-known works of art such as “The Girl with a Pearl Earring” by Jan Vermeer van Delft or “The Kiss” by Gustav Klimt.
Images generated by DALL-E 2 in the style of the expressionist Egon Schiele are to be shown in the museum.
Recently OpenAI presented creations of some artists in a blog post. Among other things, the Austrian writer and curator Stefan Kutzenberger used DALL-E 2 to create pictures in the style of the artist Egon Schiele, who died at the age of 28, which he would like to exhibit in Vienna’s Leopold Museum next to the artist’s real paintings.
Further details on the measures for more diversity can be found in the OpenAI blog. The system remains in a private testing phase, and according to a May update, the company is adding about 1,000 additional testers to the waitlist each week. An article on DALL-E 2 by an author who has access to the AI system will soon be published on heise Developer.
To home page