AI Data Vision

What is DALL-E 2 ?

DALL-E is an AI system that can create realistic images and art from a description in natural language created by Open AI (an AI research and deployment company also known for ChatGPT).

For those who wonder, DALL-E 2 is an improved version of DALL-E 1, with more capacity, higher resolution, and greater diversity in image generation (DALL-E 1 vs. DALL-E 2).

Generate variations of a well-known painting

DALL-E 2 is able to generate variations from an original image. Here I generated variations of the famous “Girl with a Pearl Earring” by Johannes Vermeer.

The produced variations have a common set of characteristics: a young white woman, dressed in blue and yellow tones, her face turned three-quarter style on a black background.

DALL-E 2 learned from the composition of the input image to generate a set of images meeting these criteria. Finally, if you describe the painting to someone who has never seen it, would they recognize the original?

DALL-E can also add context around your input image. Maybe that girl was taking a selfie?

DALL-E can regenerate a part of an image. To continue with my example, let’s say the girl was a rabbit.

Let’s revisit Grant Wood’s American Gothic

I propose to push the idea of augmenting a painting with several iterations of image context generation. I was able to create my own version of Grant Wood’s American Gothic.

For those who would like a little explanation, here are the requests I pushed to DALL-E regarding the generation:

• “Another house with a dog on the grass”.
• “The roof of the house with a UFO”.
• “The woman is holding a wand”.
• “The sun shine and there is a rooster on the roof”.
• “A meteor approches”.

Conclusion

The future of Art

Through these illustrations, DALL-E demonstrates its ability to accelerate the creative process. DALL-E has been used to create video clips (although Meta’s Make-A-Video is more suitable, but not yet available to the general public). It has also been used to create magazine covers.

DALL-E and more generally AI questions the relationship we have to art. Will the improving technology surpass our creativity?

DALL-E does not understand references to physical things in our world. For example, when I ask DALL-E to generate the image corresponding to “the girl is taking a selfie”, it doesn’t understand that the smartphone is used to take a photograph; but it recognizes that the image of an arm stretched out in front of the person with the phone facing the person matches my statement.

DALL-E has very few ability to generalize (like many AI) and can only extrapolate at best. This prevents it from generalizing to new situations, new styles or generating from new ideas.

Ethic considerations

DALL-E has also been pointed to racial and gender bias. For example, the term “Flight attendant” refers almost exclusively to images of women. Open AI still needs to work to ensure the representations that these tools can convey.