GPT-3 can Create Incredible AI Image Prompts

What is GPT-3?


GPT-3 (Generative Pre-trained Transformer 3) is a large-scale language model developed by OpenAI. It is a deep learning-based natural language processing (NLP) model that uses unsupervised learning to generate human-like text. GPT-3 is trained on a massive amount of text data from the internet and is designed to predict the next word in a sentence. It can also generate short passages of text, which can be used for summarization and question-answering. GPT-3 is the largest and most powerful language model ever created and is capable of understanding, generating, and manipulating natural language.

GPT-3 can be prompted to generate many realistic answers to chat-based input, including responding in poem, with factual information, and with usable code. There is still a long way to go before the model is perfect, as it’s poems aren’t great, will often confidently state incorrect facts and will provide code that doesn’t run. Despite these shortcomings, GPT-3 is a huge stepping stone in the direction of incredible AI tools to improve society.

Learn more by visiting their site: OpenAI

How can GPT-3 create incredible AI Image Prompts

GPT-3 can be prompted to output a set of words that describe the image similarly to how we would prompt for an image. Although the AI doesn’t have an idea of which words exist in the model, due to the vast dataset of Stable Diffusion, often we’re able to create great looking art.

In this example, we’ll be using MitchJourn-E to create the Stable Diffusion images and the beta app for GPT-3 Chat.

To get GPT-3 to create these prompts, I’ll start by specifying “Using only nouns and adjectives”. That should eliminate any connecting words that aren’t as important in image prompting. Then I’ll ask “describe an incredible work of art”, which will start returning the output to plug into our image generation apps. Sometimes it will only output a few words, so I’ll also put at the end “Using 50 words or as many words as possible”, something to get it to output some more useful words.

Examples:

GPT-3 prompt:

Using only nouns and adjectives, describe an incredible and detailed painting of a landscape with as many words as possible.

Output 1:

Majestic mountains tower over a sprawling, verdant valley. A crystal-clear river winds its way through the landscape, cascading over rocks and boulders as it flows towards a distant, azure sea. Wildflowers dot the rolling hills, and a single, graceful tree stands tall on a grassy knoll, its leaves rustling softly in the gentle breeze. The sky is a deep, rich blue, with fluffy white clouds drifting lazily overhead. The whole scene is bathed in the warm, golden light of the setting sun, casting long, surreal shadows across the landscape. It is a truly breathtaking and awe-inspiring sight.

GPT-3 prompt:

Using only nouns and adjectives, describe an incredible and detailed painting of a landscape with 50 words.

Output 2:

Mountains, cliffs, sky, clouds, sun, trees, grass, river, rocks, waterfalls, wildlife, flowers, colors, beauty, serenity, majesty, vastness, depth, perspective, realism, skill, talent, artistry, wonder, amazement, awe, admiration.

Output 3:

Vibrant, colorful sky stretches above rolling hills and tranquil lake. Majestic mountains tower in the distance. Lush, verdant forests and fields dotted with wildflowers. Sun shining, birds singing. Tranquil scene captured with stunning detail and precision.

GPT-3 prompt:

Using only nouns and adjectives, describe an incredible and detailed abstract painting inspired by a landscape with 50 words

Output 4:

Vibrant hues, sweeping brushstrokes, and majestic peaks form an awe-inspiring abstract landscape painting. Lush forests, sparkling waters, and towering mountains are captured in a mesmerizing display of color and texture. A sense of majesty and wonder permeates the artwork, drawing the viewer into a world of beauty and inspiration.

Additional Prompt Info

Negative prompt:

lowres text error cropped worst quality low quality normal quality jpeg artifacts signature watermark username blurry artist name deformed disfigured poorly drawn out of focus censorship amateur drawing bad art poor art messy drawing portrait person people text signature watermark frame framed display

Steps: 50

One of the seeds: 902193268

Width: 640

Height: 384

Prompt weight (CFG scale): 7.5

Sampler: k_euler_a

Output Images

We can use these GPT-3 outputs and plug them into MitchJourn-E to create some beautiful images:

Output 1:

Majestic mountains tower over a sprawling, verdant valley. A crystal-clear river winds its way through the landscape, cascading over rocks and boulders as it flows towards a distant, azure sea. Wildflowers dot the rolling hills, and a single, graceful tree stands tall on a grassy knoll, its leaves rustling softly in the gentle breeze. The sky is a deep, rich blue, with fluffy white clouds drifting lazily overhead. The whole scene is bathed in the warm, golden light of the setting sun, casting long, surreal shadows across the landscape. It is a truly breathtaking and awe-inspiring sight.
Majestic mountains tower over a sprawling, verdant valley. A crystal-clear river winds its way through the landscape, cascading over rocks and boulders as it flows towards a distant, azure sea. Wildflowers dot the rolling hills, and a single, graceful tree stands tall on a grassy knoll, its leaves rustling softly in the gentle breeze. The sky is a deep, rich blue, with fluffy white clouds drifting lazily overhead. The whole scene is bathed in the warm, golden light of the setting sun, casting long, surreal shadows across the landscape. It is a truly breathtaking and awe-inspiring sight.
Majestic mountains tower over a sprawling, verdant valley. A crystal-clear river winds its way through the landscape, cascading over rocks and boulders as it flows towards a distant, azure sea. Wildflowers dot the rolling hills, and a single, graceful tree stands tall on a grassy knoll, its leaves rustling softly in the gentle breeze. The sky is a deep, rich blue, with fluffy white clouds drifting lazily overhead. The whole scene is bathed in the warm, golden light of the setting sun, casting long, surreal shadows across the landscape. It is a truly breathtaking and awe-inspiring sight.
Majestic mountains tower over a sprawling, verdant valley. A crystal-clear river winds its way through the landscape, cascading over rocks and boulders as it flows towards a distant, azure sea. Wildflowers dot the rolling hills, and a single, graceful tree stands tall on a grassy knoll, its leaves rustling softly in the gentle breeze. The sky is a deep, rich blue, with fluffy white clouds drifting lazily overhead. The whole scene is bathed in the warm, golden light of the setting sun, casting long, surreal shadows across the landscape. It is a truly breathtaking and awe-inspiring sight.

Output 2:

Mountains, cliffs, sky, clouds, sun, trees, grass, river, rocks, waterfalls, wildlife, flowers, colors, beauty, serenity, majesty, vastness, depth, perspective, realism, skill, talent, artistry, wonder, amazement, awe, admiration.
Mountains, cliffs, sky, clouds, sun, trees, grass, river, rocks, waterfalls, wildlife, flowers, colors, beauty, serenity, majesty, vastness, depth, perspective, realism, skill, talent, artistry, wonder, amazement, awe, admiration.
Mountains, cliffs, sky, clouds, sun, trees, grass, river, rocks, waterfalls, wildlife, flowers, colors, beauty, serenity, majesty, vastness, depth, perspective, realism, skill, talent, artistry, wonder, amazement, awe, admiration.
Mountains, cliffs, sky, clouds, sun, trees, grass, river, rocks, waterfalls, wildlife, flowers, colors, beauty, serenity, majesty, vastness, depth, perspective, realism, skill, talent, artistry, wonder, amazement, awe, admiration.

Output 3:

Vibrant, colorful sky stretches above rolling hills and tranquil lake. Majestic mountains tower in the distance. Lush, verdant forests and fields dotted with wildflowers. Sun shining, birds singing. Tranquil scene captured with stunning detail and precision.
Vibrant, colorful sky stretches above rolling hills and tranquil lake. Majestic mountains tower in the distance. Lush, verdant forests and fields dotted with wildflowers. Sun shining, birds singing. Tranquil scene captured with stunning detail and precision.
Vibrant, colorful sky stretches above rolling hills and tranquil lake. Majestic mountains tower in the distance. Lush, verdant forests and fields dotted with wildflowers. Sun shining, birds singing. Tranquil scene captured with stunning detail and precision.
Vibrant, colorful sky stretches above rolling hills and tranquil lake. Majestic mountains tower in the distance. Lush, verdant forests and fields dotted with wildflowers. Sun shining, birds singing. Tranquil scene captured with stunning detail and precision.

Output 4:

Vibrant hues, sweeping brushstrokes, and majestic peaks form an awe-inspiring abstract landscape painting. Lush forests, sparkling waters, and towering mountains are captured in a mesmerizing display of color and texture. A sense of majesty and wonder permeates the artwork, drawing the viewer into a world of beauty and inspiration.
Vibrant hues, sweeping brushstrokes, and majestic peaks form an awe-inspiring abstract landscape painting. Lush forests, sparkling waters, and towering mountains are captured in a mesmerizing display of color and texture. A sense of majesty and wonder permeates the artwork, drawing the viewer into a world of beauty and inspiration.
Vibrant hues, sweeping brushstrokes, and majestic peaks form an awe-inspiring abstract landscape painting. Lush forests, sparkling waters, and towering mountains are captured in a mesmerizing display of color and texture. A sense of majesty and wonder permeates the artwork, drawing the viewer into a world of beauty and inspiration.
Vibrant hues, sweeping brushstrokes, and majestic peaks form an awe-inspiring abstract landscape painting. Lush forests, sparkling waters, and towering mountains are captured in a mesmerizing display of color and texture. A sense of majesty and wonder permeates the artwork, drawing the viewer into a world of beauty and inspiration.

Create your own beautiful art

MitchJourn-E is a free and open source tool available to download on GitHub. Paste your GPT-3 prompts directly into the interface and start generated the most AI AI images you’ve ever seen!

Download now from GitHub: https://github.com/MitchOkabe/MitchJourn-e

What is Prompt Engineering?

Simply put, prompt engineering is the process of carefully crafting and tuning prompts for AI generation. A prompt is a piece of input that an AI system uses to generate output, such as a text, image, or audio clip. By carefully designing and fine-tuning these prompts, we can help AI systems to generate more accurate and engaging outputs.

Now, you might be wondering why prompt engineering is so important. After all, don’t AI systems generate whatever you type into them? While it’s true that AI systems can generate stunning outputs by simply typing whatever you wish into an input prompt, the results can often be unstructured and unpredictable. By providing carefully designed prompts, we can guide the AI system and help it to generate more coherent and useful outputs.

For example, imagine that you’re building an AI system that’s capable of generating pictures. Without prompts, how would it generate a scene of “A statue on the side of a cliff in Japan”? It may choose to generate a beautiful statue, then put the cliff in the background to the side, and the setting is Japan. Or it may have been trained that pictures of Japan, that are by cliff sides, are usually accompanied by Japanese architecture, so a traditional Japanese building becomes a focal point instead of the statue. However, by providing carefully designed prompts, such that different concepts are more or less weighted based on how the model is trained, the AI can create incredible and well imagined art.

An image without prompt engineering.
incredibly detailed (A statue on the side of a cliff in Japan)1  (cinematic lighting hyperdetailed hd 4k 8k sharp focus highly detailed concept art splash art by greg rutkowski):1
(A statue on the side of a cliff in Japan)1
An image with prompt engineering.
incredibly detailed (A statue)1.3 (on the side of a cliff )1 (in Japan)0.7  (cinematic lighting hyperdetailed hd 4k 8k sharp focus highly detailed concept art splash art by greg rutkowski):1
(A statue)1.3 (on the side of a cliff )1 (in Japan)0.7

Prompt engineering is an essential aspect of AI generation. By carefully designing and fine-tuning prompts, we can help AI systems generate more accurate and engaging outputs. Without prompt engineering, AI systems will generate results based on how the data was trained and will create biases in the resulting work. Prompt engineering allows us to guide the AI system and create more coherent, useful and artful outputs.

Start Creating, like a prompt engineer!

MitchJourn-E is a tool that incorporates Prompt Engineering as a core concept when creating images. You can define weights for each part of your prompt and add helper words and negative prompts automatically. MitchJourn-E is free and open source on GitHub: MitchOkabe/MitchJourn-e: Windows desktop application for Stable Diffusion (github.com)

Image Generator: MitchJourn-E

Introducing MitchJourn-E! The text-to-image generator app for Windows. This app aims to enable intuitive, fast, and exciting ways to generate new images, using the power of InvokeAI and Stable Diffusion on Windows desktop.

What is MitchJourn-E? The name is a play on words, referencing the popular image generation services “MidJourney” and “Dall-E”. This app provides an easy-to-use desktop interface for the popular Stable Diffusion fork called “InvokeAI“. MitchJourn-E is an open-source Windows desktop application that allows you to type in a description of an image and the app will create that image using the Stable Diffusion model.

A screenshot of the MitchJourn-E application showing an image generated from the prompt "a beautiful scenic landscape of a Hawaiian sunset"
A screenshot of the MitchJourn-E application

Download for Free

MitchJourn-E is an open-source free download available from GitHub. Click the green “Code” button and choose to clone or download as a zip. Within the downloaded folder, run MitchJourn-e.exe.

Generated Image Examples

Now I'm fucked up and I'm missing you

Features

Text-to-Image: enter text prompts and generate an image

Image-to-Image: reference an existing image to create images with similar composition, style, and size

Create variations: reference a previously generated or local image and create similar but different variations of it

Negative prompting: enter prompts that are banned from generation, usually generating inverse results. Combine with regular and image prompting to fine-tune results

Prompt blending: define two separate weighted prompts to generate an image of intersecting concepts

Prompt Helper: a customizable menu to quickly add commonly used phrases, artist names and styles to speed up workflow

Continuous prompting: Images can be generated one after another, while changing the prompt and the parameters of the prompt to fine-tune the image

Sequential prompting: Images can be generated based on the previous image, allowing prior results shape future results

Upscaling: generate images quickly at low resolution, then upscale them after, for an overall fast way to generate full sized images

High Resolution generation: Generate images at resolutions higher than usually possible, using a 2-phase approach of generating a smaller image, upscaling it, then using the upscaled image as a reference, generate the high-resolution image

Aspect ratio presets: define presets (or use the defaults) to create images in common ratios like 16:9, 2:3, 1:1

Creativity slider: quickly modify multiple parameters with one slider movement that will result in more and less “creative” results, useful for moving between photo-realism and fantasy

Prompt noise: add Perlin noise to the prompt to subtly (or drastically) modify the starting point of the image generation

Latent limiter: limit the amount of data available to the generation, which can generate more photo-realistic/true to source images

Seamless tiling: generate images that can seamlessly tile together, great for clothing patterns and game assets

Generate progress images: save every step the generation makes before creating the end result (for you noise lovers!)

Output sorting (experimental feature): automatically sort generated images into folders based on the prompt used

Prompt folder (experimental feature): reference a folder of images and generate new images referencing each image in that folder sequentially. Good for creating videos

Out-painting (experimental feature): generate additional resolution to existing images, based on the prompt

Metadata Extractor (experimental feature): reference an image and view all the associated metadata. Typically, images generated with Stable Diffusion have the prompt and other parameter info saved within the metadata

Installation

  1. Download and install InvokeAI to the root of the C drive. Follow this separate setup guide (easy) to install Anaconda, Python, and all the other prerequisites automatically: https://invoke-ai.github.io/InvokeAI/installation/INSTALL_INVOKE/#installation
    • When complete, you should have a folder structure like C:\InvokeAI-Main\
  2. Now you need to download a model file. Here is Stable Diffusion 1.5 (v1-5-pruned-emaonly.ckpt) (recommended for best general results). You will need to create a hugging face account before it shows the download link: https://huggingface.co/runwayml/stable-diffusion-v1-5
  3. Rename the .ckpt file to “model.ckpt” and place it here: C:\InvokeAI-main\models\ldm\
  4. Now relaunch MitchJourn-E, and you can start generating images!

Where credit is due

Thanks to Emad Mostaque and his team for creating the Stable Diffusion platform and model which this application uses as it’s backbone. Additionally, thanks to Lincoln Stein and contributors for creating the Stable Diffusion fork which this application leans on for the majority of its image processing features.

This application is a user interface that leverages the hard work of many creators to create beautiful art. In keeping with its predecessors, this project is open source and can be remixed and added to. Feel free to fork and collaborate using GitHub.

Thanks for reading ✌