At CODE, we use AI to generate art. This may be the cover of a magazine, but most of our art is crated for web uses. (Antoher example would be the header artwork that adorns this blog post). We need artwork for our web site, but - perhaps more importantly - we need art for social media sharing and similar scenarios. Unfortunately, it is cost prohibitive to generate individual works of original art for tens of thousands of articles, and do so in formats matching all the social media channels, no less. Fortunately, that equation changed with AI!
Generating Art for Articles
I recorded a short video that shows the whole sequence of how we generate art using a series of separate AIs. Here it is:
Here is the short version of the steps we take:
We have an autor who submits an article, which - after going through our editing process - ends up in our article management system, which is shown in this video.
This doesn't have anything to do with art generation as such, but it is still interesting that the next step we usually take is have AI generate a short synopsis of the article, which we then use in scenarios such as article searches, when we want to show a list of articles with a short description.
The actual art generation process kicks off when the user uses our art-generation interface to enter a prompt used to have AI generate an image. (Example prompt: “A rocket ship leaving earth orbit”)
Most of the time, it is hard to think of a good prompt, since our articles are rather abstract technical articles, with (usually) no obvious visual representation. (What does database performance optimization look like exactly, to pick out a representative example?). This is where the magic kicks in! We have an AI that can read the article, and not just understand it in technically accurate terms, but it can then creatively come up with ideas that go well with that article. This includes not just a general idea, but it suggests art styles, useful metaphors, pop-culture references that might be helpful, and more. The user can tweak that prompt if desired.
The next step is to then ask an image generation AI to use this prompt to generate an original image for it. We use different AIs for that. The video shows the use of OpenAI's Dall-E 3 for this, but we often switch the model, use local models, let models compete with each other, and so forth.
We have the ability to not just take a detailed look at the generated image, but also see what internal suggestions the image generation AI made (see below), which can be useful to further fine-tune the prompt and generate new versions of the iamge
When we find an image we like, we use yet another AI that uses an internal tool called “Olympus Image Composer” to create different format versions of the image and add other appropriate elements, like the article title, the name of the author(s), logos, and more. The user can change the details of the compositions, if desired. When the user chooses to save the compositions