AI Animation with Runway: Creating Powerful Animated Imagery (featuring ChatGPT & Midjourney)

1. Intro: Animated imagery has been one of the fastest growing feuds in the AI world. Now in this course, we're going to learn about how to create your own animated images out of AI, and putting the, all those together to create a short film or a video. Hi, I'm Arnold. I've been in the creative industry for over the last decade, working with advertising, marketing. And over the last year, I've been implementing AI into every single part of my creative workflow. In this class, I'm going to run you through how to create a story on Chat GBT. Then creating the images for those stories on mid journey, and then finally animating them on runway. Now the best part about this class is I'm going to teach you the theory and framework behind all of this. So regardless of your platform of choice, you can use these techniques to create your own animated short. Using whichever tool that you like. This class is perfect for content creators, artists, or someone that wants to experiment with AI. Whether you're a beginner or you've had a lot of experience with AI, and just want to keep up with what's happening. This class is perfect for you. And the final project for this class is we're going to be making an animated short with a story behind it. So if you're ready to begin, let's get started. 2. Project: Okay, let's talk about the project now. This project is going to be exciting. What we're going to do is we're going to come up with a story using AI. And then with that we're going to organize it and come up with a shot list. And then with that shot list, we're going to generate the visual representation, AKA the photos. And then finally we're going to take those photos to our animator of choice. And then we're going to animate them. And then finally, we're going to compile that into a short video. And ideally, what I'd like to see from you is a fully animated story. But if you just come up with the story and the images that go along with it, that'll be cool with me too. All right, let's go on to the next lecture, where we bring your idea to life. 3. Idea to Storyboarding: In this section, we're going to learn about how to make a story. Typically I'd like to start with an idea. Then after that, we're going to use AI to augment our idea. For example, we're going to start with the idea of ancient Chinese cats in armor. Basically, we're going to have a story revolve around that. Now I'm going to show you all the steps to generate this story and organize it into a shot list. And then finally put everything together so that it's ready to go for you to generate images that follow along with this story and then animate it afterwards. Now for this section, we're going to use chat GBT. But if you have a language model that you're very comfortable with and that you like using, feel free to use that. Instead, we're going to start this off by going on chat GBT and making a story for us. Now if you have an idea already, it's still good to run it through chat GBT. That way you can have a bit more expansion into your story and a bit of our structure. So what I'm going to do is I'm going to come up with a story. And my story is actually going to involve cats in ancient China battling over a litter box. So I'm going to type in generate a story line. In ancient China where there is a conflict and resolution, where the cats are warring over a litter box. It's going to be a fun little story and we're going to see how this turns out. Of course, this is going to be just a first idea that it gives us. We're going to edit this, and along the way change it up a little bit, and it came up with this story about two warring factions of cats that were fighting over this gift box or this box that they found. And in the end, the story ends with both of them coming together and finding peace. Now, if you want to read this story, you can pause here and read it. And after you read it, we're going to go through a couple elements of this that I want to highlight. So the first one is, there are going to be characters in this. Now we notice here that there's Siamese shadows and Persian protectors. Now these are going to be the two factions of cats, like the two different characters in this story. And what we're going to use with that is that we're going to incorporate these elements to show that they are different groups. So the first one would be Siamese. So Siamese cats, and the second one would be Persian cats. So now we have two different types of cats. And then as we generate images for this story, we're going to include these two different types of cats. Okay. Now to get an idea of what types of shots that we should do, let's ask Chad GBT to also break this down. And then I asked it to generate a shot list of ten frames that could tell this story. Now what this is going to do is give us ten frames of how we can tell this whole story into a little short film. And here it's generating a list of frames for us. Now, we don't have to use these exactly, We can just use this as inspiration. So let's go through and look at what they gave us. The first one is a wide shot of ancient village. And it shows the tranquil village in ancient China with traditional architecture and cats roaming freely. Now of course, we want to anthromorphize the cats, so we're going to have these cats, being the villagers next close up of the ancient litter box, which highlights its unique and revered appearance. Possibly with a mystical glow to emphasize its importance. Okay, so here you know, think of some ideas that could make it have a unique and revered appearance. So maybe it's in jade, maybe it is shining, Maybe it is in a golden palace or something. Basically, we want to expand upon these ideas, because Chachikes good, it helps us come up with ideas. But in order to take it to the next level, we have to add in the human touch to it to add in our creativity, to add in what makes this shine. So the next one, it's introduction of the Siamese shadows, which captures the sleek and cutting Siamese cats in a stealthy stance. Possibly in the shadows, or preparing for a stealthy mood. Now for this, we can have an introduction to these Siamese cats. Whether it be in their palace, maybe it be in where they're training, or they're all wearing armor or something. This will depend on what we deem would be fitting for these Siamese cats. And the same thing with the Persian protectors. These are also going to be Persian cats instead of Siamese cats. And then your job here with these two factions is to think about what could possibly represent these two factions as we are generating images for them. And then next is the action sequence of the clash. So this is a dynamic shot of the Siamese and Persian cats in midair leaping and dodging around each other, showcasing the conflict, because now they're fighting over the mystical litter box. And then this is where the Elder's intervention comes in. So then these old wide cats who are serious and concerned discuss a solution to the conflict. So we can have them here gathered in a circle talking and discussing what could happen that could help solve this conflict. So ideas that come to mind for me, maybe they're sitting around a campfire. Maybe they're in a circle table, like King Arthur and the Knights of the Round table, and they're all discussing what things to do. And then finally, the next shot, which is the contest challenge, because now they're in a contest, they're trying to determine who is going to win to get this magical litter box. And so there's three here. And the first one is a rooftop challenge, which captures the Agilian speed of cats as a race across the rooftops of the village. The next one is hunting tests, where this shows cats hunting and showing they're like, stealthy and skillful. And then finally there's the puzzle solving one where it's a focus shot of the cats engaging in a complex puzzle. So maybe they're solving a puzzle, like a crazy puzzle of some sort. Which could be maybe a rubic cube, which I think for a cat would be pretty challenging because they don't have hands like us. And so these are some challenges that these cats are going to do. The battle each other and we're going to determine which is the best way to represent this. And then finally, the ending is where it ends with a peaceful shot of both clans sharing the litter box. Because ultimately that was the goal of all of this. And so it says, end with a peaceful shot of both clans with the village in harmony in the background. Perhaps the cats are playing together and elders looking on approvingly. So some ideas that come to mind here. Maybe these cats are holding hands and they're like laughing, running around in this peaceful place. And the grass is green and the sunny outside. So basically you have this list of shots that you can have for your short film. Now we're going to build on this some more. This was a nice draft of how our story is going to be and now we need to really cement these ideas and generate out the images for it. All right, from this lecture you should have learned how to expand a story idea. You also should have learned how Chat GPT is used to organize ideas. And then finally, it should give you a taste of what you can do and show you the possibilities of what a language model does. Up next we're going to talk about generating the images. 4. Generating Scenes: In this lesson, we're going to learn how to generate these images. Now, there's a formula that I typically like to use and essentially that goes like this. It's the scene followed by the subject, be it ancient cats like we were doing ancient Chinese cats. And then finally, the style of this whole generation, whether that's anime or whether that's a cinema camera, or whether that's black and white. This will be followed up right after that. Now you could get as advanced as you want and complex as you want, but for the sake of this example, we're only going to use one element. It's going to be a long lecture as we go through every single scene. But you'll see all the steps that I'd take and all the initial changes and thought processes as I go along. So now we have our shot list that we can build off of. Let's generate some images. And as you can see here, I already have a bit of fun that I was just trying out cat warriors in Japan. But let's go and continue on with this. And we're going to start with this first shot, which is a wide shot of an ancient village. So here it recommends showing a tranquil village in ancient China with traditional architecture and cats roaming freely. Now I'm going to do the wide shot and I want to do the tranquil village as well. Right here we are on a mid journey. So I wanted to imagine a wide shot of tran, ancient Chinese village with cat villagers roaming freely and having a fun time. There are mountains in the back and the grass is green and Okay, so I'm also going to do a 16 by nine. So what I have here is a wide shot of tranquil, ancient Chinese village with cat villagers roaming freely and having a fun time. There are mountains in the back and the grass is green and beautiful. So let's test this out and see what we get. Now, before we go on to generate more, I'm also going to explain that we should add parameters. And not the kind of parameters that give it the size and whatnot, but the parameters or guidelines that will keep the style of these images uniform. So we're probably going to want a certain cinematic style or a film style, so that way when we're generating these images, it doesn't just become all sorts of different styles. Because as we're generating right now, it doesn't really know what style we want. So if you look at these images, it may be the same style, it may not be the same style. And even right here, you could see the bottom right one and the bottom left one is kind of different styles, like the bottom bright one is a bit more cartoonish and the one on the bottom left is a bit more of a three D looking thing. So this is cool, this is very beautiful and relaxing and tranquil. But we want to have more emphasis on these cats. So let's go back and here I'm going to do. Imagine wide shot of tranquil, ancient Chinese village with anthromorphized cats surrounded by mountains and filmed on a R Alexa cinematic camera aspect ratio 16 by nine. So right here I added the Arri Alexa cinematic camera. So it's going to give it this certain look that has more of a cinematic feel to it. Because an R Alexa is a cinematic camera, like a cinema camera for big budget productions. So I'm going to copy this so that it is something that we can post into our next prompt. So I'm going to run this and see what we get. So this is a wide shot of a tranquil ancient Chinese village with anthomorphized cats surrounded by mountains and filmed on an R Alexa cinematic camera. Basically this first shot, we just want to show this scene and, and show the setting of what's happening in this village, this fun cat village. Okay. So as you notice it's not really showing the anthromorphized cats, which is making cats look kind of human. So we're going to put that in front. I think the wide shot of tranquil ancient Chinese village thing is throwing it off and it's building off that scene. So we're going to have an anthromorphized cats in ancient Chinese village surrounded by mountains. And Ari Alexa, cinematic camera. Now this is going to start off by generating the anthromorphized cats. And then it's going to add the background of the ancient Chinese village surrounded by mountains and whatnot afterwards. So let's do that now. Again, it's not really showing the cat the way that I wanted, it's just showing random cats in a Chinese village. So I'm going to try again and do cat villagers in human clothes, in an ancient Chinese cat village. And finally we have what we're looking for, cat villagers. So this one on the bottom left and the top right has cats as humans, essentially. So we want something like this, something in this theme. And I do want this to be more of a wide shot, so potentially we can expand this and see what we get. So I upscaled it and now I'm zooming out two x. All right, and from what we see here, I like what we have. I'll choose one of this. Let's go with the bottom right one. And then we'll upscale this and save it. Now of course, I am a little curious, so I will zoom out one more time just to see what else it can add onto this scene. But with what we just saved, I already like that, so we're going to use that for our short film, Al, right? And this is pretty funny. It's a cool looking one. It's these cats in a village. Some of them look kind of gangster. The only usable one I see is the top right one. And I do kind of like it, but I think we'll stick with the wound that we have so far. It seems like that's a bit more powerful in my opinion. So we'll move on to the next shot. Now, the next shot is the close up of the ancient litter box. This is what they're all fighting over. So I'm going to copy the close up part and then we're going to generate it. Now the rest doesn't really stick out to me, so I'm just going to generate it on my own. Now I'm going to do imagine close up of ancient litter box with emerald detailing and golden elements. Now we're going to add filmed on airy Alexa cinematic camera and the aspect ratio of 16 by nine and this should be a very magical, powerful type of ancient litter box. At is literally driving them to go to war, like the different factions of cats to go to war with each other for. Let's see what we get. While we're generating that, let's get a bit more creative and go close up of ancient litter box with majestic beauty, beyond description that all the cats love instantly and adorned with emerald and gold and filmed on the Realexa cinematic camera. Now I thought the last one with a close up was a bit too close. So I'm just going to go Ancient litter box, maybe this way It's a bit zoomed out. So let's see what we get. And I like this, but there shouldn't be a cat in here, especially a normal looking cat, since we're doing cats that look like humans in this. So let's try this again. Now, I do like what we just did, so I'm going to copy that. But I'm going to change some things up. I'm gonna imagine it. I'll paste the same thing and I will add ancient Chinese. They'll have to stay with the theme. And then I'll take out the part that says that all the cats love. And let's see what we get. Okay, so we have these ancient Chinese litter boxes, and actually I think that's really cool. The top right, bottom right, and bottom left looks really nice. I guess this one has a steaming something steaming in there. Even the top left one looks really nice too. So which one should we go with? I think the right ones on the right side have a very ancient Chinese theme to it. So let's go with that. And I'll go with the bottom right one. It looks a bit more magical and I will save this picture, Al. Right now, the next part is introduction of the Siamese shadows, which is the Siamese cats. So let's go and imagine Siamese cats in ancient Chinese armor standing in Chinese building. Filmed. Filmed on Aria Lexa cinematic camera. And generally I'd like to have a couple renditions running at the same time. So I'll copy this and then I'll add a bit more to it. I did make a minor change just to see what it might come up with, and I ran two other ones. So now here we have the first rendition come out. This looks really cool. Kind of makes them look like the bad guy, though. And this one just finished, but as we see here, they're not in the human looking thing, so we won't be using that. But I like this top left one, This looks really cool. And this one on the top right also looks really cool. And finally, we just have this one finish, which looks really cool, but it doesn't highlight the Siamese cats. But the bottom right one, I guess does. And I do like that. So let's go with the bottom right one. I'm going to upscale that and we're going to save it. Here are Siamese cats. And I'll save this image. Okay, next we're going to do the Persian cats. So we're introducing the Persian protectors. And the Persian cats will be, imagine Persian cats also a similar scene, where Persian cats in ancient Chinese armor standing in Chinese building, filmed on the Aria Lexa cinema camera. Now let's give the Persian cats a different color. So maybe a white Chinese building. And as we generate this, I also like to have it generate a few times, so we get a couple renditions. So a good way to do that is to have repeat. And then we'll have it generate two others. So as it's generating the first, we're doing two other ones as well. Okay, so here are some renditions that we got. Now this one looks really cool. They look really majestic here. And I do like this first one a lot. This also looks really cool. They all look really cool. But the first one looks. Very Persian cat, like the bottom left one also does look very shows off that they're Persian cats. And let's look at some of these other renditions now. These also have that look as well. I like this one on the top right, especially, and over here. So a lot of these turned out pretty good. Now in my opinion, I think the top right one here really stands out as Persian cats, so we'll probably use that one, so let's go with that now. It does kind of seem like they're not so much in focus, but let's upscale and see what we get. And after upscaling it, it looks good. So let's go and keep this. And I'll save it, and we have our Persian cats. Now the next scene that we have is the action scene, where both of these factions are fighting or dueling and there's a conflict. So I'm going to imagine dynamic shot of argument between Persian cat in ancient Chinese armor and Siamese cats in ancient Chinese armor. Filmed on the Ari Alexa cinema camera. Now I will repeat this and I'm actually going to change argument to conflict that way. Mid journey gets a bit more creative with the ideas that it has, and it doesn't seem like most of them are fighting. But let's, let's see what we have. Okay, So these shots are fighting and this is here fighting as well. This looks like they're agreeing on a truce. Okay. So not much conflict in these photos. Let's create some more conflict in here. And I'll make it a fighting Persian cats in ancient Chinese armor against Siamese cats in ancient Chinese armor. So let's see what we get. And I'll repeat this maybe two times, and we forgot to make it 16 by nine. So let's change that. So I like that, this looks like they're fighting and the ones that we've generated looks like they're fighting. So it would be an easy change. All we have to do is just use the same prompt. And as you can see, it didn't use that prompt, so it turned out different. But here I am using the prompt again and then we'll run that. And theseenes are cool, but it doesn't show the two sides fighting. So we're going to run it again. Now, I'm going to repeat it this way we can have many different renditions to look at. I'm going to repeat it three times. Okay. So we have some really cool renditions here, or renders here. So I do like this one right here, but it does look more like a main coon than a Persian. So maybe we can't use that. This one does look cool. So let's take the ones that we like now. I like this one on the top left, so I'm going to upscale that. This one on the bottom left is also cool. But these cats aren't the types of cats that are in our story. So we can't use it. But we can try to, we can try to vary region it and change the types of cats. So let's see if that works. Now, I'm going to save this one first. And also the top left one I like as well. Okay, so here we can go and vary region. Now for this part, we'll try to change the faces of these cats so that it fits the criteria we're looking for. And in our case, Siamese cat. Both of these should be Siamese cats. So let's hope we can get that. And what this does is it selects the area that gets changed. So I'm going to highlight this whole face here, and let's see what it comes up with. Sometimes it works really well, other times it doesn't work too well. So it's kind of a hit or miss. So it does change the cat's faces, but these aren't quite siamese cats. So we could run in a few times and see if that makes a difference. We still are lacking a fighting scene though, so let's try and get that as well. Now, since I feel like it doesn't separate out the breeds of cat out too well, we're just going to generate fighting cats in ancient Chinese armor, and then afterwards try to fix their faces to be the breeds that we want. Okay, so this one's cool. This one looks like it's battling. So I'm going to save this top right one. I'll upscale it. This looks more like a tiger. This could be any type of cat. Okay, so I'll do this as the fighting Persian cat. This one also looks kind of like a fighting Persian cat. Just so we could fill up more scenes of this fight. Now I'm going to imagine and repeat a couple of these. Now I'm going to do fighting Siamese cats in ancient Chinese armor. I'll repeat it two times. That way we get eight renders. And then I'm going to do the same with the Persian cats. Now, we did already get one that I thought would be fitting for a Persian cat scene. But let's go ahead and just generate some more so we have more options to look through. Okay, so here are renders of what we got. So let's start from this top one, fighting Siamese cats. None of these look Siamese, in my opinion. Maybe this one, but that's kind of a stretch. So let's see what the next render has for us. These also don't really look siamese. So let's go and do these again. So I'm going to just do Siamese cats in ancient Chinese armor fighting, filmed on the camera. And then I'll repeat this two times. Okay. So let's go back and look at the Persian cats. I just realized we looked at Persian cats, but this is the Siamese one. And it still doesn't look too Siamese cat like to me. So I don't like any of these. But let's go and see the Persian cats. And these all look kind of like tigers and lions and leopards. So those don't work either. But this kind of looks promising maybe on the bottom left, but I'm still not too happy with these renders, so we have to run it again. And sometimes this is just how it is when we're working with AI images, is that the images don't come out exactly like we want. So I'm going to do the same thing. I'm going to imagine Persian cats in ancient Chinese armor fighting, filmed on the Ari Alexa camera. And then I'll repeat this two times. That way we get eight renders. And here is our Siamese cat. There you go. And another Siamese cat here. It looks like he's battling somebody. So if we can make this a Persian cat, that would be perfect. But at the moment, that doesn't seem like we can. So I will take this one on the bottom right, since that is obvious that it's a Siamese. And then for the Persian cat section, oh, this looks pretty scary, huh? This Persian cat, I do like this one. But let's see, what else have we got? Oh, these look really cool. I like these a lot. I like these top left and top right, and the bottom right also looks really cool. Okay, so I'll upscale the top right one, but there's a bit of a hiccup here, is that there's some humans in the back. So what I'm going to do is I'm going to try and vary region for this and let's see if we can change this into cat warriors. And if we can't, I'm just going to change it into something else. Okay, so it's going to be Persian cat soldiers and ancient Chinese armor. Now looking at the Siamese cat scene, it doesn't look so realistic. So let's go back and change it a little bit. Let's see what else we can get if we render it a few more times. Okay, so now that we did that, it kind of looks mysterious in the back and it's not 100% human, which is good. We can just use this actually. And since it's kind of ambiguous, they could actually be cat warriors. So I will upscale the bottom left one. I like that the most, it's the most mysterious. And then we'll save this one for the Persian cat battle scene. And I'm going to render out a few more Siamese cat battle scenes. Okay, so what do we have here? These two Siamese cats here. But they're fighting each other, so that doesn't work. We have some Siamese cats here with swords. Yep, I think I could use this one. It is a bit more aggressive than the other one, but it would be nice if this bottom left one was a Siamese cat. So let's see, we can change it. If we could, we'll use that one. And if not, we'll use one that is more obviously a Siamese. So I just did a very region and we're going to wait and see what we come up with. And from here, it looks promising. It looks like we're getting a Siamese cat. So the render just finished, and this bottom right one looks like a Siamese cat. So let's go with that one. I'm going to upscale that and we're going to save it. Okay, so now we have the waring of the Siamese and the Persian cats. And to just add more to the depth of the scene, we can save some of these images that might look cool. So this bottom left one, I think would add to the story. So I'm going to upscale that. I'm going to look through these other ones as well. And I actually like this bottom left one, so I upscale that as well. That could fit somewhere in the fighting scene. For the most part, we could just add fighting scenes that would add to the story. So I upscaled a couple other ones since our main two highlights, the Siamese and the Persian cats are already done, we can just have these ones to add to the story. So I'll save this one, and then I'll save this one. Then I'll save this one, and then this one. Okay, so we got this fight scene down now. The next shot is an elder's intervention, gathering of wise old cats with a series and concerned demeanor. Okay, so I'm going to copy that and then I'm going to paste it here. I already like the concept of this. I will add that old cats in ancient Chinese attire. And then actually I'll even add discussing the solution to the conflict. And then again, we're going to put this filmed on this Ari Electra cinema camera to stick with a similar theme. And then I'll repeat this two times so we can get eight renders as well. Okay. So this is what we got now. I think it would be cool to have like a council of cats. This is cool, but there are people in here and we want a full on cat society. So let's change this up a little bit. A gathering of wise old cat cat council and ancient Chinese attire in a government building. With the same demeanor. And then we'll repeat this a few times now. I like this concept here, but the person in the middle is not a cat. So we're going to try and upscale that and change that person into a cat. But I like the concept of having cats in a government setting. So I'm going to make this man turn into a wise old cat in ancient Chinese official clothes. Let's see what we get. Okay, so it looks like now it is a cat meeting. All right. I like this so I'm going to upscale the second image. I think I like that al one the most. Now I do think there's a black line on top that might end up bothering us in the future. So I will generate a few other ones. I want to make this cats as ancient Chinese government officials in a government meeting filmed on the Ari. Alexa, we'll render that and repeat it again. It looks like these cats are here for business. I like this one on the bottom right in particular. But they are humans here and we don't want humans. Let's go with this one on the bottom right. Now I want to make them a bit more official. So I'm going to go in very region and give it a crown or these official government head thing. So I'm going to put ancient Chinese government hat. Now some of them could be wide. So I'm just going to give it a bit of leeway here and let's see what we get. And not much of a difference, sometimes this happens. So let's just keep what we originally had and I'll save that. We're going to do the contest challenge part, which is the rooftop race, the hunting test, and the puzzle solving part. So essentially we're just going to show a couple cats doing these activities. And to start off with, I'm going to do the rooftop race. Now I'm going to imagine cats in ancient Chinese armor running on the rooftops. And I'm going to repeat this a few times so we can get a couple renditions. I'm also going to do the other challenges as well, so we can save on time. Okay, So I ran all three, running on the rooftops, hunting and then solving a puzzle. Okay. So it seems like the running on the rooftop one looks good. Well, we could essentially use the ones of them running and we have these here. I do like that first one, the top left one. I like this bottom right one as well. This one's cute, but it looks like a baby cat. So let's go with this bottom right one. I like that one. I'll upscale that, and I'll also upscale this one. The top left one of them running. Now the hunting ones kind of just look like they're standing, so we'll do something else for that. And then the puzzle solving one, well they're just standing here as well. Let's see what else we got. And it looks like they are just standing. So let's have them play Chinese checkers or something, or chess. For the hunting one, I'm going to have cats in ancient Chinese armor. Or archer cats. So they have a bowl and that looks like they're out and ready to hunt archer cats in ancient Chinese armor. Filmed on Arri Alexa cinema camera. So I'll do that and I'll repeat it twice. Then I'll also have the puzzle solving one where they're playing chess instead. Well, I'll have Chinese checkers and then I'll repeat that two times successful. We have the cats hunting here. The images of it with the bow. I like both of these. The bottom two are amazing. I'm going to upscale the bottom left one and then for the puzzle sobbing one. These all look good actually. So any of these will do. I'll probably upscale. No, here's the other renders as well. And they're also really good too. So I'll probably just upscale this top right one or this bottom left one because it looks more official. There's many choices for this because they all look good. Okay, let's go with this top left one right here. I like that a lot. Okay, so for our final scene, we're ending it with harmony and unity. So end with a peaceful shot of both clans sharing the litter box with the village in harmony in the background, perhaps with cats playing together and elders looking on approvingly. Okay, so let's see how we can do this peaceful shot of both clans. Now, it's probably hard to include the litter box. So I'm going to take this out. But maybe I could do sharing gold litter box and I'll have peaceful shot of cats wearing ancient Chinese apparel. Sharing gold litter box with the village in harmony in the background, perhaps cats playing together and elders looking on approvingly. Okay, now I think this is going to be an interesting shot because it can go many different directions. And it's really up to mid journey to come up with what it thinks this is going to look like. So we're going to let mid journey do its thing. I'm going to repeat this two times. And let's have it play out. All right, based on this first render, I'm thinking it's not coming out the way I imagined. So let's try again, and this time give it ancient Chinese armor, since these are warrior cats of two warring factions. And we'll start with cats wearing ancient Chinese armor. This way, that's emphasized. And I will run it twice that way we can get a couple more renders so we have more to pick through. Okay, it does seem like this is the direction that we're headed, but it's not exactly what I imagine. I'm going to render a few more times. This way we have a bit more options. I'm doing the same prompt, and I rendered it three times with a repeat, three. All right, so here is what we have. Okay? This looks like a town of cats in prosperity. Here's some more. And they are sharing the golden litter box here. That's really sweet. And this is pretty funny here too, Look at this cat's face. But here are some more and I think I do like some of these. My favorite one maybe, is this one on the top right, where it's like these cats are in a peaceful village. But actually I like this one a lot. The one on the bottom left, that's pretty funny. This one on the top left also looks pretty cool too. So we can choose either one. The top left one a bit more prosperous, right? There's all these leaves falling down. So maybe let's use that. And since this is a bit focused on the subject, I'm actually going to zoom out. I think it's going to add more to the story. All right, let's see what we got here is the expanded scene and all right, it really does show cats enjoying this litter box or a couple litter boxes, and it actually looks like a happy ending. So let's go with this. Let's go with number one, which is the top left one. And now we have all our scenes for this short that we're putting together. So as you learn in this chapter, there are many different renditions that you sometimes have to do as you are coming up with the images that you're generating. And since we had a basic structure for us to build our story on top of, it was a lot simpler for us to just use your imagination and build on top of this skeleton, the structure for us. And lastly, there are certain types of edits that you can do, especially in the prompt or if you don't like the image that you had rendered. But you like a big part of it, you can use the very Region part. Now up next, we're going to start to piece this together into something animated. 5. Animating Lesson (Runway): In this section we're going to learn about animating the images that we have. So we're going to bring them to life. And here there are more than one platforms that are available. So it's really up to you to choose whichever one that you have the most affinity with. And in our example here, I'm going to choose runway now. The ultimate goal is for you to make an image dynamic. So whatever means that is to get you there, that's the one that would work best. So I'm going to go through how we use runway and the specific spots that I brush and use so that it can bring this photo into life. All right, let's talk about the animation part now. My favorite editor so far is runway. And let's go on the runway site just so you can see what it is. And runway is this AI animating technology that lets you animate starting from an image. It also has other capabilities as well. But for this class, we're going to focus on its image to animation capabilities. So here we just logged in, and then from here I'm going to go start with image. Now there are a lot of different things that you can do on this program, but for today, let's just focus on this now. Start with image. We can insert what we made earlier. So with all of these cat images that we made, I'm going to put them in there and animate it. And you're going to see the steps on which parts that are chosen and how different things are animated so that they look out or turn out cohesively and look nice as we put it together. Now the first one is this shot introducing the village. So I want to bring that in and once it's uploaded, you'll see that there's a few options here. Now we can press Image and Description, and this is the easiest way to do it. We could just add what we'd like to see in the video or the animated image. Or we could do some of the other more Vance controls down here. Now personally I'd like a shorter duration. And I'll have the remove watermark since I am a paying member. Now let's start with a simple description. I'll have Lively Village Scene, and then I'm going to generate that. Once it's done generating, you can press the play button to get a preview. Now, this one is a bit weird, as you start to see these cats turn to humans and also disappear. Personally, I like using the motion brush, which lets me have my own creative control. So let's go and edit with motion brush. This lets you control a specific area. Now, they do have their own creative touch on it, but you get to, for the most part, control what is happening. For this one, I would just control some of the main images or the main characters. And so that way you can have something that is moving. For the person that is looking at this scene, that way it looks more dynamic. All right, so for this I'll just have them move like a little bit up or down. And I'll have them move a bit forward. Maybe I'm putting a bit on the vertical y axis and the proximity z axis. And that's with brush one. Now if I use brush two, this is controlling different elements in this scene. And that could be maybe something in the sky, maybe the trees are moving. And this is when you want to have things move, but in a different direction or a different style than what the other brush was doing. So this, I highlighted some of the trees in the back. And then I'll have it move maybe on the x axis. And then this seems enough, so I'll press Save, and then we can generate that. And it just finished generating. So we'll preview by pressing Play. And here it does look pre believable, just a bit of animation. You can see the trees move in the back. We'll probably keep like 2 seconds of this clip, which is fine. So I'll download this and then I'll move on to the next clip. Now for the next one, we have this, which is that sacred litter box that everyone is chasing after. So I'm want to upload it to Runway. And once uploaded, we see the same screen again. And then I'm going to go on the motion brush. Now of course, we could just do camera motion, which is super easy this way we can maybe just pain and when we paint, it's essentially just giving it a bit of motion. And that's all you really need sometimes because that creates the dynamic movement in a photograph. So I gave it some horizontal and a bit of pain. Then I'll save this. And then for the motion brush part, I'll highlight the outside, like the windows. So that the light can have a bit of ambient movement. And just a bit of ambient movement just gives it the lively feeling that you want. And I'll save that and generate okay with our generation done. Let's press the play button so we have a preview. And we have this beautiful little panting movement. That's just like when you're on a camera and you're shooting the shot and you're painting your camera to the right. Perfect. I'm going to export this one and we'll use this for sure. Now we'll move on to this third shot, introducing the Siamese cats. And I'm going to do something similar. I'm going to have the panting movement as well as the motion brush. So the motion brush, I like to put on their eyes that way, it gives them a bit of life and perhaps they're looking away or looking at a different direction. Or you could just have ambient noise so that it moves a little bit, or it just has that twinkle. And I will save that and then we'll run it. Okay, let's preview what we have, and this is perfect. I like how this looks. They're just slightly looking to the left. And there's enough of movement to give this scene life. So I'm going to save this one and download it as well. Now, moving on to the next, we have the Persian cat. And with the Persian cats, I'm going to also do a motion brush. And then with these motion brushes, I'm going to have them also do something very similar where they're just looking towards the other side. And now I want to have them look towards the right side. Whereas the other group, I had them look towards the left side. And I'll save that, and then I'll also do a camera movement. I have to zoom, Zoom in a little bit. And also a bit of a pan with some horizontal elements to it. And then I'll save this and then we'll generate. Now I wanted to do a bit of a pan so that it gave it that camera movement as it highlighted the characters from the left to the right. And also the twinkle in their eye or at least a bit of movement in their eye so that it gives these characters live. All right, so let's preview what we have. And now we have this pan, as well as this one cat on the right, just looking to the right side. Which I think is actually a very good element to how this image is processed. Now I love this, I'm going to download it. Moving on, we have these fight scenes that I want to highlight. So we're going to animate these. And for the sake of time, I'm not going to go through every single one. I'm going to animate it. And then we will move on to the one where it's a group of elders trying to come up with a solution. So right now I will go and animate these fight scenes. Essentially, we just want some clash of action. So I'll show how this is done, and then the rest will just follow a similar methodology. So for this I want to have the motion brush where I can control some of this. And I want to have these paws moving. We want to capture the elements that are going to be very dynamic. So the paws are moving maybe, right, because he's punching. And then the other paw on the other guy will use a different brush. Since all of these are going to be moving in different directions, it's going to be kind of a bit of pandemonium really. So I'll use brush to, I'll highlight the other cat's paw. And then I'll make it move to the left, maybe even up a little bit. And I like to keep my movements very subtle. I'm not trying to do something very exaggerated. And then with brush three, I'm going to have this cat in the back have a bit of ambient noise so that it seems like this cat is alive. And looking at this, I also think I should go back and use brush one and give a bit of movement to this cat's face. And then I'll also use brush two to give movement to this cat's face. And with that, I will press Save. And then we're going to animate it. We want this to be very dynamic. Okay, Generate. Okay, so this is finished. We're going to press play. Just so you can see what's happening here. These cats are moving. They're fighting. And personally, I would like to see a bit more movement on the arm here. So I'm going to go and brush again. This time I'm going to use brush four and then we're going to have this movement of this arm, we're going to have it move a bit more to the right and we'll see what we can do. So as that's making and we're doing that edit, it happens underneath here, and then we can watch the progress from here. Which at the moment just went 11-21% and it is a bit slow. So patience is important in these steps. It's just part of the nature of AI currently. And here is our edit. Let's see what we have. So it does look like he's throwing a punch to the white cat. But at the same time, the paw here, the right paw is lost. So we could go back in here and try to see what's going on. Now it looks like blue is highlighting this part of the paw. So maybe if we take that out, then it will do its trick. Now perhaps we could even use a different brush, like maybe brush five and highlight this paw and move it down. Okay? So with brush five, we'll highlight this, and then we'll move it down. And let's go and run it. Let's see what we get. Okay, so this is finished. Let's play it. And as we see here, this one is the most believable one. It looks like it's doing a punch in slow motion. I like this one, so let's say, and as you can see with these action scenes, it could be wild. And All over the place. And we have to use all of our brushes to control the scene individually. And there we go. We'll save this one. Now, the next scene is this with the elders. And this is a subtle scene. With a subtle scene, I would probably just have very subtle movements, nothing crazy like we just did. And so I would do camera motion and then a horizontal pan from left to right and maybe a bit of a zoom in. Now, zooming in is always nice because it's one of those techniques that in modern day videos are very appreciated. It just highlights the subject and it gives a bit of dynamic movement to the scene, so it's more engaging. Now, we could also go on motion brush and give these cats a bit of emotion by moving their eyes or having a bit of ambient noise in the background. With these, I want to have their eyes highlighted and have a bit of ambient noise there so that something happens there. And then maybe for the lanterns in the back, we could have it swinging or at least just moving a little bit. And they'll also have that as ambient noise. And let's have this run. So I'll save it and then I'll generate. Here is our generation. Let's press play and see what we get. Okay, this is actually pretty good. I like that it is panting and it's giving us a scene of what's happening here. And it really shows how big this room is, because it's panting from the left to the right. All right, now I'm gonna Dolla this and move on to the next scene. Now the next few scenes are the competition between the two factions of cats. They're going to be much like the dynamic action shots that we were looking at earlier where they're fighting. So with this, I'm also just going to generate one for the sake of time and for you to understand the thought process and concept of what is happening. Okay, and I'll demonstrate on this image where it's a cat running and it looks like he is running in full speed. Things to note are how his leg is a bit blurry from the motion blur. Arm is moving, intense facial features and this arm is also moving. These arms are moving in different directions. Legs probably moving to everyone else is also moving as well. So those things are things that we're noticing and then we're probably going to have that be the things that we have highlighted as things that will be moved. I'll go onto the motion brush. And then with motion brush, I want to highlight this arm as the first one. And make it move to the left because I'll be swinging hard. And then I'll use brush two. And then I'll highlight this other arm as the arm that's swing to the right. And then brush three. I'll use it on this leg that's running. He's been running this whole time. And we'll have it in the proximity part as well as in the vertical axis. Now I like where this is going, so I think we could just run it also, I would like to add this green on the brush three, so that he moves the other leg as well. Okay, so let's save this and then run it. Okay, so it's not generating. Let's give it a preview. Now sometimes the preview does get weird, and as we see here, there's a strange face of a cat popping out. And that's no good, so we have to change that. And that's brush one. Now with brush one, I could even just maybe delete it. And so let's go without brush one, And then just the other brushes. Now we could also add a bit of ambient type of dust getting kicked up. That way it brings this scene to life and then go from there. So I'm going to add some ambient noise which is highlighting this dust. And then I'll press Save. And then from there we'll generate again. All right, here is the second video. Let's play it and see what we get. Okay, it's much more slow motion and subtle, but this might just be enough. Now, I could possibly add a camera motion in here, which will be cool. I'll do a zoom and then I also do a horizontal paint. Now since it's so subtle, this will be a effect that would be very desirable. So I'll go with that. I'll save it and then I'll generate it. And then as it's generating, we can watch the progress again. This one seems to take longer than the others. And here is our latest rendition. Okay bit of movement, which is pretty perfect because this looks a lot more dynamic than the last render that we did. And I like this, so I'm going to save this and we will use this for sure. Now we're going with our last photo of the cats living in prosperity. And I'm going to drag it in here. And then we're going to make our changes as usual. Now for this one, I want to highlight some of the fun elements that are happening, like the leaves falling. So with the leaves falling, we could highlight them and give it some motion. And if we give it some motion, it will be so much more fun. So I want to highlight those and then I'm going to make it fall down. So why access? Bring it down a little bit. Meanwhile, I'll go on brush two. I'll give these cats eyes a bit of a twinkle so that we can see that they are alive and active. I'll do ambient noise for that. And then let's save it and run. And so here's our first generation and it looks pretty good to me. Everyone looks happy. They're thriving in this village. And it's beautiful like the autumn or fall when the leaves are coming down. So I'm going to download this and save it. Now from this section, you should have learned a couple key things. How to use runway to animate our photos or pictures. Choosing specific elements of this image that we have so that it moves and it creates a dynamic movement. And you would have learned that it doesn't always come out perfect on the first try, sometimes you have to go back, highlight specific areas, highlight things that didn't turn out right, and then make the changes as necessary. All right, so from this lecture you should have learned that we typically like to do simple natural movements. Since AI is not super complex and complicated and advanced yet, it's best to stick with something minimal. And that to keep it as realistic as possible is the best way to go. And next we also learned how sometimes just a light touch is all that you need. And then finally, that certain areas are the places that you bring to life. So you don't just animate the whole thing, but maybe certain times you would move just the arm or the leg or the part of the jacket that the wind is blowing on. Up Next, we're going to put all of this together. 7. Conclusion: Congratulations on finishing the course. Now. To summarize, you should have learned how to create a story using Chat GBT or a language model of your choice. And from there, how to organize your story. And then using that story, we brought it to an AI image generator. From there, you learned how to think about the thought process behind creating the images using that image generator. And then finally bringing it over to a image to animated AI platform. And from there, we started working with different parts of our imagery to make it animated, to bring it to life. And then finally, you have the final cut. Once again, congratulations on making it this far. I'm excited to see what you post in the project section. And if you can help me out, leave me a review and let me know what you like about the class so I can continue in doing more of that. And I'll see you in the next class.