Transcripts
1. Introduction: Creating images with
Runway is one thing. Having those images
move in a way that is intentional is a whole
different challenge. So welcome to the Runway AI Video Generation
master class. My name is Hosta Kachii and
I'm a visual designer with over five years of
experience working with creative tools
and digital content. And I'm here to show
you how you can create Runway videos with more
control and purpose. We're going to start by learning the foundations of Video
generation inside Runway, how to prompt movement and how video prompting differs from
generating still images. Also going to look at how you
can plan your shots so that you're not wasting credit trying to describe
that perfect scene. Then we're going to create
videos using texts and images, build connected scenes, and use keyframes to control
movement and transition. From there, you are going
to learn how to transform your footage from changing
your background, weather, lighting, and time
of day to refining your videos with color grading, stylization, and upscaling. We will try performance transfer character swap, add
sound effects on audio, and bring multiple
clips together into one final
polished sequence. Finally, you're going to create your own short AI video using everything that
we have learned. By the end, you're going to know the full Runway AI
Video generation workflow from the idea
to the final product. Let's get started. Mmm
2. Welcome & What You’ll Create: H We're going to learn
all there is to know about generating
videos in Runway. We're going to start
from the foundations, what there is to know before
we hit that generate button, how it's different to prompt for video versus for image and then see how we can manipulate the video once
we have it, as well. So in this course, we're
not going to cover anything regarding
image generation. I'm only going to
focus on generating videos and not really go through the foundations
of prompting, you know, the overview
of how Runway works. I'm also going to
spend less time on every single feature
within Runway and jump straight into the
video features only. So it would be helpful if
you knew about references about prompting and how to
edit footage after generating. So videos with Runway, there are a couple
of applications. You could generate
video for some ads. If you have a small business, you can generate videos for
social media, something fun. You can tell a story with
these cinematic settings and you can do all sorts of manipulations regarding
camera direction, lighting, environment,
transitions, pacing the cuts and anything else that you would usually
do with video. So first, let's
jump into prompting for video purposes
because as I said, this is very different
from image prompting.
3. Prompting Motion & Camera: So how do we prompt motion? With images, we can say, create a product photo
of this perfume bottle. But with motion, we
need to describe some additional parameters
such as duration, the action, the setting, lighting, any sort of changes
from frame A to frame B, and we need to just keep in mind everything else that
comes with a good video. So that would include details, focus, and et cetera. In the homepage, if you
scroll down to tool, you will find the
video tab right here. We have image. I'm sure by
now you know how that works. We have video and then audio. Now, audio pairs well
with video because if you have some dialogue or you want to create
a certain mood, you will at some point
need some audio to go. Now when I say audio, it does not only include music. I'm also talking about
sound effects, dialogue, and any other thing we need to tell that story that
we're trying to tell. These two are going
to be the focus of this course and
we're also going to explore the video apps. You can see there's a
lot there and of course, the audio apps as well. What should Runway be keeping as a reference when
generating your footage? For example, if I want a video of this character riding on
a skateboard like this one, need to have a reference of his face because I want to make sure that
throughout the video, a random person doesn't show
up and it is my character. So that image that you upload for to limit the generations
that Runway does, that is going to be
your references. And again, it could
be a character, it could be a product,
it could be a landscape. Color palette or anything else. So that's a reference.
We also have keyframes. So if you have two
images and you want to basically generate
the in between, that's going to be
your keyframe feature. So basically, I want to tell a story of this duck
doing a backflip. My first image could be
the duck standing still, and the last frame could
be when he's done with the backflip and he's
doing, like, a nice pose. And the in between is what
I'll describe over here. Also have the edit option. So for example, if
we have a video and there's a random
ball in the background, we could use the Edit
tab to remove that ball. And again, I'm going
to go through each of these with real examples. And lastly, we have
the extend tab, which as the name says, it's going to extend the duration and also
keep whatever you had in the last frame consistent in that
new extended footage. Now, down here, we
have more references, but this is the same
thing as uploading here. It's just on two different end. Can generate HDR if you want, but for beginners, you
may not need this. In order to access the HDR, you can upgrade to MAX. I'm currently on P, so that's not going to be
something I want to cover. Next is the resolution. So we have different options. The higher you choose here, the more credits that are going to be taken
from your account. So you would, you know, you would need to choose
the appropriate amount when generating
whatever you have here. Now, bear in mind
that we do have an upscaling feature
just like images. So if you generated
something with four ADP and you're not
happy with the quality, you can upscale it to, you know, increase the dimension and also get more details
from your video. So you don't always
need to be on ten ADP unless you want to. Then we have the Aspec ratio. You can choose that. The
duration of our video, it goes up to 30 seconds. And then whether
it includes audio, if you don't want it
to have any audio and you want to generate
that audio later, you can keep this
off and we have some advanced setting
for the output. If you want to download
this yourself, you can go for MP four or MOV. Once you're done, you can
just choose your model next. Now, there's tons of
models that are external. Obviously have runways own
model like Ruby and ALF 2.0. But there's a bunch of
familiar names here as well. So if you just go to the side, you can find in, you can find ByteDance
and so many others. And you can go on each one
based on what they offer. So there's a little
text on the side. This tells you what that
model is going to support. So, for example, I cannot
use this model for generating video because it
only supports edit videos. And when I click on it, you can see it brings me
to this new tab. So unless I want to edit videos, I don't need to go on that one. If I want to do multiple
things at the same time, we can go for
multimodal control, and then there's a
bunch of other things. Now, what we're doing here
where we put text and we expect a video output,
that's called text-to-video. And when we start with an
image and then expect a video, that's going to be
image-to-video. So you can read these
and choose accordingly. All right. Now, back
to the prompting, we saw what these little
buttons are about. We now can describe our
very simple scenario. So my idea is going to be
that duck doing a backflip. So I need to first think
about what's in that video. I know there is a duck, but what is that duck wearing? If they're not wearing any that's something
I can mention. And then what color is the duck? Where is the duck situated? Are they outside
indoors at a studio, or is it just, like, a
plain white background? And then the action, the
duck does the backflip, but what happens after? Do they fall down? Do they
get up, do another backflip? What is the sequence
going to look like? Then if there's any other
parameters like the lighting, an additional element
around, for example, the duck is doing its thing, but then baby duck
walks in the screen. That's something I
can mention here too. But we're going to start with a very simple scenario
which includes one subject, the duck and I'm not going
to have it wear anything, so we don't get a giant prompt, and we're just going to describe what's in
our head right now. A yellow duck standing still, then doing a backflip,
and that's it. So that is a subject, like space for adjectives
as well, standing still. So that's going
to be one action. Then the backflip is
going to be action two. Now, it does need some reference because if
you just say what you want, Runway can really doesn't
have a starting point. So it needs to figure out what the duck looks like and then go about finalizing the details and doing the action
that you want. So that's going to leave a lot of room for hallucinations. So what I'm going to do
first is go to image. You can see why these are placed right next to each other. I'm going to just copy
this and put it here. But I'm going to get rid of the second half because
this is one instant. A yellow duck standing still, and then we can describe the
background in a white broom. Here I could say
studio lighting. Now notice that I did not say anything about
camera movement, transitions and all of that, because this is only a
starting point over reference. So let's just keep it like that. I'll have it for four outputs, and I will use nano
banana for this. Let's think about
the aspect ratio. It doesn't need to match with the video ratio
because this is again, only going to be a reference. So just to be safe,
I'll do a 16 to nine. That's what I'm going to
do for video as well. A landscape shot. And
then we can generate. So here is my duck. As you can see, it's
exactly what I asked for, and I have a couple of options. So that's why it's good
to start with four. If you want to make any
adjustments to your character, you can just drag them in
here and state that change. Like give him a red hat
or make him bigger, turn it into a different animal or anything else that you want. I will be using this one. I think it's cute. So I
easily can switch to video. You can see that it
did that already. Um, and I could just click on drag this one into
my reference frame. Now this prompt from
before was saved, and I could continue
building upon that prompt. Whatever subject you have, make sure you have that
reference right over there. Then we can say, then doing a backflip
camera zooms out slowly. So that movement of the camera, we're just going to
describe it like that. You could say neutral lighting or neutral studio lighting. And that is it. We have
a very simple scenario, and I'm going to go for the Gemini model because nano
Banana was a Google model. I want to keep that
consistent with Gemini and we are going
to generate this. Let me just make sure
everything is good down here. So 16 to nine, same as my video, my image. And for the duration, we may need to
increase this so that there is enough time for the
duck to do the backflip. If you're not sure
about the duration, you can also click on Auto. It could be up to
Runway to decide. Then you can decide
on the audio. For my model, it does
have to stay on, so I just click Generate. Here is the video. Let's turn on the audio. So that was an attempt
of a backflip, but I think it
looks cute anyway. Notice how Gemini just added a random song because that
was the models requirement. But our character stayed very consistent
throughout every frame. And if we look at the prompt, it executed exactly
what I asked for. We're now going to watch
this one more time. I'm going to mute it and
we can take a look at the segments of that prompt
that did that performance. You can see the camera is
coming out very slowly. That is this part right here, so camera zooms out. That's just the camera moving
away from the subject, and then we put in slowly. It's not really fast
and it's very gradual. If I put it in again, pay
attention to the frame, it's going to come out further from the
subject very slowly. And then we have the backflip. Again, an attempt of a backflip. It even looks at the camera
for a second, and that is it. So very simple movement here, and I could continue
building upon this prompt. So now we know the
general idea of what video prompting looks like comparing to what we
did here for images, you can see how it kind
of differs from video. I mean, we still
have that subject and the lighting
and environment. The only difference is
that we're going to be talking a lot more about action, movement, and camera
control as well. So if we did not have a subject, and let's say this
was a landscape, the movement would
be the trees moving. It would be clouds moving or there's a bunch
of birds in the sky. So those are still movements. And as long as you're describing something
that is not static, you are prompting for video. Now in the next lesson, we're going to get more into details about
this prompting. We're going to stick
with our story here. We're going to learn
something very important, which is how to plan
for our sequences because I could continue making videos like this with
the same prompt, and it would just waste
all of my credits. So we're going to see
how we could plan better by building
more references and then learning how to integrate those references individually
into our prompt.
4. Planning Video Without Wasting Credits: So we're going to stick with this very simple
sequence right here. But what I'm going to do is plan more with these still images. So we kind of plant initially with the
look of our subject, but you can obviously
do so much more when it comes to the
still image production. You can see that we have a
limit of five references, and we can even add
video references. These things that
you see right here are your limits to
adding references. So now I'm going
to plan better by adding in my duck the one
that I liked as a reference, and then we're going to put this duck in a
better environment. So duck standing on a stone floor with
a forest behind it. Place duck, front and center. Now I have my subject
and my background. I'm going to keep everything
the same and now generate. So now instead of putting
in the duck alone, making my video and then editing my video to
get that background, I could start here from
these still images because they take up less credit
compared to video, and I could easily visualize what my final product
is going to look like. Here's my little duck. It's the exact same thing. I could, you know,
make my changes, add another reference,
combine them together to get that
final first shot. So I'm going to
use this guy now, put that in, and I will
describe the mid air part. So that's something that
Runway struggled with. It kind of showed the duck flying midair and
then dropping down, but that's not
really a backflip. So what I'm going
to do is remove that and describe the
duck doing a backflip. That's it. Send that in. And this way, I could
tell Runway that this is the kind of backflip that I want and not so much
what it came up with. Now, that looks a lot more
like what I was going for. We have a couple of options. So I think maybe this guy, I like the feathers
that it added. I notice the power
of references here because it's the exact
same camera position, same duck, same background,
and everything. It is just a different action. So now what we could do is
switch back to video now that I have two references
and get a better result. So same prompt and everything. I am just going to drag in
my two images, first one, and then second one because
the duck is going to be stationary at the start
and end of the video. We have our start and end. What we can do now is
maybe fluff up the prompt a little bit so that there is a little bit of control
over the timing. Is he doing a backflip in slow motion? Is
it really quick? Does the duck fail? So we can talk a little bit
about the speed here as well. So standing still
and waiting Oops. A yellow duck standing still
and waiting hesitantly, then doing a perfect
backflip, let's say. The duck then looks around
in amazement, I guess. So camera zooms out slowly. And once the duck is
back on the ground, the camera is still. And then neutral studio
lighting, and that's it, really. Alright, so two references, and we can now send this in
with the exact same settings. Then I'm going to show you
how you can integrate each of these references at certain
points in your prompt. Okay, so let's take a look at our duck doing another
attempt at a backflip. B. Mom. So everything was perfect except
for the actual flip. Again, Runway had a hard time visualizing the back
flip of a duck, which, you know, it's
a good confusion to have because they
don't do backflips. But what we're going to do now is perhaps give
another reference with our other tools to help Runway better
understand this part. So I would say everything
was pretty normal. If we look at the
thumbnail down here, the flip is normal, but then
at some point, I guess, around here, the head and the tail kind of
flip can see right here, and that's something
we call hallucination. So it kind of didn't
know what to do, and it came up with something completely unrelated
to what we asked for. But then after that point, everything looks pretty good. And then the end where we said the duck is looking
around in amusement, it did a good job as well. So what I did now is
add another image, another reference from
our previous generation because we had our duck at different stages of
the back we have this one where the
duck is about to land, and then I believe we added this one where the
duck is mid air. And then, of course, we have our duck being totally still. So just by putting
in more references, more still images,
and the same prompt, I was able to get this clip. So I was able to get this video just by adding
a third reference. So as you can see, it understood the backflip better just by
me adding in more references. Now, if you felt like
you need to give more direction to Runway
using the images, you can instantly call
them in your prompt. So we can say standing
still and then reference to the image where the duck is standing still. So
that's Image two. In my case, we'll do
at sign image two. Then we can do the other
two images over here. So we can say starting from
at Image three, I think. Yep. Image three and ending
and ending like this. So this is how you can
give more direction. For my example, I
don't need it anymore because it understood
exactly what I wanted. But this is a technique that we are going to use later on. So it's good to know how to
call your references and, of course, how to
use references. So now we know the
general structure of how video prompting works. We can now get into more details as to how to do text-to-video,
image-to-video. Then further on other things. So let's jump into our
next section and see how we could generate
more detailed footage.
5. Text-to-Video & Image-to-Video: There are two ways to get started with generating a video. The first one is describing
what you want with text, and the second one is to describe it visually
with an image. So that is the text-to-video feature and the image-to-video. And I'm going to show you what they are and how they work, and that way you can
see which one you could reach out for whenever you
have a certain project. In the previous lessons, we used a combination of both. We described what we
wanted with text, but we also had image references to show Runway what
we were looking for. But now I'm going to show you each of these
elements separately. So text-to-video on its own and image-to-video on its own so that you could
see the difference. If you go over to video
and just do the first tab, you can easily describe
something you want in motion. So I'm going to go for a
falling apple for this example, and I'm just going
to list it here. A falling apple, and
then we can stay with a blue sky as
the background. So that's my visual in my head, but I'm going to
describe it with words, and then you can
decide on, you know, the settings down here and
we're just going to generate. And while this is
being generated, I'm going to copy this
prompt, go to image, and get the visual
representation of this prompt so that I could use it for my video
and not say anything. So that's the benefit
of image-to-video. It takes away from the part where you have to explain
everything in words. So we have two things
running at the same time, and we can see which one will generate a
better video for us. This is the video where we just described what we want
without any references. So it just showed an
apple falling from different angles in
different scenarios, and it did do what I asked for. So that was the apple falling, like I asked, but it was
via different scenes. So we had the regular falling, and then it goes underneath
the apple low angle. Then it goes from the side, and there's this
random water droplets. It's falling from the
sky, low angle again, close up with the droplets, falling through the leaves and then onto a floor with grass. Runway took creative
control about the setting. The only thing I
mentioned was a blue sky, which it did keep consistent
throughout all the scenes. Let's see what the image-to-video
is going to look like. I'm going to use this image, and we're simply going to
drag it in as a reference, and I'm going to remove original prompt and
just say falling, you know, something
that would just tell Runway about the
motion of this subject. I'll just say apple falling. The thing about the
background and the tree should be pretty self explanatory
because of our image. So just apple falling
with this image, and we can see if
there's any difference between the two videos. So this was a completely
different scene compared to what we had before, but notice how it starts
with our reference. It does hallucinate
a lot with the stem. Like I think around here, the apple kind of just flips the other way without turning. So that's something,
but it's still the same background as
the previous video. The only difference
is that it only cuts to one scene throughout
the whole thing. So the same idea, two different ways
to approach it, they can have completely
different results. And you could also
combine the two, like we did with
the Duck example. The only thing that you need to remember is that
text-to-video is going to need a lot of detail
compared to image-to-video. Now, just for a simple exercise, we're going to go back to
the duck situation we had. I made a video project, and I'm going to go back to
the duck the duck session, basically everything we did. We're going to try to get this final result
using only words. I'm just going to show you
the text-to-video version so that we can compare this with the result
that we will get. We can say a yellow duck. We're just describing
what's here. Yellow duck standing on a
stone floor in a forest. Is looking at the
camera and then hesitates for doing a backflip. The duck lance and is very
excited or very amazed. Maybe we could say
the duck lance and looks around in amazement. So that's kind of the emotion
that's happening here. We can compare it once again
though we forgot the camera, camera slowly zooms out
as a duck standing. Okay. So camera slowly zooms out as a yellow duck standing
on a stone floor in a forest is looking at the camera and then hesitates
before doing a backflip. The camera is still when the duck lands and
looks around in amazement. So let's compare and make
sure we have everything. And yeah, so the whole motion, we just described it in
words. No references. So Runway doesn't really know exactly what
the duck looks like, what the background looks like, and it kind of has a lot
of creative control here. So here's the version without any references.
Let's take a look. Oh wow wow wow wow. So that's a completely
different version compared to what we had before. First of all, the duck
looks very different, and the camera is overhead. So that's something that
Runway took control of. I didn't really say
where the angle is. I just said that it zooms
out slowly, which it does. So that zoom out. Moving further away
from the subject, that's the camera movement
that we called for. And then the backflip, just like the previous lessons, it kind of had a hard time
doing the full backflip, so you can see the power of references for an
example like this. I'm trying to find where
it fails to do a backflip. Yeah, so right over here, there's a confusion as
to where the head is. And once the duck does
land, there's, like, a sound of a human saying, Wow, I didn't really ask for that, but it does look around in
amazement, and that's it. So you can see how
image-to-video can be helpful when you're trying to keep consistency across your characters
and in action. So we saw that with the
backflip, for example, if I just said the
word backflip, saw it in the previous example, but just like this video,
there's hallucinations. So the duck is, like, mutating
halfway and then it lands. Whereas if I include image-to-video components
into my project, I could get a decent
backflip like this video. That is the main difference. If you ever hear image-to-video
or text-to-video, this is exactly what it means. H
6. Multi-Shot & Scene Builder: So when you go to the homepage, there's an app section. You will find a lot of apps
here that are going to work alongside the text-to-video
and image-to-video. So the first thing
we're going to look at is the multi shot video. I'm going to click
on it like that. So this tool basically allows
you to describe your story, and then Runway is going to
take care of all the visuals. Of course, you can
include some references, but it's very cool in terms of Runway building that sequence and deciding on that timing. So there are some examples here. So this is like the two small
mice having a discussion. You can see the
scenes are flipping. There's dialogue, and
everything is consistent here. So the characters are
not changing midway, and the color grading
is consistent, as well. We have this cartoonish example. Again, everything is
in one color style, one anime style in this case. And we have tons of other
examples, as you can see. So now what we can
do is start over. Going to delete this thing, and we're going to just describe a very simple story together. I'm going to have a
story where there is two parrots and they're
talking to each other, trying to figure out who
has the nicest feathers. Choose your own example
or follow along. But essentially, you can first start with how much
inputs you want to have. If you put it on auto, Runway is going to get
your concept and then work on it afterwards to
get you a better result. But you can take full
control by describing every single one of those shots and also reference images. I could have 20 shots here
or I could have one shot. Depends on me now,
whereas this one, it's going to depend on Runway. We can decide on each one. Since this is very easy
and straightforward, I'm going to switch to
custom so that we can see our input brought to life. I don't really have a parrot in my assets and we can
see that it's optional. This is actually a good test to see if Runway will keep all of the generated footage consistent without me putting in
any visual references. Shot one, we can
say two parrots are talking to each other
and start to bicker. And then we could say,
this is a white shot. So that means the camera
is not too close, and then everything
is in the scene. The second scene
close up of well, let's describe which one the colors of each
of the parrots. One is green, and
the other is blue. Close up of the green parrot
looking very annoyed. Then we can say
Blue parrot shows his or its wings to the
green parrot in a proud way. Another shot. We could
say green parrot. Scoffs and then flies away. So one of them, I have a
description of the camera. Same thing here for
shot two, but the rest, I'm not going to say anything, so we can see how
Runway handles that. Again, no references here. Down here, we can
decide on the audio. I'm going to turn mine on because I'm saying the
parrots are arguing. And it would be fun if
they had some noises. We can choose the aspect ratio, the resolution,
and the duration. I'm going to make
this 10 seconds. If you want to
include references, you can do it down here as well. These are your recent files. And if you have anything saved, you can go over here, and
let's click on Generate. So we have four shots. Let's see how Runway
puts it all together. So it took a while,
but let's look at our fight scene between
the two parrots. Lap. Apago. Lap, lap, ap. So that is a very good
video that Runway made, and it kind of adhered to everything that I
described right here. This that we're seeing
is a white shot. So white shot of
the two parrots. The green is there,
the blue is there. The talking part, it kind of understood it as yelling.
That's what we heard. And then Scene two is the
close up of the green parrot. And then this is supposed
to be very annoyed. It's kind of cocking
its head to the side. The details are great
here, which is good. And then scene three, we had the parrot show off its wings, which is doing right now. Then I guess the
head moving left and right is it being very
proud of its wings. And then the last shot is the green parrot scoffing
and then flying away. And then we can see
this additional detail of kind of looks like leaves, but I guess it attempted
to do the feathers, but it's completely fine. So this is purely
text-to-video right now because I didn't include
anything visual for Runway. It did a really good job
with the details and the consistency
across every shot. And if you ever wanted, like, a full video
where there's a story, you can always come
to multi shot Video, the app to get this result. Now let's move on to
our Scene Builder, which is where we
have one frame, and we want to
animate that alone. Now, the next tool
that we can use for describing one scene
is the Scene Builder. So that's underneath what
we were doing right now. And there's two steps to this. First, you describe
that first scene, the first frame, and then
you animate that frame. So it starts with few words, or you could add
references again. And then here, actually, Runway will see what you wrote and what you
uploaded as a reference, and it will tell you some cinematic effects
that you could add. For example, camera
movement, overlays, color grading, and
anything else that Runway would suggest
for your description. So let's go ahead and describe
the same parrot situation. We're going to say two parrots, rainbow one, green parrot, one green and one blue parrot standing next to each
other and looking annoyed. So that's the first scene
that I'm envisioning. Let's look at these
suggestions down here. It's going to first ask me some clarifying questions.
So no references. We can answer the first one. What visual treatment suits
these irritated parrots? I'm going to go with realistic. Second one, how they should express their annoyance.
I guess, this one. Third thing is, I think I'll
go for, like, a nature. Then we have composition. Camera capture their attention. We could say, side by
side profile sounds good. All right, so 16 to
nine, same aspect ratio. We're going to generate it, and the animating scene is where we get to describe the
motion if we want to. So it's already generating
the first frame for me. This is what's going
to go over here. And then, again,
we're going to have a little quiz about
the motion part. So this is my scene,
my two parrots. You can see how immediately
it saw my input, and now it's asking
me questions. And again, if you want, you can describe
something else here, but usually this
should do the job. So how should the camera move
to capture the interaction? We could say, maybe
stationary character. What physical movements should the parrots perform
on their perch. We could say, you know, the wings should open and close, just like we described
in the other example. How should the background
respond to the presence? Number one sounds good. We could turn on audio
and then decide on the resolution and then
click on Generate. If you want to
change your frame, you can do so right here or exit completely
and start over. Let's generate our parrot fight and see what it looks like. Now let's take a
look at our video. Hi. Hi. So this is purely based on one frame and all the answers that
we provided Runway with. Now, this is one scene, meaning that there is
one action happening, and it's not meant
to be that long. Whereas the previous tool, it was meant for a
longer sequence. There's a story that
we're trying to tell. As you saw, we described the parrots fighting and
one of them showing off, and then the other
one flying away. Whereas here, they're only
fighting, so it's one scene. Now, the style is
very different. But again, we're able to see some issues regarding the
performance of Runway. Notice how in the first scene, this parrot's eye is like
a dark color and then it turns into a
regular parrot eye. So that was interesting. But we can see how the same thing happens to
the other parrot actually. That's something you
would have to fix if you were using this for
a polished video. The details here
though are incredible. We can see that it maintained the style that we asked for. The subject facing each other. That's what we asked
for here as well, perch is here and there's nothing really going
on in the background, the movement, the ruffle of the feathers that's
happening here. And then when the
parrots do something, you can see that the perch is
kind of moving up and down, and then we get this
random final scene where it's a close up
of one of the parrots. If you want to extend
your videos to a story, you can use these two apps. They're really helpful, and
the more you work with them, the more comfortable
you're going to get combining them and using
the different parts. As you can see, whatever we
produced in this lesson has a lot of options after we
have that generated footage. So you can build on top of this to tell that story
to the final touch. And these are things
that we're going to look at in the future
lessons as well. So we saw these two
things where we let Runway kind of take control
of the pacing and the timing. What if you wanted
to have control over that yourself and
you want to decide at what time this parrot does this and how long they
do one action for. So that's going to
be our next app, which we'll look at
in the next lesson, which allows us
to use keyframes. So have control over
every single scene.
7. Keyframes & Controlled Motion: So keyframes are
instances in our videos, and every video is made
of multiple frames. So there are two things that allow us to work with
individual frames, and that is within our toolkit right here and then
in one of the apps. So all we have to do in order to work with these
tools is to provide two singular frames
and have Runway figure out the middle
based on our description. So the apps one, let me just go from the start. When we go to Apps Video and scroll all the way down
to animate keyframes, this is where we get
to create transitions. So transitions are how the video goes from
point A to point B. You may have seen it
in videos already. If something fades out,
that's a transition. If something gets blurry and
then another scene comes on, that's also a transition. So with this app, you get to create transitions
based on your inputs here. Now, keyframes is also mentioned in our
tool when we go to video keyframes these two are different things
because the first one focuses on the in between, just the transition part, whereas this one is asking us for the start of the
video till the end, and then we can have Runway complete the action in between. So I'm going to show you both of these so that you know
what each of them are for because they may be
confusing due to their name. So let's start with
the easier one, which is in our tool. So go here, Video keyframes and we're going to provide
Runway with two things. You have the option to upload
this from your computer. This could be a
photo that you have taken or it could be something
you have downloaded. We can grab something
from our asset folder, and then we can also upload from another device
with a QR code, but that's not really
what we're going to do. Before I start here, I'm going to make
these frames myself, and you guys can follow along, so we have the same things. Quickly let's switch
over to image, and we're going to
describe something that would have a
start and a finish. We can go for a beautiful
landscape, maybe mountain. Landscape with a
tree in the center, fluffy clouds in the sky
and lush green trees. So I just want a very
simple landscape. But the center
object is the tree. So that's what I want a
very tall tree, let's say. And then down here, I'm
just going to go for four, 16 to nine, and then the
rest of these settings. So no references, and my idea for this lesson is to
have this landscape. And then in the second shot, there's a nice bird
on the branch. So there's a start
and then the end. And I want Runway to
animate this bird coming in and make sure it's the same bird that
I provided in my. So we have a bunch
of trees right now, but what I want to do
is maybe zoom in a little so that there is
enough room for our animal. Because if I ask for a
bird to just, you know, sit over here, we
can't really see it because we're really
far away from the tree. So let's pick the one
that is the nicest. I'll grab this guy, put
it in as my reference, and then we can say close up of the tree branch with
the same background. Same settings. I'm just trying to come up with
that first scene. So now I have a better scene. It's the same background
again, as you can see. Let me just zoom this
in. So same mountain, but we just have a nice
close up of our branch. I'm going to pick the
one where we could put a bird on it,
maybe this one. So this will be my first frame. I'm going to favorite it so that it's easy to
pick out later. And now we're going to put
this as our reference, and we can say a bird, maybe a red bird
sitting on the branch. And that's it. If you have
a specific bird in mind, you can name it here,
but this is going to be the end of my
video or my clip. So it starts with
this nice scenery, and then a bird flies in
and lands on this branch. A very simple scenario.
And there we go. Again, it's the same thing. I'm just going to open
it and double check. So I think this one
is the closest, and I'll favor this, as well. So I have my start
and end frame. What we can do now is switch
to video, go to keyframes. So since I have my session
open on the right, I could just grab this
in as my last frame. And then the first frame
is the one we favorited. And I'll put this in here. So now we want the
description of what happens between
frame A and frame B. So we can say bird flies in
and lands on the branch. And that's it. So
choose your model. I'm going to set the duration to auto because I want Runway to decide how slow or how fast
the flight is going to be. And, of course, we have
all our other settings. Just send that in and we
can see what we're going to get with this version
of keyframes in Runway. Now, while this is happening, I'm going to go over to the next tool so that we can have two things
happening at the same time. Again, I'm going to put in my
first frame and last frame. But this time we're only
focusing on the transition. So because I have my assets, I could just click on
it, click on Favorites. That's why we harded H one. Go for this guy, double
click, and then last frame. The transition that
I want is a fade in. Now, I didn't say fade
in with black or white, let's see what Runway
decides for this scenario. Notice how here we don't have a lot of options because again, this is only the transition. It's 8 seconds, so
that's how long the transition is going to be and then we can send this in. Our bird video is completed for the first
tool. Let's take a look. You can see how that
motion of the bird flying in was made
using this tool. Just for reference, this
was our first frame, and this is the last
frame right here. It starts and ends with the exact same images
that you provided. Now for the transition, it's going to be very
different. Let's take a look. So as you saw,
there is no motion. It's more so image one, transition to image two. In my case, it was a
fade in transition. So it just very simply
brought in this bird. And it also added some
motion to my bird, so that wasn't really
the case with the image. But this part right here of the bird kind of
appearing into my scene, that is the animated
keyframes capability. So they both have
keyframes in it, but it doesn't mean that
they do the same thing. Bear in mind that
Runway cannot do really technical
transitions because it's working with
just two frames. So there isn't a lot
of control for you to handle the speed or the
different frames that come in. You just put in two inputs
and there is an output. So just try the very
simple ones like fade in, dissolve, swipe
right up down left, and those sort of things. Now we know how to
generate video in Runway. We know how to do it from
the start, from a text, from an image, and then the in between with keyframes
and Scene Builders. In the next lesson,
we're going to look at how we can edit a video. So we have our clips. They look great, but maybe we
want to extend our videos, remove something from the scene, maybe add a character, and basically the things
that we see right here. These are the options
that will pop up every time we generate
a video in Runway. So we're going to take a
look at what they are. And that way you will
have the full toolkit to work with videos
in this platform.
8. Edit Studio: Now we're going to dive into editing a video once
we have that video. So we saw how to make
it in numerous ways. But now we're going
to try one of Runway's most powerful tools, which is the Edit Studio. So for that, I am going to build a very simple video that you
can also follow along with, and we're going to
use the Scene Builder because I want one
action, one scene, rather than a multiple
of them so that we can see the power of that
tool in full focus. In the Scene Builder, what I'm going to do is
have someone just walk across the city or across
a forest or something, but I basically want one
subject and the video to be in a way where we could easily replace
the background, add things in,
change the lighting, and not lose the main
components of our video. So we're going to
start with the frame, say a young boy
with a, let's say, fruit basket walking
on a pavement, maybe with a neutral background. That's my imagination. And then we could answer
these questions so that we can get the final
scene a lot easier. So the visual mood, we could say a
digital illustration. Actually, let's say, we could do I think Golden Hour is nice. How is the boy moving? Stride forward or maybe, I think the first one's good. Environment. We can
say the countryside. From what angle we could say
side profile walking away. Alright, so we're going
to finalize our prompt. I do want to make sure that
the environment is as if the cameras on the side of the road and not so
much in the road. So I could maybe
add a reference. Alright, so this is
just a camera note. Let's send this in
and see what we get. If the image is
not to our liking, we can always go back
and make the changes. And the good thing about
that is that we're not generating the entire
video for every change. So here's our character. He is walking here to here. He has his fruit basket.
Everything looks good. And we have that golden
hour that we asked for. Now for the camera,
I'm going to do, how should the camera
follow this child? I think it could track
from left to right. How does a child
carry themselves? We can say, maybe
he's having fun. I have my settings, let's
generate this video. Let's take a look
at the footage. So this right here
is a good sequence, and now what I can
do is edit it. Maybe I don't like the
shirt that he's wearing. I don't like the environment. Maybe I want to add a character
and do things in between. So once we have our footage, you may have noticed that we have a lot of options down here, and the Edit Studio
is the Edit button. So once we click on this, we're going to be brought
into a different window. Where we will be
getting a timeline, just like any other
video editing tool, and then we would say the
changes that we want to Runway, and it will do that based
on the selected frame. So this is what
the editing studio looks like. We have the preview. This is our original footage, the name, and this is the timeline that I was
telling you guys about. So this red thing is going to
show us the selected frame. So whatever we see here, it's this instance marked
with this red line. And then the purple line
around the timeline is what in the video right
now, so the duration. So if I grab the first bit, drag it to the left, the right, we are going to shorten the video
from the start. So this is the end of the video. This is the start and I just
got rid of the beginning. I could do the same thing
from the other end. Maybe we don't like the ending. I could cut that out completely, or we could save
something in between. So somewhere in the middle.
So we don't want the end. We don't want the start.
We just want the center. So you can play around with these if you want to
do some trimming. There we go. Back to
the full duration. And then down here
is our edit box. So this is where we
would say the changes. As you saw, there is no
tools for us to use, like all the other
editing programs. So it's mainly just runways way of understanding
what we want. So we have two options. We have editing the
selected frame, which is what we're
seeing right now, or you could upload a frame to kind of blend
into what we have. I'm going to use the first one. Have the output options. These are more professional, but MP four is the default. We have the aspec ratio
of the edited footage. If you want it to be
exactly like it was before, just choose that ratio so Runway does not switch
it out later on. Now, when it comes to models, you have only the options that work with the
editing studio. So this is what it looks like, and then the resolution. So for example, I could say, let's go to the start, add a hat. A red hat. I'm going to cut the duration
so I don't waste a lot of credits because
the main the star of the show is the Edit Studio. So a few seconds,
I just want to add a red hat for my subject. We are going to
generate this frame. And whatever frame we generate, it's going to go
on the right side. And then based on the one
that performed the best, we're going to
generate the video. The edited frame, if
it's not to your liking, you can do another prompt. It will go here, two,
three, four, five, then you choose the
one you prefer, and then you export
that to the video. So it's not going to
generate a video right now. So there is hour before, this is after. So
that's all it did. And if I'm happy with this hat, I could now generate it, and that's going
to go over here. These are two
different selections. So this is the famous
model for the Edit Studio. But while this is happening, I'm going to show you
what the studio looks like if you approached it
from the apps section. Over to apps and then
go to Edit Studio. So this is the same procedure, but this is where
you would normally access it if you wanted
to start from scratch. So I'm going to select my
video right over here. You can upload a
video if you want. And as you saw, it's
the same thing. So the video is done, and now my subject
has a red hat. Now, notice that it cuts short. That's because we did trim the duration of our video
before we generated it. So if you wanted to
do the whole thing, you would keep the
whole duration. But that's basically
the editing studio. So you could be at a red hat, remove the subject, add a
bird, whatever you want. You will choose the
frame where you can best work with that change. So, for example, if I
wanted to add the hat, the frame where, like, the subject's head is kind of covered, let's
see. Like here. May not be a good frame because I'm not really seeing
all of his head. So that's why I chose a frame
where his head is isolated, and one way could
have an easier time putting that heat on my subject. And you could just continue
building versions and then, export the final product. Now, this is one way to access the editing studio the other way is to get it from the apps. So when we go over here, video Edit Studio is
the exact same thing. The only difference is
that you're starting from an uploaded footage
or something selected. So it's not like a
continuation of a session. You're actually building
a new session here. So if I wanted to
do the same change, I could select my
video, this one. And then you can see
it's the same interface. You could do the editing
studio with something you made in Runway or something that you had before
on your computer. You can upload it the same way. So we're going to take a
look at some of these apps that have to do with editing
and existing footage.
9. Change the Scene: We're going to be
using the same video of the young boy walking
with his basket, but now we're going to target those changes using
different apps. Now, in the video section,
if you scroll down, there's a bunch of things
that are very specific, such as changing
the video lighting. So that's only for light. We have the weather,
so that's going to target the conditions
only time of day. We have options to color grade. We have options to remove things from video
and so on forth. So these are going
to do one task. But when you put
it onto one video, you can kind of get full
editing experience. We looked at the Edit Studio, and it didn't give us a lot of control over certain things. We just had to rely on the
way we write our prompt. But now we're going to start
by doing something very simple such as removing things from video. So
let's choose this. Again, I'm going to use the same little boy video so that we can see the
changes one by one. So simply select that
video from your assets. Make sure you're choosing
the one that is the original and not the one
that starts with this. That's your Edit Studio change. Double click, we
have our footage, and as you can see,
this is the original. He does not have a hat. Now we're going to say the
thing that we want to remove. So I'm just going to
say remove subject, since there's only
one, and that's it. Let's generate this and take a look at a
before and after. So without the subject, it's just going to be
the camera panning across this field at
this time of day. It could be nice. All right. So this is our video without the subject.
Let's take a look. So as you can see, our subject
is nowhere to be seen, and I didn't even
have to say anything because this tool is
going to target one task. So that's our removed subject. Let's take a look
at our next app. The next tool is going to target the backdrop
of your video, so that's anything other
than your subject. So if you scroll down again, you're going to see
video backdrop, and we're going to
put our subject, the little boy walking and just put him in a
different environment. So maybe he's in the city. We have some presets here
and some other places. Select your video, same thing, and then you can either choose
one of the presets here or do something completely
different using the box up here. I will put our
subject in the city, and I'm not going
to say anything in this box because I
want to see how Runway is going to deal with the camera movement and
what the subject is doing. So let's take a look at our
subject in the city now. So he's no longer in the farm. So you can see how
seamless that was, and it's interesting
that it added the elements from
the original videos. So in the original video, there were like bushes
here covering his face. Runway replaced that
with these fences. And then around here, where the subject was
going towards, like, a farm gate, it's
showing this metal gate. So it kind of adapted the original elements
to city elements. And that's interesting to see. So this was a cool app. Let's try another if we
want to change the weather, that's going to be a lot
of different components. That's going to be the lighting, the color grading,
maybe camera movement. And that's something that this app is going to
target right over here. So changing weather conditions, again, choose your video, and then it's going
to do all the work based on the preset that you provide and
optional descriptions. So here's an example. I'm just going to
use my subject here, and I could maybe put him
in a snowy environment, or let's actually go for dark because it would make
sense with his outfit. Can include optional
details here, but you don't really
have to as long as you're choosing
one of these presets. Let's generate that and see our subject walking in the dark. It finished generating. Now let's see what the changes are from this point
to this point. So all I did here was turn the lighting the
weather into dark, so I didn't add
any snow or rain, but notice how it completely shifted that golden hour
instance that we had, and it made it look
like it's past sundown and he's
about to go home. And that is our second app. Let's look at the next one. Our next app is for
the time of day. Now, this is different
from weather because weather was going to add
additional elements, such as snow, rain, some leaves if it's, like, windy, but time of day is just about night or
day. So that's it. And you can access
it right over here. I'm going to choose my
video one more time. And then, again, we
have the presets. So all of these work
in the same way. There's presets. There's
an optional box, and then your input video. So I'm going to look at this, and we could try
maybe early evening, so something a
little more pinkish. Or actually, let's
try the high noon, so it shouldn't be
golden hour anymore. So this is for the time of day. I finished generating. Let's see the difference from the sunset to the high noon. So now the lighting
has changed and it does look like it's maybe
like 1:00 P.M. Right now. All the elements are there, and even the colors of the bushes were adjusted
to match this lighting. And our last app for this
lesson is the lighting. If you want to change
the light of your video, you can easily do that
with this tool right here. So you could maybe isolate the subject with,
like, a spotlight. You could add blue light, and there's a bunch of other things. What I'm going to do is choose something that would
make sense for this subject. So I guess I'll do spotlight, so it should darken the
surrounding and just put our subject
as the main show, and I'm going to generate. So I chose spotlight as
we saw, and immediately, you can see the difference
between the first frame here and the first frame here.
So let's take a look. Everything was darkened
except our subject, and you can see that
the spotlight is moving with every single
movement of our character. So those were five
different apps that you can use to target specific
things into your videos. And when you combine
all of them together, you can get a fully
edited footage on top of the editing studio. So for my next app, I could use this input instead of this and then
build on top of this. So maybe I want it to be like a spotlight look, but
at the same time, I want a thunderstorm so I would combine the video lighting
app with the weather. And together, I will get that final shot that
I'm looking for. So we looked at the
different editing apps. Now in the next lesson, we're going to focus on
the style and the color grading because that's also something you can
do within Runway, just like you would
want to do with a regular video editing tool.
10. Style, Color & Final Polish: Now, the last couple of editing apps that we're
going to look at, as we said, is regarding the colors and the
stylization aspect. So we're going to once again use our boy at the Farm video. But this time, we're
going to scroll all the way down to stylized
and color grade. And then later on, we're going
to see how we can upscale our final product just
like we can with images. So first, let's start
with the stylized video. Once again, I'm going
to grab my footage. So we have all the different alterations that we did so far, but I'm going to
use the original. And then down here, we
have some presets that we can say I want to do a cartoon
version of my subject, I could easily
grab a preset down here and then have
Runway do it for me. Again, if you don't
like these presets, you can describe the
style that you want. The only downside to this is that in order to
get a good result, you would have to write a strong So if I say something
generic and then generated, I might get something
irrelevant to what I asked for, but I have lost X
amount of credit. So start with a preset, and if you want, you can build on top of what's available. So I want oil painting, but let's say I wanted that to be only in black and white. So using this preset, I could kind of go over here and say black and white
oil painting. So you can combine
the two, as well. So I will go for
watercolor because I want this to look
like a children's book. So we're going to use soft pastels in
addition to the preset. So soft pastel colors,
and that's it really. Then we can click on Generate. So there's my watercolor
character. Let's take a look. So these are very
nice bright colors. You know, it works well with my child character, basically. And the only thing
about it is that it's a little inconsistent in
the first couple of frames. But near the end, it
actually, you know, changes the character
into a two D one, and, you know, the surrounding
environment is perfectly done. Our next step is going to work with the footage that
you have initially. So it's not going to change
anything in the video, but more so the filter
that you put on top. So that's color grading. If you go to video, scroll all the way down, you should find
color grade video, and this is where
we get to add and manipulate with the moods
that our footage has. So here's an example. We have a normal warm video, but we could turn it sad and
cold using the color blue. And of course, the exposure was dropped down here as well. So I'm going to
grab our footage. And I will do something
completely different. So here we have this
warm upbeat moment where this kid is
just walking around. But maybe I want to make this
look like a sad noir film. I could easily grab this preset. Again, you can combine it with some descriptions and
click on Generate. So now we have this
very depressing version of our initial video. You can see that we even have the apples kind
of semi saturated, which adds to that
noir film element. The contrast is great, and it's really a
fun thing to do with your footage if you want to add a little bit of mood to them. And you can see the
prompt that Runway made based on that one
preset that we chose. Last app that we're
going to look at is the upscale video. So again, very straightforward.
It's right over here. And this tool actually uses the top as AI model
right over here, which is only for upscaling. So this was the same scenario
with image upscaling. So if you have ever done that, this may sound familiar to you. So if we have a video
that's perfect, but it's low quality or
it's not large enough, we can easily come over here
to upscale that resolution. So the cool thing
about this is that you can use this with
your own videos. So it doesn't always have to
be an AI generated video, but maybe something you
caught on your phone as well. So let's grab the
footage one more time, and I'm just going to
click on Generate. As you can see, I
don't have a lot of options regarding the new
resolution or the new quality. For now, it's only one
option, the default. So it's only four K. So
this is what we have now. You can see the four K label. And now we just have
an upscale footage. So nothing has changed regarding the colors,
lighting, subject. It's only about the quality. So whether you made the
video in Runway or didn't, you can always come
here to upscale it very nicely with Topaz AI. So those are three other apps that you can add on
top of the others that we learned in order to get that perfect video
before you export. So far, we have relied on videos that barely
had any dialogue. We did have the parrots, but they weren't
really speaking. And so in the next
couple of lessons, we're going to look
at characters. So when we have a
subject like this, but they're going to say
something that we decide. So that way, we can see how we can control mouth movements, the sound that comes out and the consistency across
all character videos.
11. Creating a Talking Character: M Let's take a look at how we can work with
characters inside Runway. So the way we're describing
a character here is that there's going to be a
subject that is speaking. And so it's very important that the mouth movements
are accurate, and it looks like the person is really speaking in the video. There are a bunch of tools that contribute to this
character scenario, and they all start from
a reference image. So that's what we're
going to do first, build that reference
image of our subject, what they look
like, and then get a very nice portrait
of that person. So we can go to tools
and then head over to image and just
describe your subject. This could be an animal, it could be a man, a woman or anything else that you want. But just make sure
that their face is clear and there's
nothing in front of their mouth and that the subject is not too
far away from the camera. I will be going for a human
subject for this lesson. You guys can follow
along or just switch out the human subject
to something else. We can say a portrait
of a man with red hair. Let's make him look
a little different. And we can see a close
up portrait. That's it. And again, I'm going to
stick to nano banana. They're really good
at realistic images. So if you want to
do a human portrait or a hyperrealistic portrait, you can try out nano
banana, as well. Alright, let's try this out. I'm going to do four
versions 16 by nine, and then we can
refine it further until we have that
perfect subject. Here I have four different
images, as you can see, they're all close to the face, and I can clearly see the mouth. Now, that's very
important because Runway will need some space to push the lips apart and together in order to
make this person speak. Now, you want to make
sure that there's nothing around the
mouth as well. So if there's a plant here, you want to remove that because Runway is going to need some decent spacing around the mouth to manipulate the
skin into talking. So I will go with this image. It's very clear and we
can now switch over to apps where we can access
the character applications. So let's go down and
grab our first one, which is image to dialog. So we have the image right
now and we are simply going to drag it in here or just select it from our assets. Over here, we can describe the words that this
person is going to say. You can put one
word, two sentences, or any other amount as long
as it's under 130 characters. Let's say hello, my name is and we're just going
to generate this again. This has a limit of 8 seconds. As you can see, I did not
mention anything else. I just had to click a button. Now, if you want to do
a very clear dialog, you can use an image like this. But this also works with an image where everything
is not so obvious. So I'm going to grab
the Duck example from a couple of lessons back. And we are going to see
how Runway is going to work with that to see something. We can say I'm so happy, and we're going to
generate this, as well. I will do another example where the subject is not looking towards the camera,
something like this guy. And now we have three
different scenarios running, and then we can see which
one performs the best. So our three scenarios
are ready to see. Let's check out the first one. Hello. My name is Paul. It's BotalEducation, Aspics MTMs and blood for
H syncs ever all brain. Hello. So notice how it
said the first part, right, because I only
provided one sentence, but it continued to speak
in a weird language that is probably a addition
that Runway gus did. So I think it's because the
video has to be 8 seconds, and in order to fill out that
extra seconds, it did this. But notice how even though
it's unclear language, the mouth movements
are very on point. So we're just going
to take a look again. My name is Paul.
It's Brasiducation, spic MTMs and blood for
the simplest ever all by. Everything regarding the
facial expression, like here, I had kind of like a beat way of saying my name is,
and he's smiling. And then everything else, there's a normal amount of
head movement, blinking. The teeth are very
clear as well, and every single word is
pronounced perfectly, even though we cannot really
understand what he's saying. Teil tucs, aspeic,
mites, and bloods for H, psbus every little then his body is moving as he's
speaking into the camera. So in terms of dialogue, Runway is very powerful,
as you can see. But this was the ideal
scenario where we have the subject looking
straight into the camera, and there is not a lot of room
for Runway to hallucinate here because everything is
already set out perfectly. Let's look at our second example where we had a non
human subject. Nilo, hi. I'm so
happy to be here. So even though we had a
duck in this example, you can see that it
did the expression and the body movement for this
happy sentence very well. But it only starts with that
foreign language again. And for a duck, we can't really see if it's being
pronounced perfectly, but I would say the middle
part is pretty good. I'm so happy to be here. You can see that
the body's moving upwards near the end
of the sentence, and that is pretty good. In our last example, the subject is not looking at the camera at all. So let's see. I'm so happy to be here. I'm so happy to be here. So it's interesting that it decided to have the subject
walk towards the camera. But again, the mouth
movement is right. It kind of, like, stretches
out near the end. But it was able to say the
sentence that we asked for. So that's image to dialogue. No matter which type
of image you provide, it's going to say what
you want it to say, and it works for human animal. You could try an alien
or something else. It will all be okay. But as we saw, the more
clear the image is, the more natural the result
is going to be like. So that's our first
app where we could get dialog in instantly
without much input. Our next app is going to be the character
script to video. So this is more about the script and less about the visual. So here, what we can do
is decide our character. There's a bunch of
presets, but like I said, the most important input is the text script or
the audio script. If you have an audio file, you can just add it right here, but I'm going to do
something with text, and that way you guys can
follow along as well. Let's provide a bunch of sentences for our
character to say. We have our script. I'm now going to choose the voice that is going
to say this paragraph. You can grab from
the presets here. The tags are really going to be helpful because you
can see the age, the gender, and basically
the tone that they have. For example, Billy is hyped, whereas Bernard is
more confident. So you can decide based
off of that, too. So let's preview the sounds. Okay, I'm ready to get started. Don't wait. Call today for
your chance to save Big. Hey there. I'm ready
to get after it. Let's go. And if you want, you can do a search
up here, as well. So we can say, I will go for some narration here because we have this story.
So let's see. Oh, goodness gracious.
You are, too. Hello, thanks for this morning. I made a macho. From the
depths of the angel. Well, you could
always come over. My father. He know so
I will go with Miriam. I'll click on it, and
now it's selected. You also have the
option to clone your own voice and generate
one from a description. So when you click on Clone, you just either speak into the computer or upload a sample, and it's going to study that
sound and give you a preset. That's one option, and then
the other one is where we can describe a certain tone
that we're going for. So you can say,
here's an example, old British man, calm,
clear, and relaxing. Any sort of adjective that
you want for that voice, you will put it in here in addition to the
age of that voice, their gender, accent, if you
want, and anything else. Once you're done,
you hit generate, and that's going to be in
the library for you to use. I will stick with
Miriam for this and I now have to
choose my character. We have a few options. If you click on them,
you can see a preview, but you can also
upload your own image. This could be a real
photograph or it could be another AI generated image.
We're going to try both. I will just grab let's see if
we have a female character. I just have this image
from a while ago, I'm going to grab it and
let's grab Miriam again. And then we're
going to generate. If you want to grab
an image like this, just go into the
image generation tool and write a portrait of a woman with the
following features, and then you can bring
that image in here. We're also going to try
one of the presets here, so I'll go for the cat. The default voice is Rachel and I'm going to have
this run at the same time. Here is the character
that we set up. Let's turn on the audio. It was pouring down on a Tuesday afternoon my hot
cup of tea by my side, and I was just about
to fall asleep. It sounded like the
sky was about to tear and I suddenly
recalled why I'm here. It was pouring down. So you can see the facial
expressions really do match the words that
the subject is saying, especially near the end where it said the word suddenly and it kind of paused to say
the rest of the words. So that's the first option, and notice how it kind of
cropped into my subject. We initially had
a vertical image, so it did that
adjustment on its own. The second example is
Runway's own character. So this is preset for us, and when we clicked on it, it automatically gave us
this cat and this audio. It was pouring down on
a Tuesday afternoon, my hot cup of tea by my side, and I was just about
to fall asleep. It sounded like the
sky was about to tear and I suddenly
recalled why I'm here. It was pouring That's
another good example, but the facial expressions were not as strong as
our first example. You can kind of play around
with the presets and your own images to see which one works best
for your project. So that's how you can
create a talking character, but there's still more apps
in regards to such instances. For example, we also have
something called Act two. This is an older
tool within Runway. If you have used Runway before, this will sound familiar. But basically, you're
able to transfer a performance onto
another character and do different things. So we have this to check out, and we're going to see
how we can get more into dialog and working with
multiple characters.
12. Act-Two & Character Swap: Mm hmm There are two more apps that we can use when it comes to characters. The first one is going
to be the Act two, which is more about the
transferring of a performance. So for this, I do have a resource for you
guys to download, and it's basically a video of someone freestyling
into the camera. And what I'm going
to do is replace real human with something
I made in Runway. So let me show you what
that resource looks like. So this is it right here. We have the subject. They're dancing, sort of, and they're speaking,
and there's just a lot of movements
with their body. So we're going to
have Runway transfer this performance onto
our custom character. So I just uploaded
that video right here. You can drag it into this box, and then we're going to
decide on our character. So you can either
make a character yourself or use the stuff
that we have made so far. What I'm going to do is make a very simple image of an alien, and I'm going to have the
alien do this freestyle rep. So in a new tab, I'm going
to you know, go over here, grab our image
generation tool and say a close up portrait of a green alien wearing chain
and with face tattoos, wearing a gold chain or maybe silver for
the green contrast. And let's generate four
versions of this prompt. Going back to our session here, we will upload that
image into this box. You can either drag it in
here or select the asset, and we can see that we
have a bunch of options. And then we simply
choose the voice. Now, for me, maybe
I should choose a different alien because I feel like this guy is way too
close to the camera. So we could say a
half body portrait, and I'll go for one
just so I could get the positioning right first and then made
different versions. So far the face tattoo request
and the chain request is, I'll put it really nicely. And I love the details
that Nanobanana provides. You can see that right here. Now we have a somewhat
similar body portion to our video, the original. I just need this subject
to look at the camera. So what I'm going to do is make another generation
this time with four because I know that
this prompt is good to go. All right, so we have
a bunch of options. I think this guy is, you know, very suitable
for what we need. And what I'm going to do
is go back to my session and select that asset
from this window. So right over here,
double click, and we are going to
get some options now. So there's an option for you to include the body movements. And when we looked
at the top video, we did notice that he's moving to the side, he's
moving backwards. He's moving his shoulders, and that's exactly what this
thing is going to pinpoint. If you want the character
to just be still, you can turn this off, but
I'm going to leave it on. And then we have
facial expressions. So how identical do you want
it to be to this video? I will increase mine
to five because I want the alien to kind
of replace this wrapper. If you want, you
could also include voice for this sequence, but because my original video
is not really saying much, I'm going to leave that out. But if you wanted
to include that, you can just come over here, search for the voice that you
want and just click on it. Mine will be without that. Again, five gestures turned on and I'm just waiting
for this to be uploaded. Here's our rapping lien. Let's see how he performs. And. Followed the exact performance. It only did not include the hands that the
original video had, but I would say that's okay. If we had the alien in a similar position where
its hands were in the shot, maybe it could perform better. But what I'm going
to do is now play both of the video
side by side so we can look at the
facial expressions because we set that
to the maximum. So it should be identical. Alright, so this is our video. I'm going to just minimize
it and then play each one. So the looking down, this
one's a little delayed. But the head movement, I would say is pretty good. It's only the arms
that are an issue. So even the end where he
kind of closed his mouth, that was replicated at the
end of the scene here. So it did a better job with
the facial expressions. But again, if you have
a character image that is closer to the driving video, you could get identical results. Alright, so that
is our first tool, and then we have our final
option for characters, and that is the character swap. So basically what
this does is that it's going to detect a
video that you have, take out that character and replace it with a
different character. So here's an example. This was the original video, and this is the character
it then turns into this. So there's a bunch of inputs
that you're going to need. First one is the original scene. And then we can describe
the shot that is going to go into
animating this scene. And finally, we have a still
image of that new character. So we're just going to upload
something for the scene. I will grab our Alien again. So let's grab a different
version, maybe this one. And I want the alien to start
walking towards the camera. So subject walks to the
or towards the camera. As camera goes back. And then I want to replace this alien with a
different subject. So let's generate
one real quick. Go to make a new
tab with the tools. We can do a polar
bear portrait of a polar bear half body. So this is the
result of my prompt. I should have said a portrait of a standing polar
bear because we're trying to replace
that character. Let's try that one more time, and then we can kind of
go along with our app. So we need a portrait
where there's enough details so that Runway
can put it into this video. Well, right now,
it's an image later, it's going to be a
video with this prompt. So now I have a better result. I basically changed my prompt to a tall standing polar bear. But now I want the polar
bear to look at the camera. So I'll just put looking
at the camera, half body. Since we have the reference, it should be all good to go. I could technically feed
this into our session, but I want to start
at the right place and kind of have a facial
angle like our alien. There we go. So I'm
just going to go over here and select
my polar bear. This polar bear needs to be in the city instead
of the alien. So already this
looks very peculiar, but let's see what the wrapping
polar bear has to say. So as you can see the prompt of the subject walking towards the camera as the
camera pulls back, that has been
executed perfectly. The only issue that it
has, but, you know, I'm thinking there were no
ways around it is fitting the big polar bear head
into this tight collar. But I would say it looks okay. I like the way that the
lights are reflecting on the polar bear because it is walking through
a crowded city. I would say that's pretty good. You can combine this
with an image of yourself if you have
a nice travel video. It's a pretty fun
tool to work with. And combining that with
the previous tool, you can kind of get
different shots of your character doing
different things. So, for example, if I wanted the polar bear to
walk a certain way, I can record a video of
myself walking that way and then feed it into our
Act two application. So that's how you
can build on top of the other apps and get
that performance that you're depending
on how much work you want the characters
to do in the video, you can go for each of these
character applications. We looked at four
different applications. They're all really interesting and very straightforward to use. So we can turn to
each of these tools, depending on what we
expect from our character.
13. Sound Effects & Audio: M Mm hm. If you want to intensify an emotion or add some
mood to your videos, you can consider
adding some music. And that is something you can create right here in Runway. We have an entire category dedicated to audio
in our apps section. So once you're done
generating your footage, you can come here for some
additional elements and then get a finalized
cinematic product. There are only two options. One of them is SFX, which is basically
sound effects, and then we have stylized audio. So that's pretty much it in terms of what
you can do here. Can easily start by just saying the sound and if it's
a very common sound, Runway should be able to generate something
decent for you. But if it's something
more specific, you would have to describe
it in more details. We're going to try
a bunch of things. I will start with
something very simple, such as academeo. Very simple. I already have the animal there, and I can't really do much, but you can decide on the
second for the output. Maybe I want a
five second audio. Click Generate and it
will make it for me. Mu. Breeze. There we go. That is very clean
and very straightforward. Now, the thing is that
once we have an audio, there isn't much
to do down here, like the previous apps that
we saw in the lessons, but I will try to
now kind of upgrade our prompt into something a
little bit more complicated. So, for example, if I have a video of spaceships
and aliens, I will want a sound effect
that would match that. So an airplane
sound is not going to work for me because
this is fictional. So something like
a description of that object that we want
the sound effect for. So for the spaceship, I labeled it with sci fi, so it's not going to
be something normal, deep rumbling to show that it's really big and it's
shaking the entire floor. It's kind of fly by, and then
it has this type of engine. So I'm expecting a lot
of sounds, but not like, you know, mechanical sounds, but more like a
futuristic kind of sound. And then we have low
frequency base drop. So I'm trying to show
that there's, like, maybe a threat coming
from the spaceship, so that base drop is going
to help with that mood. And then we can
say something like cinematic SFX just to
make it more interesting. I'm also going to do
another version where we don't have all these
adjectives and descriptions, so you can see how much
details can play a part here. So we can say spaceship
or just yeah, spaceship flying by, and then
we can do coma cinematic, something very simple,
generate that as well. And then we can compare the two. This is the first one
with the longer prompt. Low frequency drop.
I'll play it again. If I have a video where there's
a space shift flying by, this will go
perfectly with that. Then we have a simpler prompt and this is the
result from that. So it's still caught onto the spaceship idea,
but as you saw, it was a lot more loud and it sounded more mechanical
than our first example. So as long as you say that
main subject or object, it's going to understand
what you're going for. But adding in those little
details before and after your noun is really going to help with the way the
sound is portrayed. Once you have your audios, you can just download them
right here with this button. You can share them and
then reuse the prompts. Again, there's not a lot you can do with the audio
once you have it, but you can pair it with the video on a video
editing platform. The next audio tool that we
have is the stylized audio. This is where we get to turn an audio file into someone else, and by someone else, I
mean the models down here. You can either speak into the microphone right now
or record your audio separately uploaded
here and then easily mask your voice or that
voice with the option B. So I have a sample audio for
you guys to try out as well. I'm just going to
upload it right here. And once we have
that, we can now, you know, completely
change that voice. So this is a preview of
what it sounds like. I really don't know
what is going on. I think that we should get away from here as
fast as we can. So, obviously, we can choose another female voice
to mask the person, but we could also completely
change the age and the gender and the tone
using a different audio. So I will go for, you know, a different gender,
someone older, and let's try Billy for that. Not gonna lie. I am super
excited to be here. So let's try this
as you saw this is very different from
the original voice. Sell like that, and
then it generate. At the same time, we can go for female that is
very different, too. Hello, thanks for calling. Mi. So let's go with
Leslie. Generate that. 7 seconds, 7 seconds. Let's see what the
first one sounds like. I really don't know
what is going on. I think that we should get away from here as
fast as we can. So this is interestingly
made with 11 Labs, which is another very
popular platform for AI Audio generation. We do have a separate course for that if you guys
were interested, but now we can see that it's
integrated into Runway. So this is a nice collaboration. So that was the first audio. Let's look at our second one. I really don't know
what is going on. I think that we should get away from here as
fast as we can. So very clean, not a lot
of options for me to edit, but as we saw, there's
not a lot to edit. So it's very straightforward. You input your audio, choose the model, and
get your outputs. So we can clone our voice and generate one with
text description. So let's go ahead
and clone a voice, and then we'll see the
text generation option. Now here you can upload an audio or record
your own voice, put it into the tool. But one thing about uploading, make sure that you have the
right to upload something. So if you downloaded an
audio from the Internet, you cannot input it here unless
you have the permission. Make sure you have that
permission because if you just upload someone's audio to problems later on if
that person finds out, and also when I upload
something here, you can see that you have to agree that you have
all the rights. So make sure you do before
you hit the checkmark. I'm going to basically clone my own voice because you
have the right to do that, and I'm just going to
upload a audio sample. I'm only doing this because
I'm recording right now, but you can record straight into Runway and clone your
audio that way, too. So I just uploaded a
sample audio of myself, and we're going to hit generate. You guys know what I
sound like already, so it's going to be interesting to see what Runway
makes of that. So I'll say my voice for
tutorials, let's say. I have all the right
to my own voice, and we're going to add that. So right now when I play this is just the original
audio file playing. So to put it to the test, we can use one of the dialog
apps in our video section. So let's open a new tab. So I will use the
character script to video, and we're just going to choose our voice instead of my voice. I'll put in something
very simple. I'll describe the scene. There is a long rainbow
road leading to the sky. It is in the afternoon. So, this is my script. I just described what
I'm seeing here. Let's now select an asset
for the visual character. So I think I'll go
with a polar bear. That's gonna be me. We
have our script, my voice. Let's generate. Now,
while this is happening, we can check out
the other way that you can generate
your own voices. Now, the second option it's
just the text version. So you can describe someone
who is middle aged, female, high pitched, you know, these things that can
describe an audio. This is going to
need the Max plan. So if you're on P, you can just use the clone
a voice option. Let's see me as a polar bear. There is a long rainbow road leading into the afternoon sky. There is a So it kind
of sounds like me. I don't know why the bear
is speaking from its nose, but other than that, it did
clone my voice in a way. It's not identical,
which is good. And I can now have my voice portrayed by
different characters. Those are the two apps that
are offered by Runway. We do have a bunch of other third party tools that are integrated
inside here as well, but they're not by Runway.
That's the only difference. So that's why there's
a little break here. I'm not going to show
you every single one, but basically, you can go based off of what's
being said here. So if you want to do text to audio, you can click on this. And you can describe
any sound here, and using this model, you will get an output. And then you can
do music as well, I believe, text to music. We have different
models for that. We have 11 labs one more time, and we have text to
speech options as well. So you can use both
the Runway tools or these third party ones to generate any type of
audio that you want. I'm just going to show you
the text to music because that's a rather new
capability by these models. So let's just go into this one. And I will describe a sad Slow piano
introduction for a Nir film. That's my music, the
one I have envisioned. Let's generate it
and see what comes out from this very
simple prompt. Let's hear it. So that's a very good music coming out of my simple prompt. You can describe the different
instruments, the pacing, and if you wanted
to kind of duck at some point for like
a dialogue scene, you can also do that
within the prompt. And once you have
your audio files, you can then use them for
any dialog apps that we saw. So recall that you were
able to put in your script, choose a voice, or
upload an audio. So now that you know
the audio tools, you can then upload it into the video interfaces and get a final sequence by
combining the two.
14. Loops, Stitching & Export: By this point, you may
be wondering how you can finalize and stitch
videos together. So you have all of these clips, you use all the tools. But how do you get that final
product before exporting? We have two tools that are
going to help us do that. So when you go to Apps video, we have the option to
scroll all the way down, either create a loop or stitch
multiple videos together. So we're going to look
at the loop first, which is going to be the
image that you input. It will get animated for you
and then turned into a loop. Now, we're just
going to use one of our older images from
the previous lessons. We're going to have the
bear turn the head, turn its head
towards the camera, and then back to the left side. So bear turns head to
the camera and back. And that's it. So a
very brief description is enough to create your loop. We're also going to
try another example. So I'm going to let this
guy run, and real quick, I'm going to make
a product image, and then we can
create a loop that we could potentially use
for our social media ad. So let's go to Image and say product image of
a purple perfume, something that is different
with a studio background. Just like that. Let's see
what the options are. So here are the four
different bottles. I'm going to grab this guy
and basically create a loop where the camera kind of
rushes in and then zooms out, and that's going to be
something I could use. So let's turn this into the app. I can now go back here
and select that image. And we're going to
say camera zooms in and out into the
perfume. And that's it. So let's take a look
at our perfume loop. I finished generating. Okay. So not only did it do
the motion that I asked for, but it also generated the
audio, which you can hear. I'll do it again. It only had a slight hallucination
at the start. Basically, the perfume
melts into the container, but you can easily fix
that by, you know, giving a more detailed prompt or running the generation again. Now let's look at
our second tool, which is going to stitch
video clips together. The next tool allows you
to stitch videos together, and it's really helpful once you have videos produced from a Scene Builder or from the
cinematic Brainstorm app. So this is something
we covered in the Runway course made
for image generation. So if you use one of these, you will end up with a bunch of videos following that one story. But then we didn't
have the option to stitch those videos together. And so you would have to turn to this app right here to
stitch it into one. I'm just going to
show you what I made earlier in that course. So this is what we
made in that course. We use the Scene Builder, and then we generated a video sequence out
of those scenes. So it's basically a polar bear. We have one shot. This is the second shot and
then third shot, and then the fourth shot. So I have this video. It's great, but I want to
stitch another clip onto it where our polar
bear is enjoying Well, he's done with the
ice cream, basically. So I have this image, and I'm just going to bring it to life with the video app. So what we can do is
click on this right here and I'm going to
use it in video. So now we have that sequence. Let's actually flip
the other one. So the ice cream goes, maybe the close up is better. This is the first frame, and this is the last frame. I'm going to say Polo bear heats the ice cream and
does the peace sign. Let's generate the
video and then we're going to stitch it
to this sequence. I'm just going to download this and that way I could
bring it in easier. I will provide these two videos for you guys in
the resource pack. So you can download it and try the Stitch app with me as
I'm going to do it now. The second video is almost done. Let's scroll to stitch videos. And select the first sequence. You can either upload
your clip here or select it from
previous works. That's the first
thing being uploaded, and then we can grab the
second one once it's finished. Let's see. We can bring it
into that platform too. So there's my polar bear
enjoying his ice cream. I'm just going to
download this as well and bring it onto my
little timeline over here. Alright. So we have
two videos here. You can do a bunch of
other videos as well. If you want to reorder them, you just grab the side here, pull it down, or pull it up,
whichever you want to do. And to get rid of video, you can just click on the X. So let's stitch
these two together and finish our polar bear story. Alright, so let's see what
the final story looks like. And there we go. We have
two stitched clips, and I could continue building
on the story and just, you know, use the different
tools to get there. We also saw how we could add audio files
and generate them. So if you wanted to, you know, stitch bear sound to this video, you can use the Act two and
change the voice, basically. So I could go to the
SFX Builder and say bear noises and then download that file and grab
it right here. If our bear was talking, we could kind of use one
of these voices as well. But in my case, a
bunch of, like, grunts should be
enough for our bear. That's the Stitch tool. See how we can now
export our videos. Let's say this is
the final product, and I now want to share it
onto different platforms. So by default, you do have the share option on
all of your videos. If you click on this button, you can share it with a link. Share it with the workspace, which is within Runway.
If you have a team. Go with this guy and
share it on Whatsapp, Instagram, or any
other platform. But if you wanted to export, you would have to go
to the download side. So you can go for an MP
four or a gift file. You can give it a name, Bouvar story. And click Download. And that's it. That's how
you export in Runway. It doesn't matter what sort
of media type you have. The download and share
options are the exact same. So now we looked at almost every single
possibility within Runway. But one powerful thing is the new agent capability that
Runway allows you to do. So agents are going to be things that can do multiple
tasks at the same time. And because we have
all these apps, the agent can access
all of them at the same time and build that thing for you
from start to finish. So basically, you can build an agent for social media posts. You can build an agent for ads. You can build an agent
for content creation, and basically
design which app it uses for what part
of the process, and then you can manage those outputs and make
your life easier, really. So we're going to take a look at the agents in the next lesson, and we can see how much of an upgrade this is from
what we had before.
15. Agent, Workflows & Multi-Model Production: Now, when you enter
the homepage, we are immediately
met with a chat box. Now, this is not like
all the other apps because this is technically
you speaking to an agent. So here you just
simply put in an idea, you put in a thought, and it will help you get to
that final product. So it's not like
going over here where we know we want an
image-to-video, rather we start with a rough idea and
Runway helps us figure out which of the tools would be most suitable for our idea. So I'm just going
to say something. Let's say I want to build
a loop for my soda brand. That's my thought. I
could choose my model. There's faster responses,
deeper reasoning, and of course, advanced. You can choose the model
depending on what you're asking. I'm going to stick with 5.5 on it because it's
a very simple task. We can add references over here. We can add media as well. If we have something external, how we can bring
them here as well. I'm just going to
run this the way it is so that we can
see what happens. So immediately it looked
for my brand kit. Now, a brand kit is basically a unique identifying colors and media that would
present that brand. For example, if you think
about the brand KFC, there's always that mascot. It's always red,
white, bold letters. It's always about
fried chicken and those elements
that describe KFC. Is going to be part
of that brand kit. Now, here it's a
more visual kit. So that's mostly the
colors, the styling, color grading, and any sort of visual that would go with it. You can look at brand
kits over here. Going to open up that link. I have something for my brand. I think this was
built automatically. But basically, if
you have a business already where you have the logo and the colors and everything, you can create a new kit and then use that for
every generation. That's not what we're
going to focus on, and I'm going to
start from scratch. So it's asking me
some questions. So what should the loop show? I will go with a hero. Where will it play?
I'll be social media? Do I have any
logos? No, I don't. So I'm going to put
in those answers, and then this agent is going
to write the prompts for me, use the appropriate app, and give me the final product. This is the thought that it has. It's a long prompt,
and I could read this, look at the details here and either approve or
ask for an edit. I'm just going to
approve what it gave me, and then we can see
what the agent did from that very simple idea
that I had initially. So this is the image that
the agent decided for me. I could go about
making some edits, use this for a video, for a loop or anything
else, really. As you saw, I didn't have
to do a lot of work. I just said, This
is what I want. This is my idea and the
agent did everything for me. I have this bottle that I could now take
onto the next step, either from the agent chat or export it onto the
apps that we saw. Now, this is the agent chat. Let's look at the
workflows where we get to design a system
from start to finish. Let's go over here. We have two options when it comes
to creating workflows, and when I say workflow, we just mean that
perform this task, start from here and from here. That's basically
what workflows are. We can create with Agent. That's basically the chat
bot that we saw earlier, or we can do it manually
with a new workflow. So when we click on this, close this, we are
met with a canvas. So for this exercise, I'm going to do
something very simple. But once you know
how this works, you can really apply it to any sort of task that you have. So if you're a brand designer, you can do something
in regards to that. If you do video edits, can create a workflow
for that as well. So you're always
going to start from the plus because you're
going to need a node. So a node is going to be a point in this workflow
that pertains to a task. So let's create our first node. And because everything
starts with an idea, we're going to need a text box. So I'm just going to put
this here and I'll say, let's rename it. My rough idea. That's the name of this box, and we can put in
whatever idea we have. I will continue building
on that soda can idea. So now I have this node
that is for the rough idea, and the prompt is going
to be turn this idea into a prompt for
image generation. Make sure it's realistic and follows a product
photography style. This is my main prompt
because my task is going to be maybe to generate different product images
with the soda can. We're then going to put
in that rough idea. So an ad for a fun soda can. And that's it. That's my prompt. But now what happens to
this prompt because we don't have an AI model to
actually perform this? So what we can do now
is get our next node, which is a video or image node. So we're going to
need a language model to actually generate
that prompt. You can choose one
of the options. I'm going to go with Cloud, and we're basically going to situate this node
next to this one, feed it into text
prompt, just like that. Once I hit Run, let's
give our workflow a name. Let's say soda can adds. We are going to basically activate Cloud to write
this prompt for us. So let's have this do its thing. We do have some options here, so you can choose the model. If you know Cloud, you can basically decide easily. If you're not familiar, you can choose the default, and we do have a course
dedicated to Cloud as well. So that's the setting.
There is our prompt. So from that rough idea, we now have a nice prompt, but then what happens
to this prompt? So you have to think
in this workflow way. We want the prompt to be fed into an image generator model. That we're going to
get another node. But this time it's
going to be for image. You can also feed it into
video, but as we discussed, it's better to have
an image reference first. Choose your model. I'm going to go
with nano Banana, make sure it's text to
image, it supports it, and we're going to put this next to clot and we're going to feed the response into the prompt because we have text to image. Now, once we run this, it's going to
generate our image, and we can then
either input it to a video as an image
reference or do a different model variation and just continue
going up like that. So let's run this
with nano banana, and then we can
grab another one. I'll try a different
model just so we can see. I'll do GPT. I'm going to put this
underneath and basically grab another string and fed it
into this image model. Then we're going to run.
So I have one prompt, and I'm trying different
image generators. That's the first one. Let's zoom in here. Looks really nice. Let's see what GPT looks like. And basically, you can get an
idea of what a workflow is. This is a very simple example, but as I said, you can apply
it to whatever you want. So my first point, let's actually look at
this before we continue. So this is with another
model, GPT 2.5 flare. So I could compare the two and then whichever
I prefer more. I will escalate it
to the next step. Speaking of steps, you
would have to identify your start and finish so that you know
exactly where to go. So the start of my workflow
is just a rough idea. So I don't want to spend a lot of time writing the prompt. Instead, I want to be able to write basically one sentence
or something or one word. Then have Claude do the
rest of the work for me. Then the end of my workflow, I want basically a video ad that I could publish
onto social media. So to get there, I
would have to think of all the steps that would go
into a high quality video. By now, we know exactly
what those steps are. The main thing is the
strong image reference. So we have that right here. The next step is to feed
it to a video model. So I will look into one of
the models that I prefer. I think it was Gemini. Put that right here, and I'm going to choose the
one that I like more. So let's go with GPT, grab the edge here, put it
into reference image one. And then you can do
multiple things. So just to try this out, I'm going to do both images, input it as reference images for this model's video output. And I'm now going to run this. Now, this is going to
generate the video, just like we saw in the apps. So basically, this
is the same thing, but in a different environment. Then we can do another one. Let's see. If you want,
you could do an audio. But I think that's it actually. We have our video
and then that's it. At the end, we could
maybe try stitch videos, the note that we looked at last. I'm going to try a different
video model here as well, just so we have two
things to stitch. So let's try I'll
do link and then feed these into it
as image reference. Oops. I think I did
it the wrong way. So if you made a mistake, just click on that node
and hit Backspace. Then grab another one and
put it into the right place. So we have start frame and
then the reference images. I accidentally put it
into the start frame. Alright, so we have
our next model, I could also put in this prompt. Now that I have my video, I could do another
maybe Video to video model or image-to-video,
whichever works best. Let's try the models
that we like. I'll go with Gemini
and I'll do this one. Video to video. I'll grab this and put it into the video input right here. I have a reference
image already, so let's put that
in and then we can maybe get a prompt video prompt. Put that in like that. Now I have the clod
prompt all the way here. I have my reference image. One of them, I could
do the second one too, and then we have
this video as input. There's going to be
a lot of strings, but that's basically
what workflows are. I have my first video. Let's play it. If you
want to edit your videos, you would get another
prompt editor like Claude or Chachi BT. Make the change and then connect the notes as you go.
Alright, so I have this. I'm now going to feed
it in as my Video one, and then this will be
Video two once it's done, and then we can stitch them together for our final product. You're going to
get a better idea once you actually work with it. For now, just try
to follow along so that you could get
a similar output, and that could be
your first product out of the workflows. While this is loading, there are some tools down here, we have the pointer, which is what you use
to click on things, and then we have box select. You can grab the boxes like that with that and then
the hand that you see allows you to move the canvas. That's
what we're doing. Zoom in, you can
use the Zoom tool. Zoom out, fit to screen,
brings it all out, undo buttons, and then
keyboard shortcuts if you wanted to just
speed up your process. This was the ad note. This is some templates that you can look at if you wanted to. We have our uploads and
assets and then some labels. So if you're working
with someone, you could use these labels
or for your own, like, direction, and then put
something here, generated image. And that's that. So let's
look at our second video. So they're pretty similar, but now I'm just
gonna stitch them together and get
that final product. Because once I have this,
I could easily export it with download
video, and that's it. Oops. There is our
final product. First one was 5 seconds, 5 seconds, so the
total is 10 seconds. And that's it. So I
started from a rough idea. I put that into Cloud, got a better prompt, made my images, and then generated the videos with
the final stitching. So that's an introduction
to Workflows. It's really fun and really helpful if you're trying
to do a bigger task, and you don't want to
go from app to app, but rather do everything
in one place. Now, this workflow
is saved over here. It's always going to be
automatically saved. So you can come into this
again for a different project. And like I said, you
can download this for your use just by
clicking on that button. Now that we have all the
information that we need, it's now time to jump
into the project. Now we're going
to do the project together first,
one version of it, and then you guys are going
to do one project on your own and upload that to the
Class Project Gallery.
16. Build the Final Video: So let's apply all that we
have learned into a project. We're going to be building a short video from scratch
with at least two shots. So there's going to be
some planning involved. We're going to make
the image references, then turn them into motion, and then stitch them
together before exporting. The first thing we need
to do is figure out the concept or the
topic of our video. Who is going to be in it? Does it even have a subject? Is it just nature? Is it a animal? Is it a cartoon? You have to think about what it is that you're
trying to show, and then you can go about
thinking of the shots. The first thing you need is to decide whether you
want a subject. If yes, you need to build an
image of how they look like. If no, you can skip this step. I will be doing a
short video about my character who is climbing
this snowy mountain. It's really intense, they're
struggling a little bit. I need to figure out my subject, how they look like,
their gender, their facial features,
and all of that. To do that, I will
use my image tool, so just go to tool image. And since I'm starting
from scratch, I just have to describe what
my character looks like. So a rock climber in the
middle of a snowstorm, let's do a close up portrait of a rock climber
wearing a face mask. Let's say, the middle
of a snowstorm. If you have references,
you can add it, but I'm just going to keep
everything a minimal for this. And I'm going to go for
four different variations so that I could decide on
what my subject looks like. I'll do 16 to nine
and hit generate. Now, if you want to do
something different and not cinematic or realistic, you would have to
change your model. So if you do what I'm doing now, you're going to get a
realistic image like this. But if you want to do a cartoon, you would have to switch to the Runway tools and mention to the Illustration or
watercolor Illustration, whichever you're trying to do. Now I have a bunch of good examples of what my
subject could look like. I really like this image, and now I'm going to drop
it into my reference. Now I want to get another scene that is even closer to
the subject's eyes. So I'm trying to
show their struggle, but because they are covering their mouth because
they're in a storm, I want to show that
struggle through the eyes. So a close up of
the rock climbers Eyes showing struggle
through the snowstorm. I don't need to say cinematic or realistic because I have
this as a reference now. Same model. We're gonna
hit Generate again. I will be using
this in my video as well because that's a
different angle of my subject. So if you want, you can heard it or just keep it in
your asset folder. Alright, so we can see
it's the same subject. The red is there and
the hat is the same. If they are struggling, they would have to
squint their eyes. So these two are
a better option. I will go with this one. So this is one of my scenes
where we have the subject, and I have a half body shot
as well as a close up shot. Now I need to show the
other parts of my sequence. So because they are
climbing a mountain, I should show that mountain. So we're going to do another
generation from scratch. A long landscape shot of a tall, snowy mountain in a snowstorm. Let's generate. So here is
the setting for my subject. It's very intense, and
there is a snowstorm. So I think I will
go with this one. This is one of my shots. Now, because I said long shot, I have the space to do
some camera movement. Maybe the camera
could zoom in slowly. I could do some other stuff. But what I'm saying is
that there's a lot of space from the peak, which is where our
subject would be. I can do a bunch of
edits if I wanted to. You could stop here
with the two scenes. For my third shot, I'm thinking of showing the
beginning of this journey. So maybe the subject with a backpack about to go
towards this mountain. So we're going to put
the two as a reference, and I'm going to do show
subject in at Image one. So this subject. With a big backpack with
his back turned towards the camera place subject in front of the setting
in at Image two. So that's what I'm envisioning. All right, show distance between the subject
and the mountain. So basically, before he
started his journey. And once I have these, I could, you know,
further refine them. We have a lot of tools
available to us. But the next step is going to be generating that sequence. We have a bunch of options. I think I will go with
this one so we can show him walking
towards the mountain. But as you can see, it's the same subject and
the same mountain. So references are
really helpful here. So I have my three
image references. Let's go to the tools again, switch over to video. And describe the
different motions. So if we scroll down, we can grab our first scene, put it in as a reference. So subject walking towards
mountain in a snowstorm. That's it. Because I use gemini, I'm going to use the
same video model here. You can choose your settings. I'm just going to make sure it's landscape to be consistent
with my shots here. Let's generate that
first and then move over to our second scene, which is a close
up of our subject. Get rid of the first
image and now keep this. Camera slowly zooms into subject's face as they
struggle with the snowstorm. 16 to nine again. And then the final scene
is our big scary mountain. We're gonna do a slow zoom into the mountain with an
intense snowstorm and a darkening sky. So we're just showing
that things are only about to get worse
or our subject. Depending on what you
did with your concept, you may have to edit some
of these video outputs, and you can easily do
that by now dragging this video in here saying
what you want changed. So maybe the walking is
not up to your standard, and you want to do
something different. So let's take a look
at our first video, which is the subject walking
towards that scary mountain. So even though we have one shot, this model generated different
angles of our subject, which is a nice addition. So for me, I think this is okay. I mean, he's wearing
the red outfit. The mountain is the same. This is just another angle. The bag, you can see,
did not change as well. He's walking normally. And let's see, the
snow is there. Again, same backpack
and same outfit. So those are the things
that you would have to check when you're
reviewing your shots, make sure that your
character stays consistent and everything is according to your
reference input. So that is the first video. Let's look at the second one. So this is a good video, but as you can see, the
outfit is black and not red. So I need to fix that with
the edit option right here. But let's look at
our third video before we do that adjustment. So same mountain, the sky
is getting darker and, you know, things are not
looking good for our subject. So the two videos
are good to go. I'm just going to go into the
one that I want to change, and we're just gonna
hit Edit right here. So this is going to take
us to the Edit Studio. And I can now see where
the problem is because I think the reference that we input it, let's go over here. Yeah, it did not really specify the color of the
subject's outfit. So this is a issue on our part. But again, you can just go to the editing studio and
put in your reference. So I'm going to grab one where
his red outfit is visible. I'm just going to change
the aspect ratio. Again, I don't want
a random dimension. Let's see the different prompt versions
before we generate. So what I'm looking for is
only for his jacket to be red, the cap is okay, and I forgot to mention that the mask needs to be covered up, but maybe prompt the version will pick that up on its own. So this is not really ideal. I think what we could do instead is do another generation here. That would be the
most straightforward. Let's go into this video
and just reuse the prompt. And this time, I'm going
to put in this image. So camera slowly
zooms into subjects face in at Image one. Follow the outfit
in at Image two. Because we can reference
the images like this, sometimes it's a good idea
to come back in here. Alright, let's
generate our video, and hopefully this
is good to go. I'm going to delete this because
it's not really helpful. We don't need that. Okay, so we got a whole
different composition, but let's see what
it looks like. So I think this is actually
better because it shows him, you know, mid climbing. And again, the outfit
is really good. So I'm going to use this instead because it's showing me two things at the same time. So I got a little bonus here. So I have my three videos. We can now either proceed with them as they
are or we can get additional variations
where time of day changes or the weather changes
with the apps that we saw. So I'm going to go to video
and do a variation of our last scene where it's just the mountain and show
that it becomes nighttime. So maybe we're showing
that the subject got stuck on the mountain and it's
a very serious situation. So let's go to video, and we're going to go
to video time of day. We already have the weather, but if you did
something where you didn't include the snowstorm, you can use that app as well. I'm just going to grab my
second video right here. And we're going to
turn this to twilight, or maybe we could do
midnight. That's better. Alright, let's generate this. Now in this example, I'm
doing more than three shots. You don't have to do that. For your class project, you can do at least two, but try to keep it 2-3, so you can spend more
time on the editing, not so much on the
generating parts. But we're just going to
see what comes out of this little short
video together. So same footage. It's only nighttime. Alright, so we have our clips. It's now time to put them
together and polish them, maybe upscale them before we
export our final product. So make sure that you have your at least two
to three shots. Again, you don't have to
do all the things that I did here for your own project, but I hope that you were
able to follow along to see your options when it comes
to generating those shots. So let's go ahead
and see how we can put this all together
before exporting.
17. Final Export & Next Steps: So we have our shots, and before I stitch
them together, I want to first upscale them
and maybe for some of them, add some specific color grading. Now, you may be wondering
why can't you do this after you stitch the video so you don't deal with multiple shots. The reason for that is
because this tool currently does not support clips
more than 40 seconds. So if you stitch the
whole thing together, it may be like 45 seconds and you can't really
upscale it altogether. So I'm going to grab each of
my scenes, the correct ones. So let's start with
the first one. We're going to
upscale like that. And while this is
doing its thing, we can grab the second one. Second one, third one, Generate, and fourth one. So for upscaled footage, I'm thinking I may replace
my normal Mountain odio with this nighttime one
so I don't have to make my clip extra long. So I'll end up with three scenes instead of basically three
videos instead of four. All right. So we have
upscaled footage. They will all have
this four K label. So if I just go here,
you can see that. Now let's move on
to our next app, which is to Color grade. Some of our clips go
into Color grade video, and I'm going to
grab the new ones. So the last four are
the upscale footage. So this one is fine. It
has that nighttime thing, and I said, I'm going to exclude this and
keep this instead. So I have two videos to cigrate. The first one, we can kind
of maybe grab a preset. I think too intense. Maybe I'll go with midnight
blue and generate. So that way, we can show that the subject started
his journey around nighttime and I could add to the intensity of the situation.
So that's the first one. Let's grab the second video and do the same thing midnight
blue and generate. If you want to do
your own explanation, you can maybe mention
those words directly, like high contrast with a blue tint or a
subtle blue tint, just to have a little control
over the amount of blue. I'm just going to put this
preset on it and try to match that other clip that we have where it's, you
know, during the night. Hopefully, these will not be as intense as the blue
here, but somewhat blue. So this is what I
ended up doing. I only coli grated the second sequence and
kept the first one as it was just to show that
it took a while for him to get onto the mountain and
then it turned to nighttime. So let's hit Stitch
and get this going. So once we have our final video, we can easily export it by
clicking on the Download. So now you have all that you need to work with
your final project. So follow the exact
same workflow that we did in this lesson, starting with the idea
then the image references, generate your videos, edit them if you want with the
apps that you saw, put them together, upscale, and export your footage. So these lessons were
mostly for the video apps, but we also have another course dedicated to the image apps. We did use some of them
in the couple of lessons, but we have a whole
different class for the image tools right here. If you haven't already
visited that class, be sure to do that
because once you have the full knowledge from
both image and video, the possibilities
of your outcomes are going to be endless. By then you would
have all the tools you need within this platform. And from here on, you
would just have to continue practicing
with different models, different apps, combine them, and see what sort of ideas
you can bring to reality.
18. Class Project: Create Your Own Short AI Video: Now it's time to take everything that you
have learned and create your own short
AI video using Runway. For your class project, you're going to be creating a sequence that has two
to three connected shots. Start with a simple idea. This could be an ad,
a cinematic scene, a talking video, or something
else that you prefer. Keep the concept focused so you can spend more
time on refining your shots rather
than dealing with a lot of randomly
generated footage. And speaking of a
lot of footage, make sure that you're planning every single one of your clips. Before you even hit
that generate button, you need to know what
your subject is doing, the camera direction,
the lighting, and how this scene is going
to connect to the next. Then you're going to generate your footage using
text-to-video, image-to-video, keyframes, or any other workflow
that works best for you. Once you have all of your clips, put them together using the editing tools
that we covered. You can change the environment, change the color,
add some characters, sound effects or music. While you're finalizing
your sequence, look out for pacing, timing, and if there's
any need for upscaling. When you're done,
export and share your final product to the
Class Project Gallery. You can include a few of your still frames alongside an explanation
about your process, what your challenges were, and what your general
concept is about. Keep it short and focus on a few shots that would
work well together. And most importantly, make sure that you're having
fun along the way. Good luck. And I look forward to seeing what your
videos look like.
19. Congratulations! What’s Next?: Congratulations. You have made it to the end of the course. We started by
learning how to think differently when it comes
to generating video, including movement,
camera direction, action, timing, and pacing. From there, you created your videos using
text and images, connected your shots, and used keyframes for more
controlled movements. You also learned
how to transform existing footage from changing the scenes and environments to stylizing and color
grading your videos to even learning how to upscale
your videos when necessary. We also experimented with character movement
and performances, added some sound
effects and audio, created loops, and stitched
a few videos together. Most importantly, you have
gone through the process of turning an idea into a
final video using AI. From here, you need to
keep on practicing. Try different camera angles,
movements, new characters, and see how you can use each of these tools to create
your next video idea. If you haven't already, be sure to upload
your final video to the Class Project Gallery. I would love to
see what you guys came up with and what
your ideas were. And if you enjoyed the course, feel free to drop us a review. We really appreciate
all your feedback. Thanks for watching, and I'll see you guys in our next course.