How to Dockerize Your Python Applications

1. Course Intro: Hey there, it's max and welcome to my course on how to darker eyes your Python applications. Now, as any software developer knows, darker plays a very important part in your software development career because it lets you feel confident that whatever you have locally is actually the same thing. And it's going to run the same way on a different machine as well, also in the Cloud and in a production environment. In addition, it's also going to provide you with a lot of really nice tools to easily spin up different images so that you have all sorts of software easily available. But we wanna be able to make sure that we use darker properly. And so in this course we're going to learn the basics of how to write your first Docker file, how to do a multi-stage build, and also how to use docker-compose so that you can take your Python application or really whatever other code there would be. But in this case we're just focusing on Python and actually run that using docker building the Docker image and then running the Docker container, both through the regular Docker commands as well as also over Docker Compose. Now this course is actually part of a bigger Python series. We will learn how to build out a full Python application from scratch. But in this case, we're just going to focus only on the darker part. And so the code that we're gonna be using is from this bigger Python series. And you'll actually be able to access that using the GitHub link below in the course description. However, you can obviously also just use your own Python code for the same purposes. Or you can use the code that's already on the GitHub if you'd like to. Either way, you'll also find each of these lessons as different GitHub branches and that GitHub repository below. So if you want to follow along with the code, make sure to check that out there too. So without any further ado, I'll see you inside and happy learning. 2. Docker Intro: Alright, so in this section, we're going to start learning about Docker and we're going to integrate darker into our application. Now, Dr. is a really cool program and I'd actually recommend you just Google download docker or just go to this URL here, docker.com slash products slash Docker Desktop. And for me, because I'm on a Mac, it's going to give me the Mac options. But if you're on a Windows or Linux, it will default to those and just make sure you go ahead and download that because it's a couple of hundred megabytes. So it may take a couple of minutes to download. I've already gone ahead and done this. So yeah. For Docker, docker is really nice because it helps us solve another type of problem that we already started solving when we were talking about virtual environments. So if you remember back to virtual environments, or main motivation was that we want to make sure we have everything consistent. And in some sense a little bit isolated for all of the Python packages that we need. So e.g. if our application needs fast API to run, we want to make sure that everyone who tries to run our code has FastAPI installed as well as all the other things that they also need to run this application. And that it's not telling them like, Oh hey, you're missing this installation, you're missing that installation, or they're trying to use the local installations. And then there's a version mismatch, e.g. we're using some version like I don't know, like ten point whatever. And they're using version like nine point whatever. And there's a version mismatch. So all of those things are things that we've tried to address using virtual environments. And it's great and it's been extremely useful. But there's actually more that we can do to this to make this even more consistent. And the idea here is that with darker, we can create isolated Containers. So essentially, just think of it like a box that we can build up from scratch, including the operating system. And we know that no matter where this will run, if we give it the proper instructions, it will look the same everywhere. And now you may be wondering, why would we want to do that sounds a bit like overkill. Well, actually, there are certain scenarios and I've encountered this actually myself several times where you may install something on a Mac and it may not work properly, or it may work differently on a Linux, or things are working on a Linux one way they work in a different way and then they work on Windows like a third way. Or you also have local things already installed that you may have forgotten because you've installed them like six months ago or a year or two years ago. And then someone else tries to run your code and they're like, Hey, these things aren't working. And it's just because there are some things that you installed that you forgot about that they don't have. And so using Docker helps us address all of these problems. And there's actually a lot more cool stuff that we can do with it that we'll look at down the line. But we're going to focus right now. First off, just using Docker to essentially create an extremely clean box that contains our application that no matter where you run, it will be consistent. And actually Docker is used worldwide that when you're deploying code, you will usually be deploying that code as a Docker image that will be run somewhere. So yeah, darker, very important for that to really help us just get everything consistent from the base up, from having the same operating system that's running at the same versions of R programming language, the same, and then using our virtual environments to install consistent packages. And so we just want to make sure that everything is the same no matter where you run it and darker is going to allow us to do that. And it's extremely, extremely useful because then you don't ever have to worry about oh, was there something I forgot that also needs to be installed that I have installed or is there some environment variable that I set that I may have forgotten about? Or, you know, all of these sorts of weird oddities that can come up. And then especially dealing with problems like, Whoa, why does it not work on your machine? It's working fine on my machine. Did you install the virtual environment stuff correctly? Yeah, you did. Okay. That's weird. Like why is it running on my machine and on at yours? Darker is going to help us alleviate all these problems. And then as we'll see later on, there are actually some more cool things that we can do that will make your life a lot easier with Docker, e.g. also like spinning up a database that we can use or spinning up a cash that we can use. And a lot of cool stuff to come. But we'll focus on just isolating our application first so that we can just give this image to someone and then they can run it and they will get the exact same thing that we're seeing on our end. So yeah, with that, I would recommend just Google, download docker Desktop and your favorite search engine or go to this website here, which may change with time, probably not, but docker.com slash products slash Docker Desktop. And then just go ahead and download the darker version for you. Again, I already did that. Then for me, when I unpack the image, all I have to do is just drag it from here into my applications. Once you've done that, go ahead and go into your applications. And here now you want to double-click on darker and just get it to run for me because I've already run it before. It's just going to start running. And you're actually see if you're on a Mac and the top-right corner, there's going to be this little ship that looks like this, which is going to start bouncing around on a Windows. We'll go through that in a second. But yeah, wait for this to open. It's the first time you're going to use it. It's going to ask you to read and accept the terms and conditions before the application will start. So just whenever that pop-up comes up, just make sure you go ahead and do that. And again on a Mac in the top-right corner, usually in your toolbar up there, which is currently off my screen, you should see a whale like this with the containers circulating, which means it was starting. And once it kind of takes a fixed shape, that means it's started properly. And after you have accepted the terms and conditions and the startup is starting, you should be able to go to your PyCharm. Now if you just type darker into here, you should see some output here which will indicate that the installation has gone successfully. If you have any problems with this, the first thing I'd recommend is just first of all, wait for your Docker to finish opening. And if it's still having problems, try to restart your PC, you shouldn't need to, but it's generally a good thing to try. And then when you type darker in here, you should see that it responds to some sort of output. We have here. For the Windows side of things. We're going to be on the same page and we can see here I'm already getting recommended the Windows version. Go ahead and click Download on this, which I've already gone ahead and done because it takes a couple of minutes to download. Then you're going to open up the installer. It's going to prompt you, if you want to open this, you're going to click Yes. And then we're going to open up the installer or there's gonna be two checkboxes that are going to be by default checked. Just leave those and click Next. Then the installation is going to start, which again takes a couple of minutes. So I'll just kinda let it run in the background. And this is the state that you're going to reach where it will say the installation has succeeded. And then we're just going to click close and restart. So we're going to restart our machine for this. So after restarting on the Windows side of things, we're also going to get prompted with this pop-up here. I'm going to accept the terms and just continue with that. And now Docker is going to open in case you get this pop-up to. We're just going to follow some more instructions here to get the installation process. So let's just quickly go ahead and do that. Move this out of the way. Follow the instructions here. Donald, this here, real quick. Open execute and just go through this process. And then I already have PyCharm open here. I'm going to take this and I'm just going to copy it. Go into PyCharm which already has the PowerShell open. Paste this in there. Just going to hit Restart one more time. And after hitting a restart here, if we go on to our Docker, we should now be seeing that Docker Desktop is starting to open this application. If we go to the bottom right, then also there, which for me is currently off screen. But there are also you should be able to see if you click on this little windows arrow that you always have, that you have this darker ship thing here. And then you have the containers that should be moving. And also there. If you click on it, then it should take you back to the Docker application like we have here. Okay, so now going into our PyCharm, if we hit or if we type darker in here. And then also from the Windows side, this command should now be registered with something and we'll see you in a second. Now we have output associated to it. So this is how we're going to go about the installation process on Windows. 3. Creating a Docker file: Alright, so now that we've got Docker installed, let's go ahead and actually use Docker so that we can run our application over it. Now, before we jump into that, I quickly want to show you just in general, where we're going to get these base operating systems are these base pre-installed packages from that we're going to use. So to do that, you can head to hub.docker.com. Then you're going to need to create an account here. So if you don't have one, just go ahead and sign up for that. You can sign in and you'll get to a page that probably looks something like this. And you can either head to the Explore section if you'd like. And here we can already see a list of different kind of base systems that are available to us. These may or may not mean anything to you. And we can see here python is one of those. So we can either click directly into this or if we're looking for something specific. We could also search up here, e.g. Python. And we're gonna be using a version of this. And when we see in here, we can see that there are essentially like tags associated to them. And we'll see in a second how we can use these. But all of these are essentially different versions that we have access to that we can use. And the importance of using tags is it provides a sense of consistency. If we use a tag, we're able to consistently kind of reuse the same thing. And that way, even if like, upgrades are put out or something like that, we know that our system is not going to change. It's always gonna be running under the exact same conditions, which is obviously so important for us. So for us, I'm gonna be using the Python here. But just for your information that if you ever need to use a different base or images than if you're not sure what exactly you're looking for, a quick Google or using your search engine of choice, we'll probably very quickly pointing you in or direction, but otherwise you can also just browse what images are available to the public in one spot, just go into the Docker Hub. And you'll see here e.g. for Python, this is an official image and that's usually good tag that you want to look for. Since those are generally the more trustworthy ones. Since this isn't going to be forming the base. And so you wanna make sure that what you have installed is like a proper version and not just made by some random person that may be a little less trustworthy. Although the green line, the gray line at that area, also becomes a little bit questionable what, what is trustworthy and what is not. But generally as a good indicator, the first thing is look for the official images. And when it's not, well, it doesn't have that tag. Just make sure you understand what you're using. Since it's what you're gonna be running your application on. So just being aware of that from a security perspective. But that being said, let's go back to our application. And what I'm going to do in here, I'm going to create a new file. And I'm going to call this docker file like this uppercase D Docker file. Just going to hit Enter. And now in here we're going to create our Docker file. So the first thing that we need to do is we need to specify what should form the base of our system. So essentially the way that we will go about creating these Docker containers, which you can think about. We are shipping these well-structured, packaged, isolated Containers, almost like an a shipyard. In each one of these containers will contain, in our case, like an application, but it can really contain anything that is in. And it sounds like self-sufficient, or at least has everything included that needs to be included within the single package. So the first thing that we need to do is we need to define what our base is going to be in. And so we need some sort of like operating system-based. But generally, if we just start from an operating system, it becomes annoying because then we need to install all the extra Python dependencies to make sure that we can run Python. And so much easier thing for us to do is rather than just starting out like starting at some basic operating system and building up from there, we can go a couple of steps further and we can start with an operating system that has Python installed. And then all we have to do is installed all of the stuff that we need and essentially run our application. So that's what we're gonna do. We're gonna start off with this from keyword, which is going to define where we want to start off with. I'm going to be using the Python image. And specifically I'm going to be using the tag three-point nine, which as you may have guessed, indicates that this is for version 3.9 for Python. And then I'm also going to add another dash here and do the Alpine image. Now, the alpine image is generally a nice thing to start building off of because it's relatively small. The thing that you don't really want is you don't want to install a massive operating system that has a bunch of things installed, but you're not going to be using a lot of them. So generally the alpine image itself with the Python will be enough for your use cases. But if it ever is not, There's another thing called slim buster which you can try out. But alternatively, a quick Google or a quick using your favorite search engine, whatever that may be, will very quickly point you into the right direction of what may be a better image to use for this. But for most cases, the Alpine image will be sufficient for you. And again, the benefit that we get from it is that we're not just installing a bunch of stuff and that our container becomes very big. But rather we want to keep it as small as possible so that we're not using up a lot of our system resources just to have all these things installed that we're not even using. So we're going to start from the Python image using the tag 39 dash Alpine, which again is a Python 3.9 version and comes with all of that and the dependencies pre-installed. And we're using the Alpine version here since this one is relatively small. Alright, so this is the first command that we have, which is the front command. Now we're going to look at the next command, which is going to be a run command. So here we can use these commands to initiate certain steps. And the way that we go about things and Docker is every single command that we have everywhere, every single keyword that you're using here, you can imagine you're adding another layer onto your image. So we have our first layer here. Now when we have our first keyword here, run, which we'll see in a second. We're going to form our second layer. We have another keyword here, let's say another run. I'm just going to form our third layer. And with this we're kinda building layer upon layer upon layer. Now the reason that this is nice because Docker actually also has some form of caching. So it will recognize if you haven't made any changes for like half of your stuff. And really only the last step is the one that change, then it can reuse most of the stuff from previously and it only needs to change the last layer. So keeping that in mind as a very important thing, just because when you're building your Docker images, really the things that barely ever change, you want to put at the very top, and the things that can change more regularly, e.g. the packages that you need to install or also just your application code. All of that should be more towards the bottom. Alright, so I'm starting with this run command though. Here we can tell Docker to run a specific command and you'll recognize this. We're going to run pip, install pip. And so we're gonna be running this command here, which you probably recognize from an earlier lesson, where we're going to be installing pip end using our PIP. And we have PIP available because we're installing this Python base image, which already has Python as well as the things that we need for it or all of it is pre-installed. Python version here is gonna be 3.9 again, and we also have the pipa man command that immediately comes with it. So yeah, we're gonna be installing R like this. Now another thing that we're going to do is we're going to create a new directory. We're going to create a new folder where we can keep all of our stuff. Because ultimately we're creating this container and we want to put everything that we need in one location so that it's available for us just like we have it here in our project file, e.g. and so to do this, we're going to be using this MK dir for make directory. And we're going to have this minus p, which essentially allows us to create multiple folders at once. And then we're going to just use this path user SRC slash app. And this is where we're going to be putting all of our code for our application. Now, this stuff, I don't want to get into too much detail with all the Linux commands because there's actually a separate course about that. So don't want to derail off a little bit too much here and just really focus on this. But essentially keeping in mind here, and you can just reuse the syntax is we're just creating this directory here, and we're creating these folders. So in the user folder, we have the SRC folder, and in there we have the application. And if any of these don't exist, we're going to be creating it. So we're doing this again, just so that we have a folder that we can put our stuff into. The next thing that we're gonna do is we're going to use a new command called loops work. And this is going to set our current working directory. This is going to help us because rather than always having to type this everywhere, whenever we want to run something or in a second we'll see copy something over. We can just say using our current location. So it's just going to make it a little bit easier for us so that we don't have to repeat this year the whole time. So we're gonna be selling it to this user SRC app. This is going to be our working directory or the folder that we're currently working in, that everything should be relative to alright. So now we have pip installed, we have the folder that we're going to put our stuff into. And we're also going to be just setting a relative location to this. Now what we need to do is we have access to our files, but we want to copy them into our image and put them in the right location. We're going to use this copy command. And now we need to look at what we have here. And essentially we want to now copy this. And we want to have the same structure for the most part here, except maybe getting rid of some of the junk that we don't need. So we're gonna do this in two steps. We're going to start with the pip file lock and the pip file, as well as this main.py. Since these are the ones that are more outside, they're, they're kinda one level above. So we're gonna be copying these over. So we're going to copy our pip file, our pip file dot loc. And we're also going to be copying over our main.py. And we're just going to use this because rather than having to write out the full path user SRC app where we want to copy it into, because we've set our working directory, we can just use the relative path, which means copied into the current working directory. So that's going to be copying, taking this, and it's going to be copying that over for us. Alright, so now that we have these files copied over, now we want to copy over everything in our application folder here. Now unfortunately, we can't just add our app here because it's going to take all the contents from inside and it's just going to copy it onto the same layer, which is not what we want. So we're going to add a second copy step here. And we want to copy the contents of our app into essentially an app folder. So what I'm gonna do is I'm just going to create a, another app folder within here. So our folder structure is going to be your top folder is the usr, then we're going to sRC. Then we have the app folder, and inside here we have another app folder. So you can imagine this by just this folder name instead of being offers, buy them, project would be up and we have app here and app here. And now I wanted to copy the contents of this folder, which is app, into the app folder found within here. Alright, so we've copied over our first level stuff here. It's going to be on the same layer. And now I want to make sure that all the contents and here are also in this format. And again, if I just used this, that is going to take the contents from here and just going to copy them onto the same level, which is obviously not the current structure that we have. To do that I'm just going to add another folder here that we can copy that into. Alright, so now that we have this, let's go ahead and for now, just learn how we can start our image and look at our current structure to make sure that that is okay. And then what we're gonna do is we're going to finish with the rest of the stuff which is going to be installing using the pip install and also actually running our server. But for now, let's just look at our current structure and see that it's proper. And for this, I'm going to use the ls minus l, which if you're on a Mac or Linux will, should work for you. I don t think this will work on a Windows, but that's okay because it will work in here because we're gonna be using a Linux based version anyway. But you can see what this command does is it prints out the contents contained within the current folder that I'm in. So essentially we can see here the contents here are exactly like the contents listed here. So I'm just going to run this for now, just so that we can get a look into our image that we're building here to make sure that our structure is correct. Alright, so now that we have our Docker file defined, which defines the set of steps that we're going to take, starting from a base image and copying all of our stuff over to having this image that contains all of the stuff that we need so that we can actually run our application. It doesn't have everything yet, but we're getting there. So now we have to do is now that we have this image to find here, we need to build this image. So we didn't need to run these commands and put it all into a ready package that we can just run. And so to do this, we're going to use the command called Docker build. And now we're going to use a minus t here, which is going to be the target that we want to copy this into, the name that we want to associate to this. And so here I'm just going to call this my first app. And I'm going to give the tag twist, latest. Now, latest is a tag that you'll very often see, which is. Just indicates that this is the latest version of this image. Now usually what you'll have is you'll have a tag associated to each image, and then you also have a latest tag. So that way if you just want to run the latest version, you can use the latest tag, which will probably change with time if updates are being applied. So that way you are guaranteed to always be on the latest version. Or you could also reference a specific tag, like we're doing in this case here, which means that even when more than 3.10 is ours, so it's 3.11, 13.12, 13 if that will come out or when price and four will come out, that's not going to impact us because we're still going to be sticking with specific image that is labeled with this tag here. So it guarantees consistency for us that even if things around a change, we're always going to be using the same base. So obviously there are pros and cons to each. With this, you can easily stay up with the latest version whenever you're starting it or rebuilding it. But obviously they're also caveats because if things change, they may become incompatible and then your stuff may no longer run, which is obviously not good. So keeping that in mind, now we have the target that we want to say, build this into. Now we just need to give the location of our Docker file, which I'm just going to put the dot here, which is just going to indicate it's in the current folder where we currently are at. I'm going to hit enter. Now for you, this is probably going to take a little bit of time since you need to first download this image. And then you're going to run all these commands on top of it. For me, a lot of the stuff has already been cashed. And you can see here, if we scroll up, a lot of these steps have been cashed. And so there's no real downloading being done for me. And at the building process actually goes very, very quickly as we can see. Once you've run this the first time, if you run it again, then also it will go very quickly. Because again, these layers are cached. So now that we have our image created, now we want to run it. So we have this package that's ready and now we want to run it. And so to do that, we're going to use the docker run command. And we're going to provide now the name as well as the tag that we want to run. So we want to run this image here. And I'm also going to assign a name to it. So we're going to be running this, and this is going to be running as a container. We're gonna be running this image here as a container. And we're going to give a name to this running container so that we can better see it. That we can, when we're looking at which things are running, which we will look at in a second. We will know, okay, this is, this is not one. So we can kind of give a readable name to it. So we'll call this our maybe underscore first application. You don't need the underscores, but I'm going to use them. And yeah, just hit enter here. And we're going to be running this and we can see it executed this command here, which is just listing out the contents. And the structure that we have is pip file. We have pip file dot loc. We have the app folder, which inside should contain all of this stuff. And we have the main.py. This is what our current folder looks like that we have. Now this is nice because we don't have the tests e.g. because we don't need the test if we're running our image or for running this code as an application. Because by then, or our tests I've run and we no longer need the tests there because we're not gonna be running the tests as a live application. We're gonna be running the test when we're testing locally, we're gonna be running the tests later on when we're, before we build our image. Once our image is built, we no longer need the tests. And so we can discard that and say, save ourselves just a little bit of space here too. So now that we have this running, Let's go ahead and continue with our setup here. Because really we don't just want to print out the contents and then be over and done with it. But rather we want to have a running application that we can actually use. To do that, we still need to take a couple of steps. The first thing that we need to do is we need to run our pip install so that we're installing all of the dependency that we need. So we're going to use our pip install. We're not going to be using the def tag because this is going to be our production image essentially. So we don't want to install the development dependencies because we don't need these. We're going to just be a pure application. However, we're going to use a new tag that's going to be dash, dash system. Now this is going to install everything that we're using system-wide so that one we're running it or when we're trying to access it, we don't have to go into our virtual environment, but it's just gonna be installed for a whole system, which is going to make it a little bit easier for us because then we don't have to use the PIP and command's later on again, because essentially what we have in this whole image here is one virtual environment. We're recreating this every single time and there's nothing else going into it. So we can install this system-wide because we know exactly what it is, because we're building everything. From scratch. So we're gonna be installing our dependencies like we usually do. And now what we need to do is we actually need to run our application. So we'll use the CMD command again for this. And we're going to open and close parentheses. And each of the entries are each of the elements that we'll have here is going to be one of the arguments that we're going to run. So e.g. usually when we run unicorn main dash app, which you can actually run like this. But the preferred way of writing it would be like this. So if we just do this, this is going to be the usual command that we use, Except Usually we also have this reload, whoops tag. We do not want to use this in production because this will heavily slowdown or application. So make sure that you don't use this tag in production. When we're running it locally, it's fine. It makes the process a lot easier. But when I'm running it in production, you no longer want to use this tag. So right now we can just keep it like this if we want to, we can try it out. But there are still some issues that we'll see in a second with this. But let's get to those when we get to those. So what I'm going to do is I'm going to again rebuild my image and then go into my first app, latest. So I'm gonna be overriding that tag using the docker build command. And whoops, forgot the minus t here. So in this case it's going to be taking a little bit longer because now we actually need to run this pip install system. So previously we didn't run this, but now we are running it. And so it needs to actually run this process that needs to download, install all the dependencies until this time it's going to be a little bit slower, but we can see all the previous steps here we're actually cached and so they go very, very quickly. And once this is done, which should be done in just a second. Yeah, there we go. So now that we have this spilt, we can go docker run. And we're gonna give our name, which is going to be my first app, latest. And let's reuse the same name, which is going to be our first application. We want to give the name tag here. We're gonna be running into an issue here because it's complaining that this name is already in use. And we'll get to this problem in a second. But for now, let's just use a different name, e.g. like this. This name is now different than this one. So it should be fine. But we'll get to this issue in just a second. So let's go ahead and run this. And this is going to be, we can see here the command that we've usually seen, which is our application is starting up. But now here's the thing. If we try to access it, we click on here. We don't actually have access to it. Which obviously sucks because ideally, even locally, we wanna be able to actually use Docker to do our development. And that way we can be extremely sure that what we have running locally on our machine is exactly what's going to be running in the Cloud later on. We're not going to run into any issues of things not being properly installed, things not being available. So that's what we're going to look at now. Alright, so I'm hit here, control C to kill it. And now what I'm gonna do is first, I'm actually going to use this dash, dash host tag. So rather than running it on 127 dot zero dot, dot one, instead, I'm going to run it on 0000, which just means that this will accept any interface that were coming from, which just makes it more general, which is again, nice because we don't want to specify a specific address, but essentially we just want to make it, as long as we go here, we want to have access to it. So we're gonna be using this host instead. And now also what we're gonna do is we're going to define a specific port that we want to use for consistency. So I'm going to be using port 80, 80 here, e.g. but you can use different ones too. But for this case I'm gonna be using port 80, 80. And now if we rebuild our image, just going to scroll up and rebuild and wait for this process to finish. You see most of our layers are cached because the only change that we made was actually here. Everything else has already been done. And so we essentially reload all those steps from cash. And now I'm going to just assign a new name for now, since we have that name collision previously, which we'll look at in a second and hit Run. Alright, so now we can see here are running on this host and we're running on this port. But still, if I go here, we still can't access it. And that's because our application is running within this container and it currently doesn't have access to the outside world, even on our own machine. So what we need to do is in our docker run command, I'm going to use the minus P flag. And it's going to allow us to do port forwarding. When we go from our machine and we look at it from our port, it's going to forward it to a specific port in our running container, e.g. I'm going to say forward port 808-02-8080 in the container. And this way now, when I run this name already exists, you have to change this name for now. When I go here. Now it's going to be available. And we can see if we go to the docs page, will have the docs available for us because we're on port 80. 80, which actually now gets forwarded to the container port. Now, to show you this again, let's use some different numbers here. Let's use port 8,000, which we're going to forward to the, whoops, that's not what I wanted. 8,000, which is gonna be forwarded to port 80, 80 and our container. So remember, in our container application is running on port 80, 80, but outside we're gonna be forwarding it from port 8,000. So if I run this, I adopted the name again. I'm running it. If I click on here, now we're going to have an issue because there's nothing running on port 80. 80. But if I use port 8,000, then we're gonna be forwarded to port 80, 80 inside our application. So this isn't important. Oops, show the command again. This is an important step that we need to take because we're gonna be forwarding what we have on our machine, actually allowing us to go into the container. Alright, so now let's get to this naming issue here. If we use this command docker ps, we can actually look at all current running containers, which currently there are none. But if I run this one again, the name again. Alright, if I run this one again, I open up a new terminal here. And just wait for this to finish preparing. I'm going to go out of my virtual environment. Now if I use the command again, docker ps, we can see here we have the one running container. We can see here this is the command that's being run inside. This is the port forwarding that we have going on. This is the name of the running container. When it was created. The Stata is we have to add the image that it's running. As soon as we kill. It. Can see here now it's also gone. But if we want to look at all containers that are available, and we'll go back to our previous terminal. We can use this docker, ps dash, dash all, which will show us all the containers we have. So we can see here, this is the first one that we created. Then we have the second one, the third one, the fourth one, fifth 161. And so what we can do is if we have these old containers, rather than just sitting there taking up space, we can actually delete them if we want to, which we can do using docker remove. And then we're going to specify the container ID here. So I'm going to hit Enter and it's going to remove that so we can see if I print it out again. This one is now gone. And I'm gonna do this for this one too. And let's also do it. Let's print out these ones again here. And to make this a little bit quicker, Let's remove multiple. There we go. Printing those out. Now we can see we only have our application five leftover, which also is no longer running. So if we want to delete this one, we also can docker remove. So now we have no containers and AlphaGo back to e.g. or run command. We can reuse the same name because there is no container that's currently associated to it. Now again, it's going to be working. There are still some other commands here though that you'll probably be using one of them. And what I'm gonna do here is I'm just going to chain two commands together. So I'm going to actually, first I need to get its container ID and I'm going to remove it. Alright, so we'll look at that in a second. But first, one thing that will probably be doing is what we don't want is we don't want our terminal to be blocked by this because we still want to be able to do other stuff. So there's a flight that we can use, which is this minus d, which is going to run it as a daemon process, which means it's just gonna be running in the background. It's not going to be taking out this foreground terminal space. So if we run this, we can see here now if we do docker ps, we have a running container. And if we go to our web browser, I hit refresh here, then we can see. That our application is in fact running. These were the other ones, so I'm just going to close these. This one should also not be running because it's thorough important. So now let's just kinda running in the background. So now the question might be, how do we stop it from running previously we could hit Control C. What we can do now is we can use the command docker, kill and give the container ID, hit Enter. And now if we try it again, our application is no longer running. We can see here no running containers. And still have the container available though. So I'm going to remove this one. Like this. We can also do this using the desktop UI if we want to. So if we pull this one up here and we now run our container like this, we go back to the UI. We can see here now we have a running container. If we want to, we can click into it. We can actually see the output here. There's some stuff that you can play around with if you'd like to. Whenever we're ready. I mean, we've seen how we can just do this over the terminal. If you do want to use the interface, if you want to stop it, e.g. you can hit Stop here, which is going to stop it from running. So if you have docker ps, we can see there are no running containers or do docker ps minus, minus all. See the container still available though? And if we hit Delete here, e.g. then we're also going to be deleting our container. So this was our first look at darker. And I know we've covered a lot. But we've gotten to a really nice point because now we can actually run all of our code well, except for making changes at which point where we still have to rebuild. But we can really run our application over a Docker image, which is awesome because this is going to be the exact specifications that is going to be running in when it's running in a Cloud environment somewhere. And so we know whatever is running here is essentially how it's going to be functioning when it's running somewhere else. Which is extremely great because now we have a lot of consistency in our system and we don't have to worry about how k, but it runs on my machine. Will it really run somewhere else? If we're using Docker images, then we know that even if we want to give this to a colleague, they can download it. They can build the Docker image that can run it. And they're gonna be seeing the exact same things that we have. Now. Obviously, darker is a very cool software and there's a lot of stuff that comes with it. And we're not going to cover all of them. We're also going to be looking into Docker Compose. So there are some things that we can do in darker that we're not going to be looking into here because some of the things we'll be doing using Docker compose, e.g. but even with that, obviously there's a lot of nitty-gritty stuff that you can get into. Some really fancy and complex things to really have a cool system. But for most, almost all intents and purposes, what we're doing now is essentially what you're going to be needing them or you're gonna be doing when you're actually building and running applications in accompany. And if you do ever need specific things are run into specific issues than Google or whatever your favorite search engine is, is probably just going to help you out the quickest because you know exactly what you're looking for, the base things of what you can do and then you can just quickly, There's some stuff around. But yeah, darker again, is very, very important, very, very common thing. All of my code that I'm running essentially will be running with darker and some way because it's just so nice to have that consistency of having this image that we can just deploy somewhere and I know exactly how it's going to work. And you can give it to a colleague and they can run it on their machine and you know that whatever they have is going to be the exact same setup that you have. And so it's just really nice because it allows for consistency and reproducibility. 4. Multi-stage build: So previously we've seen how we can create our Docker images. Now in this lesson we're going to go over some best practices and adaptation is you can do to this Docker image that we're creating with this Docker image template that we're creating to build a container off of. And essentially this is stuff that you'll very likely see used in many, many places. And just in general. First of all, as a quick side note, you'll probably see if you're using PyCharm that you're gonna get some sort of like plugin install requests for Docker. If you want to, you can just hit install plugin. There'll be like a blue bar and I'll just add some formatting as you can see me having it here. But if you don't want it, you can just click Ignore. But just as a quick side note, you'll probably see that pop up bar. But anyway, so for these improvements, you're going to see this generally a lot. And we're gonna look at is we're first going to create a user. This user is going to help us because we don't want to run ours whole system as the root user in the end. But instead, if we run it as a user with reduced privileges, that can help us. Because if our Docker image or a Docker container ever gets attacked, then that user that hacks, it is not going to have full system access, but it's going to have reduced access, which is nice. The other thing that we're going to look at it as multi-stage builds. And this just helps us reduce the size of the final image that we have that we're going to run, which just helps keeps things smaller. So yeah, And just as a general note, for this darker stuff, we're gonna go through this step-by-step so that you understand what we're doing. But most of the time you're going to have some sort of template. If you're working in an organization, you can probably be pretty sure they have some sort of Docker template that you can use to work off of. If they don't have an official one, you can just look into another repository and use that as a template and kind of adapted to what you need. And alternatively, if you know what you're looking for, There's a lot of templates online tool that you can use so you don't have to memorize this stuff by heart. You're gonna be probably using templates for this. You just kinda need to know what you want, why it's there. And if there's anything missing, add that in, essentially. So that's what we're going to go through it from the ground up. But most of the time when you're doing this, you can write it yourself. But it's also just easy to use a template because most of the time you're gonna be doing very similar things. Alright, so let's start off by adding our user. So we're going to add another run command here. And we're going to do Add Group minus S. To add this, we're going to create a group that we're going to add our user into in a second. We're going to add that to our system. And I'm just going to call this group my app, but you can call it whatever. Then we use this and, and which allows us to chain commands together. So first we're going to execute this one, and now we're going to run another command after this one, which is going to be add user. And we're also going to make this a user to our system. We're going to add it into our my app group. We're going to give this user the name user, and we're also going to assign an ID for it. Let's just do 1234. Now if we want to run everything as a user, we're going to use this Docker user command. And then we can either give the name here that we've assigned or we'll see in a second, we can all see is the id. And what this is gonna do is it's going to look up this user in the system, and it's going to find this ID. And then the next command or entry point that we have, it's going to run using this user instead. Which is nice because then we're no longer running this as the root user that has full access, but instead with reduced access. So let's go ahead and make sure this runs. We're going to first Docker build minus t. We'll do my app latest. And I'm actually also going to use this and hand here so that you can see what it does. But essentially first we run this. And if, if this is successful, then we're going to run the next thing here. And that's going to be docker, run. Sign it a name, my AP test. And we can map the ports like we learned previously and the image we're going to run my app, latest, which is going to be this one here. Hit Enter. Now this should go relatively quick because things should be cached and we only added two more commands at the very bottom here, we can see here the building process went very quick. And there we go running, and it's running properly. So excellent. Alright, let's go ahead and shut that down. Control C. And yeah, go on to the multi-stage build part. And here we're going to add some more changes, which we'll see in a bit. So the first thing that we're gonna do is we're going to keep this from command. We're actually going to assign an alias to it and we'll call this our base. And in here we're essentially going to be doing our installation and our setup. And then in a little bit, as we'll see, we're going to copy things over from this base that we've created into our main running image. And now this everything that's not going to be the latest one that we're using is only going to be temporary and then it's gonna be discarded, which is nice because that means everything We don't pull into our final image will be discarded and it won't take up any space. Alright, so we're creating our base image here. So now the first thing that we wanna do is still and we want to install pip nth. And now rather than because the first thing that we wanna do or the whole point of this base images, essentially we want to install some of our stuff. So what I'm gonna do is I'm going to take this here, I'm going to cut I'm going to put this over here. But I don't need the main file right now because I just want our pip file and our pip file lock. Because I just want to do the installation. I'm going to just for now create an extra copy command down here. And I'm going to take this part and move it down here, and we'll get to this later. So right now I'm just going to copy or pip file and then pip file lock. Alternatively, what we can do is use pip file star, which is going to look for anything that matches this format. So anything that has pip file upfront and then any file that has anything after that. So this is e.g. we're going to match pip file and pip file dot loc. And if we had another one like pip file, file, this would also get matched by that. But this, oops, didn't want that. We can use this or the other one. But this is just a shorthand. Alright, so we're copying over our pip file. And now the next thing that we want to do is actually run this installation. Because we want to run this created and our base image. And then we're going to copy over the installed files so that we don't have any, anything else. Alright, so we're gonna keep a similar syntax, but change things up a little bit. So we're going to keep our pip to install. We're gonna do a system-wide install. Rocks are going to add another flight here, which is going to be the deploy flag. And if you want to know what these flags do specifically, we can, because we have pip installed, we can go pip install minus h. And this is actually going to pull up a help look up, which will give you all the commands that are accessible. So we can see if we go to the system e.g. is going to be we're going to install this just on our system or deploy. So this is going to be good because essentially if something is wrong, like our log files out of date or Python version is wrong, it's going to abort. Which is good because we just want to make sure that everything is running as described. And so if something is wrong here, It's just going to abort. And then we're going to add another flag which is going to be ignore pip file loops. Which means we're going to build specifically from the lock. Alright, so this is gonna be our pip install, which is a little bit different, but still very similar to what we had before. There's still some changes that will need to make here in a second, but we'll get to those when we get to this. Now we're going to add another from and we're going to pull from the exact same image here. And then here. Now we're going to, well, we want to copy over. The thing is that we have previously installed here so that all of the packages that we installed are copied over into our main image that we're going to need. Now to do this properly, there are several things that we need to do. So first of all, we're going to add some environment variables, which actually for Docker, we can just use this end keyword which is going to set environment variables for us. So the first environment variable I'm going to use is going to be our pi root. And I'm going to define it to have this slash pi root value. And this is essentially going to, we're going to use this in a second to say where we want things installed too. Then we're going to create a Python user base variable. Now this variable is important because it tells us or tells the system the base directory for the user. And we're going to set this value to this pirate environment variable. And I can reference the value contained in this variable by using the dollar sign and then the variable name. If I want to, I can also put this in square brackets, which can be helpful if there are other things attached afterwards. So now what we've done is we've defined this environment variable to this value. And we've defined this environment variable to the value contained within here. Now we're going to use one more environment variable, which is going to be the path. Now with the path is, is essentially a list of places where our operating system will look for executables for. And because we're gonna be installing into this directory here, we want to make sure that we add this into the discoverable list of files to look for executables. For. So, to do this, we're going to reference our existing path variable, and we're going to add onto it our pirates. Slash bin directory specifically because this is where our execution cuticles are going to live. Now again, like I mentioned previously, you don't have to understand this like inside out. You don't have to be able to reproduce this by memory. The point is that you understand why we're doing these steps. Here. We're adding this because we need to have some sort of base user directory where our installs are gonna go. And now because we're adding installations, we also want to make sure that the executables can actually be found by operating system. That is essentially the steps that we're taking here. And if we don't take these steps, you'll see. And we can try that in a second. It's not going to run. So it's important for us to add these steps so that it will run properly. And all that we're doing here is just creating environment variables. Alright, so we have our pip install, pip end, which we're going to keep the same. So we still want to install pip end. We're going to copy over our Pip files. And now we're going to add one more flag in front of here. Or we're going to define our environment variable for this command. And we're going to set the user environment variable here to want one. Which means it's going to treat this installation as a user installation. Which means essentially the user location stuff that we defined above. I'm going to take effect. So we've just updated this to now treat this as a user installation. And this is the first part. So this is what our base is going to consist of. And again, the purpose is here, is doing the setup. And if you ever need to install any other dependencies, you would do this in the base component. So installing everything that we need that we can then use to install other things. All of that stuff is gonna go in here. Now we're going to have our second Docker stage. Here. We're actually going to put our final image together and we're going to take over what we need from here. And everything that we're not going to take over is actually gonna be discarded at the end, which is nice because that saves us a bit of space because we're not taking over unnecessary things. We're actually going to define these three variables exactly the same way here. So I'm just going to copy these over. Now the next thing I'm gonna do is I'm actually going to take this user command. And we're going to move this up here. Because when we're doing our copying in a second, I actually want to make sure I assign these files specifically to the user. But before we do that, we're going to have a copy command here because we first want to get all of our packages that we've installed with our PIP n over into this image. Now, I'm going to use a CH own. And here we're going to define the group first. So the group here is my app, and it should be an equal, not a colon. The colon goes here. So this is going to be the group, and this is going to be the user. And we're going to add another flag from base. This space references, this alias that we have up here. Now, we're going to copy over the contents from the pirate directory, from our base into our pirate directory on this image here. So essentially we're copying over now all of this stuff that we've installed here with the pip install, we're copying all of those packages over it because we need them, because we need those things installed to run our application. And this command, this syntax here is a special thing that we can add for Docker and interests means copy this over and assign the privileges for these copied things to our user here. And this is where we want to copy from. Alright, so now that we've copied over all the packages that we need installed, I'm going to keep this year because we're going to keep our directory. I'm going to keep our assigning of the working directory. And rather than just copying these directly, I'm now going to copy them with privileges assigned to that user. Like this. Yeah, that is it. That is it. This is our updated Docker command. And let's run this to make sure it works. And then let's go through it one more time. So I'm going to, first, I need to remove the image that we ran previously, which is yeah, this is the name that we assigned to it. So I need to remove this one first. And I'm actually going to use this and again here. And what I'm gonna do is because this helps me run it several times. If everything passes, I'm going to remove the image I created previously. This only works obviously if I've ran these two commands first. But once I run this docker run, and I just want to run it with the same name over and over. I'm going to remove that existing sorry, not image. I'm going to remove that existing container. I'm going to build a new image from our Docker file template. Assign the same tag. I'm going to run this image as a new container in this name. So let's try that. Okay? So we can see here we have two different stages also happening. We have a base stage and we have our final stage. And there we go. Our system is now still running, thankfully. But we've integrated some darker press best practices. We are now not running as the root user. Instead, we were running as a user that we've created with reduced privileges. And there's also a step here which for us probably is not going to have that big of an impact. But once you need to install other binary things to help you install other libraries, like first, you need to install some things so that you can install other things. It will help you because those things that you installed previously, once you've installed your library, you may not need them anymore. So this process is just going to help with that because essentially a lot of the setup stuff that we may need to install, but may not actually need after the installation is complete, we can discard all of that. So everything here that we're not specifically copying over in this final stage here. Currently, this is our final stage. All of including this. All of this here that's not copied over into here will be discarded. So that's just going to help us again save some memory. And yeah, this is multi-stage building with a additional are running things as a user, we can also change this instead to use the user ID that we've assigned to it here. And let's run that again to make sure that this still works. If you look up Docker templates, both within an organization or also online somewhere and they're going to follow a similar format like this. They may have some differences somewhere, but this is going to be the main goal is you have a base image and you'll often maybe sometimes also see like a base image. Then you'll see a builder image where the building process actually happens. And then you'll see the final image where the things are copied over into. There may be even more than two steps involved, but it's the same idea of doing the setup and then copying over everything that we need and discarding the rest to just make our final images a bit smaller so that we're not using unnecessary memory in our images. That we essentially we don't want things installed that we're not going to use is the whole point of all of this. Alright? So, yeah, there's one more optimization that we can do, which I'm actually going to do right now. We have two stages here. And generally, a good goal to have is have as few stages as possible. So what I'm gonna do is instead of having the main dot py outside, I'm actually going to move this into our application here. Like this. Copy this over. And in our main pot here. We still have our application defined in here. And I don't think we need to actually change anything in here, which is great. But now in our Docker file, we need to, we can now essentially remove this. We can also remove one of these apps because now everything is gonna be contained within our app folder here. We can remove this app too if we want to. Or alternatively, we could leave this in and just do it like this. And let's build and run that one more time. And hopefully I've updated all the references correctly. I'm just going to take a little bit longer because we've made some changes into our code here. And so at some point before the copying, Oh, okay. Let me quickly find what is going on here. Main.py. Well, I think this is just try it like this. And now I need to update my entry point here because our working directory is one level up. So I need to make sure I go into this folder and then run it from there. Yeah, One more time. Third or third time's a charm. Alright, there we go. So what this steps we were able to reduce her number of steps that we have in our Docker file by one, which is nice. And essentially we just had to change up some of the files that we created in some of her name references here to make sure that everything is in the right location, that we're referencing it from the right place. So now we're essentially on this level. Here, we are going into our application folder, into our app folder, we're referencing the main file. And in here we're still referencing this app, this initialized app that we've created here. So the process is still the same, but we've kinda put it into the one thing here. We've reduced our states steps by one, which is just nicer. The less steps you have, the better it is. Let's just a general good thing to aim for. Now, before we wrap up this lesson of Docker best practices, actually noticed we also want to add our user equals one flight here because we want to make sure that all of the installations that we're doing are you actually going to be with the same user level so that we're not getting some things installed on our system and some other things installed on our user. Additionally, I've also noticed when small typo, namely this should be user first and then the group. So let's quickly fix that too. And there we go. 5. Docker Compose: Alright, so previously we've taken a deeper dive into this Docker file and have restructured it in a way that's nicer following best practices. Now we're going to take a look at Docker Compose because currently when we've been using Docker file, who's been building the single file, and then we've been running it. But actually what will be nice and what you'll see also when we get to it later when we add like databases and caching and stuff is we don't only want to have our application running, we also want to have a database running that we're able to use with our application. And we probably also want to have a separate cache running we want to use with our application. And so to be able to set up this whole system that we can reuse. And there's actually a lot of cool things that you can set up and well, it's a deep topic to dive into and we're obviously not going to cover everything, but we're gonna go through the main points. And as we get to it in further points down the road in this course, we're also going to touch this Docker Compose again and add components to it. E.g. the database in the cash that I just mentioned about. But the point being that sometimes we just don't want to run one thing that's kind of isolated, but we want to run several things and maybe those things or even connected. And so to make that process easy for us, we're going to use Docker Compose. And so we're going to start off by creating a new file here. And I'm going to call this Docker compose. A Docker, Compose dot YAML, like this. If you have Docker installed, you should also have Docker Compose. So everything should already be set up for you. Alright, so we're gonna write some skeleton code first and then we're going to explain what's going on. And essentially the first thing that we're going to try to do is we're going to try to reproduce what we've had running with darker. But now instead of using Docker compose, so we just want to get the same service up and running as a Docker container that we can still access through our web browser. So the first thing I need to define here is what version we're gonna be using of the syntax we're going to use here version three. And then we're going to define the services that are running here. You can see I have the Docker plugin installed. So some of this stuff actually auto fills. If you don't have it installed, then it won't. And if you still have that notification up here, then obviously the easiest way is just to hit install plug-in and it will install it for you. But again, not really necessary. It just helps, helps with like auto-filling and some stuff. But yeah, alright, so here we're going to define the different services that are running. One of those services, e.g. is gonna be your application. We're going to define one service. I'm gonna give it the name app. We can call it whatever we want, but I'm going to give it the name app just to be clear around what this is. And now in here we're going to further define some of the things about it. So the first thing that I wanted to find is, how can I build this application? What is the Docker file that we need to reference? Some cases we may actually be referencing a Docker file that is on Docker Hub. And in other cases we want to reference a Docker file here that we have. So I'm going to have this bill parameter and I'm just going to point it to the current directory. So just using a single dot for that. Now the other thing that I want to do to get access to it, which we also did previously, is I'm going to define the port mappings. So Docker, we use that minus p Here. We're going to have this be a ports. And we can see here it's already auto fills it for me. But essentially we're gonna have a colon after here. We're gonna go on a new line. And then we're going to have a dash which indicates that this is going to be an array. So we can do multiple port mappings for we're only going to use one port mapping here is going to be again from port 80 to port 80, 80, like this. And here we can see we're starting this on port 80, 80. So we want to make sure that our outside port is mapping to the correct insight port. Same thing like we covered when we ran the individual Docker file. And this is actually it, what we need to essentially reproduce what we had with our Docker build. So let's go ahead and spin this up. So the first thing that we need to do to get this running is we can use the Docker dash composed command. And then we're going to type in here up. And we're also going to add an additional flag which is gonna be minus minus build. And so this means that we're going to spin up our container, but we're also going to build it first. So first build than spend it up. So let's go ahead and run this. I'm just going to probably take just a little bit of time to build. Most of it should be cached. And let's see here. It's just still creating it. Alright, so there we go. We have our container running. And if I click on this than I should have access to it. So if you go to the docs page and try one of these end points. Alright, there we go, Cool. So this is the default value that we have and it's working. Then if we want to take it back down, well. Now we're not running it in daemon mode. So if I just hit Control C, It's actually going to stop it, which we can see if we go back here, refreshing it is now down. But if we want to run it in daemon mode, like we did for our Docker. So having it run in the background and not taking up the foreground here, I'm gonna do the same thing, minus, minus build. And then I'm going to add a minus d flag here and run this one more time. Again, the whole thing should be cached, so it should go pretty quickly. And there we go. So now it's still running in the background. I can do docker ps to see that this is indeed running. And if I go and refresh, then again, we're running. Now, if I want to take the service down, I can do docker compose down, which is going to just stop all of this. And again, if I refresh this, there we go. No longer running. Alright, so far so good, right? Pretty simple and pretty convenient. But so far, not that much difference to using the Docker command, right? It's, it's a little bit shorter because we can kind of do the build and run step at the same time here, but still not so much shorter. So let's get into it a little bit more and see some of the other cool things that we can do. So one of the things that one of the additional things that we can do, e.g. is we can define this entry point variable. And here we're going to use the same syntax as we had in our command here. It's going to take this and copy it over. But maybe what I actually want to do, I'm going to copy this over here, is when we're building our Docker file and we want to run it somewhere. We obviously don't want to run it with the dash, dash reload flag because that's going to add a lot of overhead that we don't want. But when we're developing locally, it's really nice to have that flag because we can change things and we can make things. We can just develop quicker because we don't have to take down if we start our application. So ideally we'd be able to use this minus, minus reload flag. And so if we define an entry point on our Docker Compose level, it's actually going to use this entry points or this command instead of whatever final command we have here. So in this case, if I run this again, docker-compose up, and I build it and run it. And David mode, we're going to see in a second. We'll actually not. So I'm gonna take it down and run it not in daemon mode so that we can just see the output directly. Let's do that one more time. Here we see we started a reload our process. So that's because we have this reload flag now active. So we can see this works properly. Unfortunately, because we're running in a Docker container. So if I refresh this, this is working as intended. But if I try to make any changes, e.g. going in here and maybe just updating the name of this end point, like just add something random here. We can see already it's not restarting. And if I refresh here, it's also not updating. But ideally, what I would want is I want to be able to have this thing running and be able to edit my code. And as I'm editing just like we did when we ran it locally without Docker, I want to be able to have these updates kind of updating. The way that we're going to do that is we're going to take this down again. We're going to add one extra component here, which is called a volume or volumes. And what the volumes allow us to do is they allow us to map essentially outside storages to places inside of our container. And volumes are extremely nice because they provide us a persistent method of storage. So our containers can go down, right? Like if we have our Docker thing running and we save things, ideally, we do not want to save anything in our container because that container can shut down. And the way that our containers should be working is we should just be able to spin up a new one and it should have the exact same state as the previous one. Like it shouldn't evolve with time. If we want to store information, we're either going to send that out to databases somewhere. Or if it is a database itself, it should store it in some sort of storage that even if the container goes down, the storage is still available. And so that's what those volume allows us to do. It allows us to map external places of storage to places in our Docker container. We can use this for databases which we'll see later to persist data as we spin up and down a database instance. But we're actually going to use it for now, is we're going to map our local states of our code into the state of the code in our Docker Compose. What I'm gonna do is I'm going to map our current directory to. This place here in our Docker file. And this should not be here. So let's go over this one more time, then let's run it and see how it's working. And then maybe take another couple of moments to think about what's really going on here. But essentially what we did is in our Docker file here, we've created a new folder location user SRC app. And essentially because we're copying over this app into this app folder, in this path, this location that we're in this folder, this directory essentially represents what we have here. We didn't copy over everything, but essentially, right, that's that's what we have. We copied over the things that we needed. So if I add the mapping here, e.g. of our current location to that location in the Docker file. Then actually, and I'll show you this now, and I'll add the minus d flag again. Because now we can have it running in the background. Now we're going to map this place inside of our Docker file to this place which is currently on our local machine. Let's see this in action. So if I refresh this, take this down. Okay, there we go. So now we still have what we have here. Now let's go ahead and make a change. I'm going to add an extra a here, Save. I'm going to refresh the documentation. Now we can see our updates have actually been reflected. So again, the reason that we were able to do that is because previously we were unable to access after we've created our Docker image and we started with the container, we were unable to access its contents internally. But now we've created a mapping of this folder here to this folder here. And what we could also do is we could actually be more specific and just map. And I'm going to undo these changes here like this. And let's do docker compose down. Just so that we're starting from a clean slate. We can be more specific and map the application folder that we have here to the application that we have there. So we are, because we're copying this over in our Docker file, like this, which is going to be in this path. We're going to take a mapping of the app folder here to the location that'll be copied the app over a and our Docker file. Refresh this one more time. There we go. So what we had, now I'm going to update this one again here, save, refresh. And there we go. So again, volumes allow us to map these storages from inside our container to some sort of external storage. And this is really nice because in this way we're also able to save data outside. And if a container goes down, we can just connect it to the persistent volume. You don't have access to the exact same data. Which is obviously really nice if we have databases running e.g. and yeah, just keep that in mind. That if you're running a Docker container and you're saving to a local file. Once that container goes down, that file has gone. So always, if you need anything, always save it somewhere outside of your container because you won't really have very easy access to it inside the container and once it goes down, it's gone. So just keeping that in mind. And this is again where volumes really help us out. Okay? So volumes are pretty, this is kind of what we have right now, which is essentially the application that we've had previously. Let me take it back down. Now we're able to do a lot more with us. So we have our current application. But something that we also want to do is we wanna be able to like run our unit tests, e.g. ideally, we would run that in the Docker container because that's exactly the setup that we're going to have when it's running in production. Because that's the whole point of this, that we have this reproducible setup. So I'm going to create a new service. I'm going to call this app minus test, just to indicate that this is a test version of wrap. I'm going to take this here. And I'm just going to copy most of this over. And I'm not going to update the entry point. This doesn't really matter too much. And I also don't need a port mapping because I don't actually want to do anything with our test container. And also if I use this port mapping, then we're going to have a conflict because we already mapping our system port to this port. And so the Docker container, and we can't remap the same port twice. So this will actually cause problem. So we need to remove that anyway. And I'm going to now instead of mapping the app, just the app folder, I'm actually going to map this whole directory because if we go into a Docker file, you'll notice we're not actually copying over the test folder, but we still need to have access to that inside of a Docker container. Now when we run this, we will actually have two separate containers running. The first one is going to be your application that we're currently using with this slash slash reload flag, which will allow us because we're mapping our folder here to that place inside the Docker container to do development and look at it in our web browser and be able to play around with it. This container here is essentially going to be responsible for just running unit tests against so that we can actually test our code. Okay? Alright, so there are still some other things that we need to take care of because if we go into our Docker file, we actually notice here we do not install any dev dependencies and we need the dev dependencies to be able to run our unit tests. So now what we have to do is we have to update our Docker file so that the installation process that happens here is different depending on if we're in our application or if we're in the application that's just meant for running tests. And so let's do that first. Now what we need is some sort of way to indicate where we currently are. Now a great thing that you would usually do with this is using environment variables. Unfortunately, we don't have access to those inside of our docker container, but we can provide build arguments. The way that we can provide those is by, instead of just using this build syntax here, we can actually expand upon this. And I'm going to provide two more keywords. So the context is again going to be where our Docker file or where we're going to run this thing out from the location, which is going to just be the current location. Then. Now here we can also provide build arguments. And again, we can provide this as an array, whereas the list, and we're going to provide the name, and then we're going to provide the value. So e.g. I. Can call this name environment, environment and make sure I'm saying that correctly. I think I am. The value here is going to be test. So I'm just gonna make up a name, what I'm going to call environment. And this is going to be the value test. And now I need to do a similar thing up here because I don't want to have dev dependencies installed in our application. And so this one, I'm going to call either dev or maybe I'll call it local. So now what I wanna do is I am passing this bill argument. Now I need to be able to get access to it inside of a Docker container. And then I want to update our run here so that if we are, if this environment variable here is test, I also want to install dev dependencies. Otherwise I do not. Alright, so first things first, to get access to this bill argument. I'm going to use this args keyword. And then I'm going to put in the name of this variable here. This is now I'm going to give us access to this argument inside of our docker build here. Alright, now what we need to do is update this statement to conditionally and we're going to use an if statement for this, but we're going to use a bash IF statement to conditionally install with dev dependencies. If this environment value here is test to write an if statement and bash. And we're not gonna get into too much detail here, really just kind of going through the bare-bones to have enough to be able to do this. Because most of the time you'd like you won't be needing more than this. And if you ever do, because you know how to write conditional statements in Python, you'll understand the logic and be able to transfer it to bash. You just need to look up the appropriate syntax. So syntax is going to be if and then open and close square brackets. And then here we're going to put IF. And to access the environment variable in Bash, we're going to need to put $1 sign in front of it. So if the bill argument, if the value of this build argument that we have access to, because we defined it up here. If this value is equal to the string test. Now a very important thing here is having spaces before or after and before we open and close these square brackets. Then I'm going to put a semicolon here. This is important because if we want to have more than one command on the same line in Bash, we need to separate those by semi-colons. Otherwise, we would be creating new lines which would also serve that purpose. But because we were going to have most of this on the same line, I'm going to put a semicolon here. So if our environment is test, then we want to install with dev dependencies. So then we want to do the same thing here, but with dev dependencies. Again, colon, because this is all on one line. Now because we're going to go over one line because I don't want to have this thing or like running into infinity to the right hand side. But we can also see we have like margins suggestions here and we don't want to go over this. So to continue a, something that was on the same line. And we're going to enter a line break that we can do with the backslash. We're going to continue this on the next line. And then we're going to have our else statements. Another colon here. And then to end this statement, we do FI, So the opposite of f. So let's go over this one more time. What did we do? We created a build argument that we're providing as we're building our image with this value here, we're then getting access to the spilled argument here. And we're accessing its value here. We're comparing it to the value test using a single equal sign here. Then we have a conditional statement. So if this value is test, then we're going to do this install statement and we've added this minus, minus dev tall, some stall dev dependencies. Otherwise we're just going to do our normal install. And then we have some special keywords stuff here. Just because that's kind of, that's the way that the if statement looks like. That's what you have to write it. Alright? Okay, so let's, well, let's start these up. We don't, we're not actually running our tests yet, but we can if we want it to. So if we wanted to, we could e.g. update our entry point here to instead be something like pi test and tests. But we're going to look at a better way that we can do that in a second. Because if we do it like this, then will only be running at once when we started up. But ideally we'd be able to just executed at-will. Alright, so I'm going to just run the build not in David mode right now so that we can see the full output. Right? So this time it's going to take a little bit longer because we've updated our statements here. And so now we're no longer going from cash, but instead we have to redo this thing, which actually includes installing components, getting it from online, downloading it during the installation process. So it's going to take a little bit longer, hopefully, not too much longer because everything else should still be cached. Wait for that to finish. Alright, so our build finally finished. Everything is starting up. So we can see here this is our first container running, and this is our second one. And here we can actually see the output of the tests. We have nine tests that have passed and the other container is still running. So I'm just going to loop, So don't want that clicked on a link here somewhere. Alright, so let's take this down. Alright, so we got that running. Excellent. Now the thing that we're going to update is instead of just running these tests once. Instead what we're going to learn to do is we're going to run these tests whenever we want to against your containers. So I'm going to build this, spin this up, bill it again, run it and daemon mode. If no code has changed, you also don't need to rebuild it, which will obviously make the process go faster. So if we do it like this is just going to spin up the existing image that we have built for it. Or rather, if we do add the build flag, then it's going to rebuild. And even if everything is cached, obviously it's still takes a little bit longer. Now to run our tests against this testing container that we have here, we're going to use docker ps, dash compose. And then we're going to type in here exit exec. So we're going to execute something against something. Our target is going to be this app dash test service that we have here. And what we're going to run, the command we're going to run, it's going to be pi test tests. So we're running pi test against our tests folder here. If I hit Enter, and there we go. So we're running our tests. And now this is really nice because everything is running within a Docker container. We have several Docker containers. We have two separate containers here running. One of them we can use for local development. We can make our changes. So just see this again. If I remove this, this should still be going away. And we have this mapping because of volumes. We have several containers running. One of them is one that simulates what we'll have in development production, except that we've overrided this flag to be reloading. This one. Although we never copied over any of the test files here, because we don't want them in our final image because we use a volume mapping here. We actually still have access to all of that here. Because the way this folder looks inside of a Docker image is now actually going to be represented by what we have here locally. Then we were able to use this Docker Compose exec command to run a command against a specific target, which is the name that we have here. And this is the command that we're going to run against it, which means whenever we make changes, we can also just make our changes here, update our tests here, and then when we run a run it, we can just execute the tests against it. And with that kind of run our unit tests and everything is in a really nice reproducible Docker environment, which is awesome. And with this setup, we can then will see in the future, we can do a lot more stuff with it. We can e.g. add like environment variables. By just having an extra keyboard here, we can already see. We can fill in this stuff. We can add extra services which is gonna be like our databases and our cache, e.g. or if you're actually building multiple services that maybe want to contact each other, you can actually have all of those service, different services that you're building running at the same time. So you can test all of them as if they were running live somewhere. And this really gives us a lot of power because we're able to just spin everything up, let it Talk to each, each other, and be able to test in a very nice and reproducible manner. So yeah, Docker Compose. I hope you find it useful. And again, there are a lot more like arguments here that we've looked at or that we, that we haven't looked at that provide extra functionality. And if you ever need anything else with darker, then you can just look up what is the variable name that I need to provide here and how can I put that in? You'll be able to set that and Docker Compose. And personally I prefer this a lot more because our commands here are a lot shorter compared to having these longer like Docker commands. And again, we can define everything in these YAML configuration files which someone else can just take and run. Whereas otherwise you'd have to send them your exact copy of, um, what, what command you're running to. Spin up your container from an image if you're like providing a lot of environment variables or something like that. Whereas with this Docker compose YAML, we can have all of this pre-configured. And hopefully someone else should just be able to take all of this, spin it up, run the Docker Compose, and be able to have the exact same thing. And it just, it just makes everything easier all around because once you've reached this point, obviously it's a little bit of work to get here. But once you reach this point, you're at a really nice place. We're able to have reproducible things. You know, what you have here is how it's going to be when you run it somewhere else. It's very easy to make changes, to add new things and to keep everything like reproducible and testable and everything like that. So one quick more thing that I do want to show you is sometimes you may have e.g. multiple Docker compose files. And essentially Let's have, I'm going to create a new directory here. I'm going to call it debt test, DC. We're gonna make sure to take down my existing containers. There we go. And then I'm just going to move this Docker compose file here. So let's say we want to have different composers for different scenarios. If you ever need to have something like that, then what I can do is I can add a minus f flag to reference, where that is. If I run this now it's not going to work because it can't find a docker compose file. But if I add a minus f flag here, I can point to my file which is gonna be intestacy slash Docker, compose, YAML, that is our filename. And then we can try to run it again. And bam, there we go. It works. So just kind of like a quick aside in case you ever run into something like that. Obviously to take it down, we needed to have the same syntax here to take it down. And I'm going to probably do not want that here. Like this back here. Obviously. This is not as nice as just having it directly here and that's exactly how many keep it. But just as a quick aside, if you ever run into something like that, then that is how we can go about that too. But yeah, so that's darker. A lot of Docker docker compose. And although we've done a lot of legwork to get what here, now we've reached a really nice point where again, we have very little commands that are very short and readable that allow us to build very complex systems and have them run in a way that's reproducible, which is just super nice because this can give you a lot of peace of mind to make sure that the way that things are running here is going to be the way that things were running elsewhere. Which is which is really, really nice. Yeah. I know that some of the stuff can be a little bit confusing when you look at it the first time. It takes some time to play around with some of these things. Take some time to play around with this volume to understand what it does. Take a look at each of the things that we did here. Maybe try to add another build arc or change the naming here just to see what happens, to make sure that you feel more comfortable with it. But over time, you will be feeling more comfortable with it. And also again, it's because when we're building this stuff up, a lot of these things will end up looking similar for, for obvious reasons. So you'll generally be approaching it the same way. And most of the time you'll have like templates to work off of, to have these docker files. And then you'll just have to kind of adapt what you're copying over and how you're doing the installation and et cetera, et cetera. But a lot of this is actually like boilerplate template stuff. You know, that you can reuse. And obviously we want to go through it in detail once just taught her understand properly what is going on so that if you read it or you need to make changes or additions, you know where to look, you know what to look for. But yeah, most of the time you'll have like a template to work off of. You have a template now, you may have a template and organization and you can find templates online that can help you reach this final result a lot faster. 6. Makefile: Alright, so we've gone a long way and we've actually managed to reach a point, which is pretty nice about not having to write that much stuff to get things up and running. But over time, you'll probably even with this, you get frustrated having to write over and over the same things. And there are more ways that we can use to shorten this. To essentially have shortcuts for commands that we're executing regularly. Now, this will work on a Mac and Linux, I think for Windows you will need to do some extra setup to have these makefiles work, which is what we'll look at now. But generally speaking, if you end up working in an organization somewhere, it's unlikely that you'll be working in Windows? Not yeah, not that it's impossible to is just much more likely that you'll be working on a Mac or a Mac or Linux. So yeah, just kind of keeping that in mind. We'll go over this now because it's common to either create makefiles for yourself or to see them in another repository. Because as we'll see in a second, it will make your life a lot easier. Again, for Windows, there is a, some other stuff that you need to install to actually have these commands work. Sought, super important to get this running. So just be aware of this, that these things exist and this is how we can run them. And you can create them if you want to or you can leave them out because they're just kind of nice things to just make. Short. Syntax is even shorter. But yeah, anyway. Alright, so let's create a new file here that we're going to call a make file like this. And in this makefile, we're now going to define a series of commands that are going to allow us to essentially what we had here like the Docker compose dash, dash minus d, just so that we don't have to write this out every single time. We're going to make a shortcut that's going to run this for us. Now, one very important thing here is in, if you're in PyCharm, like I am here in this file, we want to go to the bottom and click on this four spaces and configure endurance for plain text and use Tab character instead. Because the Makefile has a very, very specific difference between a four spaces and a tab. Whereas in other cases, like in Python, e.g. it's, doesn't really, really make that much of a difference. So it's fine to use it like this. But in PyCharm you'll find I've done here, if you're using Visual Code, if you're not using PyCharm visual code or anything else, you'll find it in a similar probably location. But generally, if you run into issues that is saying that there's some missing separator or whatever. Very likely the issue that you're running into is what we want. You forgot to put a colon, which also happens. But to the other very common issues that you are using spaces instead of taps. Okay. I'll show you also on a second quick way to see that. It actually let me turn that off for that, just to show you how that would look like. Alright, so the first thing we wanna do is we're going to define a command here, which is I'm just going to call start because it's short and I like it. But you can call it whatever, like run, spin up, whatever. I'm going to call it start. When we run this command, what we want to have is this minus the build part, like this. Now, you'll notice that I'm using spaces right now because if I back, go back here, you'll notice it goes back four times. So these are spaces. And if I change this from four spaces to using a tab character instead, delete this and press Tab again. Now when I'm going left from right. So this is the top character. Very big difference for using spaces here. You're gonna run into problems and it's frustrating. Alright, so we have to start. Another thing that we use very commonly is start with build. So I'm going to call that start underscore build. And here I'm going to just put this command like this. Another thing that we want to do is take it down. So you either use takedown, but that's a long word, stop it easier word. And we're going to do docker, compose down like this. And another thing that we probably wanna do regularly is run our unit tests. We can call that unit tests are unit test to use plural because we're probably running multiple tests. And the command that we're going to run here is gonna be the command that we saw previously. So Docker Compose x minus t and it's going to be apt to test, which is against this container here. The command that we're running is pi test, tests like this. Okay? So these are four commands that we'll use pretty regularly. And here from this we have our Makefile. There's one thing that we want to add at the top, which is we want to provide a list of felonies. Which essentially tells the makefile that these are commands. And if we ever have files of these names, that when we use this with the make command, which we'll see in a second that it should treat these as commands. We're going to put all these commands here that we want to use. Just going to be start that we're going to have start to underscore build. We have stop, we have unit tests. Alright, so let me see if anything is running. It is. So I'm going to do, well, let's use our main command. So I have these two running still from last time. What I'm gonna do is now I'm going to type make stop, which is going to run this command here. When I hit Enter. And we can see it's executing this command and it's taking everything down. We can do make start to run this and it's going to bring things back up for us. So see here, if I refresh, it is Hold on 1 s. There we go. Alright, let's take it back down and stop. And there we go. Now if I refresh, there we go. So no longer running. One thing to note, if you don't want to see these commands, we can add at symbols in front of everything. Now, when we do make stop, or do I already run that? Sorry, mic, start. Now we're not going to see the command print it out. So that's just a little bit to just get stuff even here. If we want to run our unit tests, unit tests, which is going to run it against our testing application here, just running this command like we've seen previously. Bam, There we go. Another command that we maybe want to do is check our typing. So we can do check typing. And I'm going to run this command here, or running against our test container. I'm going to run my pie like this, like we did previously. And then we can do check typing that to our phony. And we'll say make check typing, which should now check the typing against our testing container here. While this is running, let's make some shorthand because sometimes maybe we want to run unit tests here in our container. But in some cases, if your containers docker compose gets very large and you don't want to have all these things running than something that you could do instead is maybe also run things locally. So maybe sometimes you just don't want to spin up a container because maybe also the container just takes a lot. So what we can do instead is also just add kind of aliases for this and then reuse them. So e.g. let's create a unit tests alias. And this here is going to be, sorry, there should be an equal. It's going to be pi test tests like this. And now if we want to refer to like pi test tests, we can instead refer to this alias by using $1 symbol here and then open and close parentheses around it. And so then we can have our unit tests like we had previously. Like this. We can also have another one which is going to be unit tests. Local like this, which is just going to run our unit tests. And this is probably going to fail because I'm not in my virtual environment right now. But let's see. Yeah. So let's go ahead and go pick Penn shell. And let's do make unit tests local again. I'm going to open a new terminal here, got myself into a weird state. Make unit tests, local. Alright, so there we go. So this is a nice way how we can reuse this and similar things, and especially where commands get longer. Sundance is also as you have like commands that go over multiple lines, which again, we can separate like using line breaks like this. E.g. if this were to go over to a new line, I could instead separated that. But if we do have longer commands and sometimes it's also nicer to just have an alias for them appear that we can reference down here. Just so that, you know, this stuff becomes more readable and doesn't become so clogged up. And yeah, that's kind of what we're doing for typing. And more, sorry, not for typing. This is what we can do for our makefiles here, which again just adds a little bit more simplicity and makes her commands a little bit shorter than do make stopped to take down or containers instead of Docker Compose down. If I want to build, I do make start build. Just shortened said again. Just a little bit. And yeah, that kind of that wraps up this. There's one more thing that I wanted to mention, not related to our makefiles, which is just going to be looking at logs. Something that I find quite easy to do when we have like a Docker container is running is if I actually go into our Docker desktop up here, you can click into these and see the logs directly here. But just as a quick aside, because we haven't really done this, is if you do want to look at logs and you want to do this over the terminal, then there's the docker logs command. So if we reference a container ID, e.g. our main application, and we want to look at the logs. We can look at it like this. Phi one to look into our testing application and just put that command in here. There we go. So obviously we don't have a lot of logs right now. But just in case you ever want to do this and you don't want to use Docker Desktop for this, then you can also do this over the command line. Since we're doing a lot of stuff for the command line. And sometimes this can also just be easier for that process. So, yeah, hopefully, you'll also find having these makefiles useful. You'll probably see them too and being used in whatever organization you will join that they'll have like makefiles there to just have a list of commands that do a certain process, just makes it all very easy to just have makefiles for this so that it's even easier to run and you don't need to tell someone else what the command is. They can just run the make command and very simple to write. And again, a very nice way to get these reproducible things. And obviously they make our life a lot nicer to because we have to write even less.