The Express. js Course - Module 3: Node Package Manager (NPM)

1. Class Introduction: Welcome back. My name is Shawn Ragunhi and I'm excited to welcome you to Model three of the Express JS Cores Node Package Manager. As a full sech developer with years of experience building scalable applications, I have worked extensively with NBM to manage dependencies and optimize project workflows. And now I'm here to share that expertise with you. In this module, we will take a deep dive into NBM the backbone of every Nord Js project. Here's what you will learn how to create and understand package to JSON, the core of any Nor Js project, installing, managing and incorporating packages into your projects, best practices for dependency management, including semantic portioning, insights into the NBMRgistry and how to handle global and local packages effectively. And even advanced topics like upgrading packages, handling development dependencies, and stalling unused packages. This module is perfect for anyone working with NJs or Express Js, whether you are a beginner or an intermediate developer. If you are already familiar with basic NJs concepts, you are ready to level up your skills with NPM. NPM isn't just about installing packages. It's about managing your project's foundation. By the end of this class, you'll have a strong command of NPM and package management, enabling you to efficiently organize and maintain new JS projects. You will also gain best practices that can save you countless hours and headaches in your development workflow. For this module, we'll put your knowledge to the test with an NPM knowledge quiz. This interactive quiz will challenge you to apply what we have learned about NPM commands, dependency management, and semantic versioning. It's a fun and engaging way to solidify your understanding and ensure you are ready to use NPM in real world projects. I'm thrilled to guide you through this essential part of the Nor Jz ecosystem. Let's dive in and take your skills to the next level. 2. Getting Started with NPM: In this section, we are going to look at NPM or Node Package Manager, which is basically a com online tool, as well as a registry of third party libraries that we can add to our node applications. So for pretty much any kind of functionality that you want to add to your application, there is most likely a free open source library or package or node module, whatever you want to call it on NPM registry. So if you head over to npmjs.com, this is the homepage for NPM. And what you see here is different depending on whether you are logged in or not. So here, I'm not logged in. You can see some basic marketing text. But what is important here is that currently there are more than 2 million packages on NPM, and all of this is free and reusable co so you can easily add these building blocks to your application. Also, if you have an idea for a node module, if you have an idea for a functionality that you want to share with others, you can create your own node modules and publish them on NPM. Now the way we add these modules to our application is via command line tool that is also called NPM or node package manager. So the first thing I want you to do is open terminal and run NPM W. So NPM comes with node. When you install Node, you also get NPM. Now let's see the version of NPM. I'm running on this machine. Currently, I'm running NPM version 8.14 0.0. Now, interestingly, if you run node Hui, you can see the version of node is different from the version of NPM because these two programs are developed independently. Now, while you are watching this section, chances are your version of NPM might be lower or higher than the version I'm using here. So in order for you to have the exact same experience as the videos in this section, I want you to make sure that you're running the same version of NPM on your machine. So let me show you how to install NPM version 8.14 0.0. Run NBM space I. This is short for install. Then space G, this is short for global because we want to install this pagage globally on this machine. It's not specific to an application, ok? Then the name of package, which is NPM, at sign, and here we set the version number. So 188.14 0.0. Now if you're on Mac, you need to fix this with pseudo because chances are the permissions on your machine are not configured properly. And when you run this, you may get a permission error because you are installing a package globally. So a simple workaround that is to use pseudo. And once again, this is only for Mac users. On Windows, if you get a permission error, you need to open up the PowerShell window with administrative privileges. Now, let's run this. Now here, you may have to enter your password because you are installing a global package, okay? Now, this is going to take a few seconds. After this, make sure that everything work properly, run NPM di ui and you should have NPM version 8.14 0.0. 3. The Core of Node.js Projects: package.json: Alright, now I want you to create a new folder. Let's call this NPM demo. We are going to use this folder throughout this section. Now let's go to this folder, NPM demo. Now, before you add any note packages to your application, you need to create a file that is called package dot JCN. If you remember, in the last section, we set the type to module in package dot JSON. So package dot JCN is basically a JCN file that includes some basic information about your application or your project, such as its name, its version, its authors, the address of its glt repository, its dependencies, and so on. Basically a bunch of metadata about your application, and all node applications by standard have this package dot JCNFle. Now, create a package JN file, we run NBM in it. Now, this will walk you through creating a package JN file step by step. So it's going to ask you a bunch of questions. For example, here the first question is package name. And you can see there is a default value for this, which is the name of the folder in which we are creating this package JCNFle. You can change this or accept it by pressing Enter. Here's the next question, which is the version which is set to 1.0 0.0. Again, I'm going to accept this. The next question is description, then the entry point, which is index or Js, by default, the testimand. Again, we are going to ignore this address or gut repository and keywords, the name of the author, the license. And finally, you can see this JSON object here. The name is NPM demo version is one, and so on. So we have a bunch of key value pairs that represent metadata about our project. So it's asking if all this information is correct. So we presenter D. And now we have a package DJSNFle here. So one more time before adding any node packages to your application, you need to create a package DtJSNFle. So as a best practice, whenever you start a node project, run NPM in it to create package Dot JASN. Now, there is a faster way to create this package Dt TSN file. So earlier you ran NPM in it, and then you had to answer all these basic questions about your project. Now, if you don't want to do that, you can supply a flag that is dash. With this, you get a package JCNFle with all those default values. So you don't have to go through each question one by one. Next, I'm going to show you how to install a nod package. 4. Installing and Managing Node Packages: In this lecture, I'm going to show you how to add a third party library or a third party node package to your node application. So we're going to install a popular Java Strip library called lodash. So onnpmjs.com, here, if you search for lodash, you can find this package, lodash, and the latest version is 4.17 0.21. Let's have a quick look. So this is how we install Lodash we simply run NPM install Ladas in the PowerShell or terminal. Below here, you can see some basic information about Ladas. You can see the address of the Github repository. You can see the homepage for this package that is lodash.com. You can see the weekly downloads. You can see the latest version, the name of the collaborators, some basic statistics like total files, unpackaged size, as well as some other basic information. So whenever you want to work with the node package, a good starting point is npmjs.com. Now, back in the terminal, NBM install, or we can use I that is shorter and faster. So NPM I lodash. So when you run this, two things happen. First, in package JCN, you can see this new property dependencies, and under that, you can see lodash. So here in package dot JS, we specify all the dependencies of our project and their version. So here the version of dash is 4.17 0.21. And we have this special cart character, which I'm going to talk about later in the section. So when you run NPM install, NPM is going to download the latest version of the Given package from NPM registry. And then it will store it inside folder, node, underscore modules. So under this, you can see we have this lodash folder. Here we have a bunch of files. One of these files is package rotation. So every node module has this package rotation, just like our node application. So if you look at the package rotation or lodash, you can see the version is 4.17 0.21. Here you can see some more information or metadata about this lodash package, such as the homepage, the keywords, and so on. Now in the previous versions of NPM, the proper way to install a nod package was like this. NPM install, let's say lodash. We also had to supply the SAF flag. Without this SAF flag, we wouldn't get Lodge in the list of dependencies in package JSN. But this behavior is changed in the recent versions of NPM, so we no longer need to specify the SAFE flag. Only run NPM install at the name of the package, and this will update package dot GSM. And also download the latest version of that package and store it inside of non modules folder. 5. Incorporating Node Packages into Your Project: So we have installed this oh library as a third party node module. Now let's use it. Back in VS code, I'm going to create a new file in the root of this project and call it index dot Gs. Now here, we are going to use our required function, load the Loads module. By convention, we use underscore referring to the Loadh library. Use the required function, and as the argument, we supply Loads. Now this is how the repair function works. When you supply the module name here, first, it assumes that this module is a core module. Now in node, we don't have a core module called lodash. So the required function thinks maybe this loadsh is a file or folder in this project. However, earlier in the course, you learn that in order to reference a file or folder, we use pd slash. If the argument to the required function was pd flash lodash, then the required function would assume that here we have a file called lodash or Js in the same folder. If not, it would assume lodash is a folder, and inside of this folder, we have a file called index or Js. But in this case, we neither have a file or folder called dash inside this product. So the required function moves on to the third step. It assumes that this module we have specified here exists inside the node modules folder. So this is how the required function resolves a module. First just assumes it's a core module, otherwise it assumes it's a file or folder, and if not, it looks for that module inside of the node modules folder, okay? So we have loaded the module, now let's use it. If we head over to lodge.com, you can see the documentation for the Load library. This library includes a lot of useful utility functions. The one that I'm going to use in this lecture is Includes. So this is how it works. We pass two arguments here and an optional argument. The first one is an array. The second one is the value we are looking for in that array. The oftenal argument is the index to search from. By default, its value is zero. And in results, we get a boolean. So let's call this method and make sure we loaded the odes module properly. So Loads dot, and look, here we get intelligence. Includes first, we give it an array, one, two, three, and then a value like two. We get the result and display it on the console. Now, back in the terminal node index start Js, and we get through. So this is how you use third party node modules in your applications. 6. Managing Dependencies Like a Pro: Now here, I have a small exercise for you. I want you to install a nod package called mangos. We use this to store our data in Mongo Ib and we're going to learn about that later in the course. So for now, I want you to pause the video and install this package and then inspect the package darts and pile as well as the node modules folder. So go ahead, pause the video, and when you're done, come back and continue watch. So let me install this package. And PM install mangos. Very easy. Beautiful. You can see our package D JCNFle is updated under dependencies. Now we have two dependencies. You can see mangos, and the current version is 7.4 0.1. Now, let's take a look at what we have inside the node modules folder. Look, we have lots of folders here. Where did they come from? We didn't install any of these libraries. We only installed Mongos and lodash. So these other libraries you see here, these are other knot packages that Mongoose is dependent upon. Now, in the previous version of NPM, we had a different behavior. All the dependencies of a given package were stored inside that package folder. For example, look, here's our lodash package. Let's imagine load dash has a few dependencies. In previous versions of NPM, those dependencies would be stored inside this loada folder inside another folder called node modules. So we had a hierarchy of node packages and their dependencies. However, this created a mess because we ended up same package being installed multiple times. And also in some situations, we would end up with a very deeply nested structure. And on Windows, specifically, there is a limitation on the number of characters that we can have in a path. In recent versions of NPM, this behavior is changed. And now all dependencies of our application, as well as their dependencies are stored under our node modules folder. There is an exception here. If one of these packages uses a different version of one of these dependencies, then that version would be stored locally with that package. For example, let's imagine in our application, we have installed this Bison module. Let's say the version that we are dependent upon is Bison version one. Now at the same time, Mongos has a dependency to this package, but the version that Mongos is dependent upon is Version two. So we cannot have the BSN Version one and Version two stored under node modules. In that case, because our application is dependent upon BSN version one, that would be stored here. But because Mangus requires Bs in Version two, that would be stored here in the Mangus folder. So here we would have the node modus folder, and inside of that folder, we would have BSN version two. 7. NPM Packages & Versioning: Best Practices: And we have quite a few folders inside of the node modules. Now, this is a very simple application. In a real world application, we are going to have lots of folders here, and the size of the node modules folder will grow significantly. In fact, in a real world application, your node modules folder maybe a few hundred megabytes. When checking your source code into a source control repository, you don't want to include this folder because every time someone checks out your code from the repository, they have to wait for hundreds of megabytes of data to be downloaded. Same is true if you want to copy this source code from one machine to another. Let's say you want to give this to your friend by email or via Dropbox. You don't want to send all the content of the node modules folder. Now you might ask. But what about our application dependencies? Our application is not going to work without these dependencies. Well, the good news is that all our dependencies are stored here in packaged JCNPck. So currently, we have these two dependencies, and we have NPM. So we can easily restore these versions of these dependencies on any machine. Now to simulate this, I'm going to delete this node modules folder. So delete. Okay, it's gone. The back in terminal, we can restore all these dependencies by running NPM install or I. So NPM looks at our package Jason, and then it will download all those dependencies from NPM registry. Now you can see our node modules folder is back along with all these dependencies. So that's why we should exclude this node modules folder from our source control repository. Now, assuming that you are using Git, let me show you how to exclude this folder from G. If you use a different source control management tool, the steps are different, and I leave it up to you to figure out how to do it. B in terminal, I'm going to initialize a Git repository in this folder, so get in it. So here we have a Git repository. Now if we run git status, you can see these files that we need to add to our Git repository. So index or Js node modules, package, dashlogt JCN which was created by NPM when we installed an NPM package. This is purely for NPM to do its job. We don't have to worry about this file. And finally, we have package dot GSN. Now we want to exclude the node modules folder here. To back in Vuse code in the root of our folder, I'm going to add a file with extension, Get Ignore. But this file doesn't have an Name. The extension is G Ignore. And here we can list all the files and folders that should be excluded from our Git repository. In this case, node modules with a slash to indicate that this is a folder, save. Now back in the terminal, if we run git status one more time, we can see that we no longer have the node modules folder here. Previously, we had node modules, but now we don't have that. Instead, we have gitignore. This is how we exclude the node modules folder from our Git repository. And finally, we run git add Period, and Git Comet DM with a message, my first 8. Semantic Versioning Explained: Earlier in this section, I mentioned this character which we call the carat character. What does this mean? Well, in order to understand this, first, we need to understand semantic versioning, which is also called SEM work. In semantic versioning, the version of a note package has three components. As you can see here, with three numbers four, 17 and 21. First number is what we call the major version. The second one is what we call the minor version. And the third one is what we call the patch version, a patch release, which is used for bug fixes. Let's say tomorrow the developers of lodash find a bug in the current version 4.17 0.21, then they will fix a bug and release a new version, and that version would be 4.17 0.22. So when they fix a bug, they will increase the patch version. The minor version is used for adding new features that don't break the existing APA. So if the lodash team add a new feature without breaking the existing APA, they would increase the minor version so that version would be 4.18 0.0. Zero because in that version, they haven't found a bug yet. So that version could be unstable. And as fine and fixed bugs, they will increase the patch version. Okay? And finally, if they add a new feature that could potentially break the existing applications that depend upon this version of low dash, then they will increase the major version. So the next major version would be 5.0 0.0, okay? So this is what we call semantic versioning or SEM or. Now, this carrot character that you see tells NPM that we are interested in any version of Lodash as long as the major version is four. So if there is a newer minor or patch version available, we would be interested in that package as well. So let's say today when we are starting this project, we are using this version 4.17 0.21. Six months later, somebody checks out this code from our kit repository. Then they restore the packages using NPM install. At that time, if there is a new version of lodash available, as long as it's version four, so there are no breaking changes. There are no major changes, then that new version will be downloaded and installed inside of the node modules port. Another way, another syntax to specify this version without using the carat character is like this, four point X. So these two are equal. Now, sometimes in some real world applications, instead of the card character, you may see Tilda, Tilda, okay? And this means we are interested in any versions as long as the major version is seven and the minor version is four. So the alternative syntax to specify this version is like this, 7.4 point X. So if there's a newer patch release available, we will be interested in that version as well. So you can see that the carat and the Tilda characters help keep your applications up to date with the latest releases of these dependencies. However, sometimes this can cause issues in the real world. For example, let's say Mongoose releases a new version that is 7.4 0.2. They may fix a bug, but they may break something else that may interfere with your application. In that case, you want to make sure that you are using the exact same version. So if someone checks out this code from the repository three months later, you want to make sure they have the exact same version of mangos that was used on day one. In order to do that, you simply remove this Tilda or carat character. So next time you un NPM install, you would get this exact version. So this is how semantic versioning and carrot and Tilda characters work. 9. Keeping Track of Installed Packages: So now you know what this carat character means. It means in this application, we have a dependency may version of Lord Ash, as long as the major version is four. Now in reality, when you restore these dependencies, chances are the version of low dash that is inside of the node models folder is higher than what we have here. How can we know what version is actually installed? Well, there are two ways. One way is to look at the packaged or JCN file for this Mongoose library. So we go to nod modules, find mangos. And here we have package or JSON. We can see version, and this is the exact version that is installed. But you can see this is a little bit tedious because you may want to check the version of multiple dependencies. You don't want to navigate through this folder, give ESA folder, look at package or Jason. Tis. If you want to see the list of all the installed dependencies and their exact version, you can simply run NPM LS. Now you can see the dependencies of your application. So we have Lodash and mangos. 10. NPM Registry Insights: Earlier in this section, I told you that if you want to learn about an NPM package, you can simply head over tonpms.com. And here search for that package. Let's say, mongoose. So here's the mongoose that we are using version 7.4 0.2. On this page, we can see all the metadata about the mongoose package. One of the properties you might be interested in is dependencies. We can see all the dependencies of mangos, but there is a faster way to find all the metadata about this library. So back in terminal, we can run NPM view and specify a package here like Mongoose. Now, what you see here is a package dot JCNFive from Mongoose Library. So you can see some interesting properties like dependencies. You can see mongoose is dependent upon Mongo E B, and this is the version. So in this case, it's dependent exactly on Mongo DB version 5.7 0.0 because here we don't have the carrot or the tilda characters. Also, it's dependent upon Bsel. And here we have carrot. So as long as the version is five, we're good here. If there is a minor available, that version could potentially also be compatible with mangos. Here, you can also see the latest version that is 7.4 0.2, as well as the other properties about this library. Now, this package JCNFle is too big. It was hard to read it. If you are interested only on the dependencies, you can run NPM view mangoes dependencies. So any properties that we have in the package dot JSON, we can add them here. In this case, we are looking only at the value of the dependencies property. Another useful property is versions. If you want to see all the versions of mongoose that have been released so far, we can run NPM view mangos versions. So these are all the versions in this list. This is useful because sometimes you may want to downgrade an earlier version, or you may want to upgrade to a newer version. In this case, you can see the history. In the next lecture, I'm going to show you how to downgrade or upgrade a node package. 11. Upgrading Your Local Packages: In the real world, as you build your application, there might be newer versions of the dependencies you have installed. So you want to quickly find out what packages have been outdated and what are the new versions. To find this information, you run NPM outdated. But before that, as we have installed the latest versions of lodash and Mongos, we first need to downgrade them. So NPM install lodash version 4.11 0.1 and Mongos version 6.9 0.1. Once done, let's run NPM outdated. So here, NPM looks at the versions of our dependencies and compares them with what we have in the NPM registry. Let's see what's happening here. The current version of Mongoose is now 6.9 0.1, but the version that we want is 6.11 0.5. Where did that come from? Well, back in package, Jason, look, we have reference Mongo's version 6.9 0.1. But here we have this carrot character that is equivalent to version six point X. So as long as the major version is six, that release can be used in this application. Now, in this case, the top version six release is Version 6.11 0.5. So that's why we have this value under the Wanted column. Next, we have the latest version. Now, in this case, the latest version has a different major release. So here we have Version seven, but currently we are dealing with Version six. In contrast with Lordsh, both our current version and the latest version, they are both in the same major release. That is Version four. So with NPM outdated, you can see all the outdated packages. If you want to update them, you run NPM update. However, this only works for updating minor and patch releases. In other words, when I run this, you can see that low dash will be updated because both the current and the latest version, they are both in the same major release. So that means there are no breaking changes. But mongoose will not be updated to the latest version because the latest version is one major release away, and it is possible that there might be some breaking changes there. So when we run NPM update, we don't want to accidentally break our application. So let's run this and see what happens. So you can see we have two updates here, Loads is upgraded to version 4.17 0.21, which is the latest version available. However, Mangus is upgraded to the latest major six release that is 6.11 0.5. And that's what we had here under the Wanted column. If you want to update these dependencies to their very latest version, you can do that with a different command line tool. So NPM install G, which stands for Global, and the name of the package we are going to install is NPM Check Updates. Okay, now we have a new command line tool called NPM check Updates. If I run this, you can see all the outdated packages and their new version. So in this case, we have Mongoose 6.11 0.5, but the latest release is 7.4 0.2. Now here you can see this line Runn which is short for NPMcheckUdates, with minus to upgrade package dot JSON. So you can run NPM check updates or the shorter is NCU with minus U to upgrade. So you can see our package dot JSN is updated, and here's the newer version of Mango. But this dependency is not installed yet. We only updated package DtJSEN. So to verify this, look here. I package DoJSN we have Mangus version seven. Now, let's install this dependency. So back in terminal, we run NPM install, and okay, now we have the latest version of Mangus. So if you're NPM outdated, we don't get anything because all our packages are up to date. Similarly, if you run NCU again, you can see all dependencies match the latest package versions. 12. Understanding Development Dependencies: So far, all the dependencies we have installed are application dependencies like mongoose and lodash. So our application needs these dependencies in order to function properly. But sometimes we use dependencies that are only used during development. For example, we have tools for running unit tests. We have tools for doing static analysis on our code. We have tools for bundling our JavaScript code, and so on. These dependencies are development dependencies. And they should not go in the production environment where we deploy our application. So in this lecture, I'm going to show you how to install a dependency and note packet called JSHint, which is a static analysis tool for JavaScript code. It basically analyzes our JavaScript code and looks for potential problems or syntactical errors. So we run NPM install Js int. Now, in order to specify that this is a development dependency, here we supply a flat, Asha save depth. Now back in package dot CJSN you can see Jent is under a different property that is DaV dependencies. So this tells node that this is a development dependency and it should not go in the production environment. Now, if you look at the node modules folder, you can see that Jent is here. So all dependencies, whether they are application dependencies or development dependencies, they are stored inside of the node modules folder. They are only segregated in package CJSN, okay? 13. How to Uninstall Node Packages: So far, you have only learned how to install a note package or how to upgrade it. But in the real world, sometimes you may want to unstall a package because perhaps you no longer require it. So if that's the case, in terminal, you are NPM unstall or UN for short, and then specify the name of the package. Let's say Mongos. Now, in package JSN, you can see mongoose is no longer under dependencies and is also removed from the node modules folder. 14. Handling Global Node Packages: All these packages we have installed so far like Low Dash and JSHint, they are particular to this node project in this folder. But there are node packages on NPM registry that are not specific to an application. These are often Amanine tools that we want to access from everywhere. They are not tied to a specific folder or a specific project. NPM is an example of one of these global packages. It's a command line tool. We can run it from any folder. It's not specific to a given project. Another popular command line tool is angular CLI. We use this to create a new angular project. If you want to install a note package globally, you use the G flag, as I showed you earlier in this section. So NPM install G, and then the name of the package. So earlier, we installed NPM version 8.14 0.0. Now, if you remove the version from this command, you can upgrade your NPM to the latest version. And of course, if you're on Mac, you need to put pseudo here unless you have configured your permissions properly. The explanation of that is really beyond the scope of this section because not all the students of this course use a MAC. Some people use Windows, and I don't want to waste their time with unnecessary materials. But if you simply search for NPM, psudo Mac on Google, there are posts on stack Oflow that guide you how to set up your permissions properly, so you don't have to run pseudo every time. So this is how you install at Global package. Now, all the other NPM commands that you have learned throughout this section, they also work with global packages. For example, if you want to find out all the outdated global packages, you run NPM G outdated. So here you can see all the global packages I have installed. The first one is no more. My current version is 2.0 0.15, but the latest is 3.0 0.1. Similarly, if you want to stall a global package, we run NPM uninstall G, and then specify the name of that package. So this concludes our lesson about working with local and global packages. And this is all about installing, updating and uninstalling node packages from NPM registry. I hope you enjoyed this section, and thank you for watching.