The Express. js Course - Module 8: Modeling Relationships with Mongoose

1. Class Introduction: Welcome back to Module eight modeling relationships with Mongoose. This class is a continuation of the Express JS course series. My name is Shawn Ragunhi and I will be your guide as we dive into advanced database design techniques. I have had the privilege of working on several production grade no JS applications, and I'm excited to share my insights with you. Some of my best work includes designing scalable high performance systems that rely on well structured databases, and today you will learn how to do the same. In this module, we are focusing on modeling relationships in Mongoose, which is essential for any application which interconnected data. Here's what you will learn how to model relationships using referencing and embedding documents, managing arrays of sub documents and working with MongaV transactions. You will also learn about validating object IDs to maintain the integrity of your database. This class is perfect for backend developers who already have some experience with Express Js and Mongo DB. If you have completed the previous modules, you are well prepared for this. You'll need no Js, Express, Mongo DB, and mangos installed, along with a basic understanding of crowd operations. Understanding how to model data relationships is key to building scalable, maintainable and efficient applications. By the end of this module, you will not only know how to structure complex relationships in Mongadib but you will also gain hands on experience in designing APIs that work seamlessly in real world scenarios. In this module, we'll work on two projects. First, you will build the car's API to manage cars in our system, learning how to embed and reference documents effectively. Next, you will develop the rentals API, which includes creating and retrieving rentals, validating object IDs, and handling transactions to ensure data reliability. These projects will give you a practical experience implementing advanced database techniques. So this module is packed with hands on lessons and projects that will elevate your backend development skills. Let's get started, and I will see you in the first lecture. 2. Data Relationship Modeling with Mongoose: In all the examples we have looked so far, we have worked with single self contained documents. But in the real world, the entities and concepts that we work with, they have some kind of association. For example, you can have a course object or a course document. And, of course, this course has an author, but an author is more than just a name. It's more than just a simple string. We might have a collection of authors where we store authored documents, and in each authored document, we can have properties like name, website, image, and so on. So in this lecture, I'm going to talk about how to work with related objects. Basically, we have two approaches. One is using references, which we call normalization, and the other approach is using embedded documents, which we call denormalization. So let's see how these work. With the first approach, we should have a separate collection for storing our authors. So we can have an author object like this. Here we have all kinds of properties, and then we'll have a separate collection where we store course objects like this. So here we have a course object. We set the author to the ID of an author document in the author's collection. Here we are using a reference. Now I need to clarify something here. In relational databases, we have this concept of relationship which enforces data integrity. But in Mongo DV or no SEQL databases in general, we don't have a relationship. So even though I'm setting the ID of an author here, there's actually no association or relationship between these two documents in the database. In other words, I can set this to an invalid ID, and Mongo DV doesn't care about that. Now, we could take this example to the next level. Let's say a course might have multiple authors. So instead of the author property, we could have authors, which we set to an array of references. So here we'll store multiple IDs. Now, for simplicity, let's just work with a single author. So I'm going to delete this part. So this is our first approach, which involves using references. There is another approach. So instead of having a separate collection of authors, we can embed an author document inside a course document. So here we can have a course object or a course document. In this document, we have the author property and we set this to an object. So here, we'll have all the properties of an author. So we are embedding a document inside another document. Okay. And this is what we call denormalization. Now, if we have never worked with nose equal databases before and you come from a relational database background, you may think that the first approach is the Vu. But that's not necessarily the case when working with nose equal databases. Each approach has its strengths and weaknesses. What approach you choose really depends on your application and its querying requirements. So basically, you need to do a trade off between query performance and consistency. Let me explain what I mean by that. With the first approach, we have a single place to define an author. If tomorrow I decide to change the name of this author from hewn to hewn, there is a single place that I need to modify, and all the courses that are referencing that author will immediately see the updated author. So with the first approach, we have consistency. However, every time we want to query a course, we need to do an extra query to load the related author. Now sometimes that extra query may not be a big deal. But in certain situations, you want to make sure that your queries run as fast as possible. If that's the case, you need to look at the second approach using embedded documents. So with this approach, we can load a course object and it's author using a single query. We don't have to do an additional query to load the author because author is inside the course object or the course document. However, with this approach, if tomorrow I decide to change the name of this author from hewn to Shevin, chances are there are multiple course documents that need to be updated. And if our update operation doesn't compete successfully, it is possible that we will have some course documents that are not updated. So we'll end up with inconsistent data. So the first approach gives us consistency. The second approach gives us performance. That's why I told you that you need to do a trade off between consistency and performance. You can't have both of them at the same time. So in every part of your application, you need to think about the queries you are going to execute ahead of time, and you will design your database based on those queries. So these are the general principles. Now we have a third approach. Which we call the hybrid approach. For example, imagine each author has 50 properties. We don't want to duplicate all those properties inside every course in our database, so we can have a separate collection of authors. But instead of using a reference here, we can embed an author document inside a course document, but not the complete representation of that author. Perhaps we only want the name property. So with the hybrid approach, our database will look like this. So we have a collection of authors. In this collection, we'll have author objects like this. We have the name Chevan, and here we'll have 50 other properties. Now we will have a separate collection of courses. In this collection, we'll have course documents like this. So author, we set this to a document, and in this document, we'll have only two properties. One is ID. That is a reference to an author document. We also have nail. With this approach, we can quickly read a course object along with this author, so we can optimize our query performance, but we don't have to store all those properties of an author inside a course document. Now this approach is particularly useful if you want to have a snapshot of your data at a point in time. For example, imagine you are designing an e commerce application. There we will have collections like orders, products, shopping carts, and so on. In each order, we need to store a snapshot of a product because we want to know the price of that product at a given point in time. So that's where we will use the hybrid approach. So once again, which approach you choose really depends on the application you are building. There is no right or wrong. Each approach has strengths and weaknesses. Over the next few lectures, I'm going to show you how to implement each of these approaches. 3. Referencing Documents in Mongoose: In this lecture, I'm going to show you how to reference a document in another document. If you want to de along with me, download this file. I have attached to this lecture. In this file, populate the Js. We have two models. The first one is author with three properties, name, bio and website. The other model is coors with one property name. In this lecture, we're going to add another property author, and there we will reference an author document in our database. We also have a few helper functions. One is create author. The other is create course, and the last one is list courses. All these functions are similar to what we have seen earlier in this section. So there is nothing new here. Now, before getting started, I want you to go to Mongo DB Compass and delete the playground database. We want to start on a clean canvas. So type in playground here to delete this database. Okay, now at the bottom of this file, you can see, we have a call to create author function, create an author called Chewn with these properties. So open up the terminal and run node, opulate JS. So we created one author, and here's the idea of this author. Okay, now copy this back to the code. Let's commit out this line, create author, and enable create course. So here we want to create a course called node course for this new author. So you pass the author ID as a string. Now, if you look at the implementation of this function, here we create a course object with two properties, name and author. Now, save the file. Back in the terminal. Let's run this program one more time. So we created this course object. Here's the idea of the course, the name of the course, but we don't have the author. The reason for this is because we defined this course model, we only added the name property. So when saving a course object, only the properties that you have defined in your model will be persisted in the database. So here we need to add another property, author. We said this to a schema type object. Type of this property should be object ID. So we use mongoose schema, dottypes object ID. Also, we set a property called ref and here we add the name of the target collection that is so in this author property, we will store an object ID that references an author document. But once again, we don't really have a proper relationship. Here, we can store a course with an invalid author, and MongoDB doesn't complain about that. So we have modified our model now back in the terminal node, opulateJs so here's our new course object. You can see we have the author property now back in MongoDB Campus. Let's look at this playground database. Here's our courses collection. So you can see we have two documents. The first one doesn't have an author, but the second one has. So here we have an object ID that is referencing an author. 4. Mastering Population in Mongoose: Now it's time to get all the courses along with their authors. So let's disable, create course function, and enable this function list courses. So in this function, we are calling the find method. We get all the courses, and we are selecting only their name property. Let's say the changes, Bag in the terminal node, opulatetJs. Look, we only have underline ID and name. Now here, if I add author and run this program one more time, look, our second document has an author, but we are only getting the reference or the object ID. In a real world application, we want to load this author document so we can display its name. That's where we use the populate method. So here a fine, we can call populate as the first argument, we specify the path were given property. So in this case, our property is author. And because earlier when defining the course model, let's have a look here. So here's our course model. When defining this, we set author to be an object ID, and we reference the author collection. So when we load a course object and populate the author property, Mongoose knows that it should query the author's collection in Mongo DB. To back in list courses funtion, we simply call populate and pass author is the name of the target property. Now let's see what happens. So back in the terminal, let's run this one more time. Okay, look, our first document, our first course doesn't have an author, so we don't get anything. But our second document, our second course has an author, and here we have a complete representation of an author document. Now in a real world application, an author can have multiple properties. Perhaps when showing the list of courses, we don't want to get all those additional properties. We just want to get the name property. So back in the board, when calling the populate method, as a second argument, you can specify the properties that you want to include or exclude. So we want to include only name. Now back in the terminal, let's run this application one more time. So this time, our author property is an object with only two properties ID and name. Now we can also exclude this underline ID property. So back here, we add dash to exclude the property, and the name of the target property is underline ID. Say, back in the terminal, let's run this one more time. And now our author property is an object with only one property. Name. It's also possible to populate multiple properties. For example, let's imagine each course has a category and a category references a category document. So here we can call populate again at category, and oftenly pick only the name property of each category document. Now, let me show you one last thing before we finish this lecture. Earlier, I told you that in Mongo DB, we don't have relationships for data integrity in our database. So here in our courses collection, it is possible to set this author with invalid document. So let's change this two instead of two C one, C. We don't have an author with this ID in this database. You see, MongoDB is perfectly fine with this operation. Now back in the terminal, let's run this program one more time. See now our author is null because there is no author with a given ID in our database. 5. Embedding Documents with Mongoose: In the last lecture, you learned how to use the references to relate documents. In this lecture, we're going to look at another technique that is embedding documents. So if you want to quote along with me, download this file, I've attached to this lecture, embaring dot js. So here we have this author schema exactly like what we had in the last lecture. It has three properties, name, bio and website. We have this author model. Below that, we have the course model. Now here, we don't have the author property, and that's what we are going to add in this lecture. So we add the author property. Now in the last lecture, we set author to an object idea. So let's take a look. Here's our course model from the last lecture. Look, here's the author property. We set the type of this property to an object ID and reference the author collection. In this lecture, we're going to embed an author document directly inside of a course document. So we set the type of this property to author schema. That is defined here. Okay, that's the only change we need to make. Now let's take a look at the create course function. It takes a course name and an author, initialize the course and save it exactly like the fore. So at the bottom of this file, we have a call to this function to create a new course with this author. So before going any further, let's open Mongoi B Compass and delete this database. We want to start on a clean canvas to make sure we are on the same page. All right. Beautiful. Now in the terminal, let's run node embedding dot js. So here's our new course document. You can see author is an object with two properties, ID and name. So this is an embedded or a subdcument. These subdcuments are like normal documents. So most features that are available on normal documents are also available in subdcument. For example, we can implement validation here. We can enforce that. Author name could be required. However, these sub documents cannot be saved on their own. They can only be saved in the context of their parent. So let's say I want to change the name of this author. Here's the course ID. Let's copy that. Here, I'm going to create a new fun ****. LaunctUdate author Ask BrsDGes to Ecodblog. Here, first we need to find a course with a given ID. So we call cours dot Fine BID, pass this course ID. Now await the result and get this course abject. Okay? Now we modify the author. Suppose dot author the name, we said this to Yvan Rebounci. And now we can call force dot C. So we don't have course author dot save. That does not exist. Okay? So let's go ahead and run this function. Update author, and here's my purse ID. Now, back in the terminal, let's run node embedding dot js. Our document is updated. So let's take a look at Compass. Refresh this is our playground courses collection. Here's our course document, other object, and you can see the name properties updated. We can also update a subdcument directly. So instead of quering it first, we can update it directly in the database. So here, I'm going to modify this code and replace fine by ID with update one. Here, as the first argument, we pass a query object. We are looking for a course with this ID, force ID, and the second argument is our update object. So here we use a set operator that you have seen before. We set this to an object, and here we pass one or more E value pairs. So here, to access the nested property, we use the thought notation. Let's say we want to update the name of the author of a course. So we pass author dot name. We set this to Peter Parker. Okay. With this, we don't need to modify this object in memory and save it explicitly. We update it directly in the database. Before we run this module, I realize that I have made a mistake here. This should be update one. All right. Let's run this again. Okay, let's check Compass, refresh here. So here's our author object. And look, name is updated to Io parker. If you want to remove a subdcument, you use the unset operator. Let me show you how that works. We use the unset operator. Now we can use unset author dot name to remove this nested property. Or we can remove this sub doocument as a whole. We need to set this to an empty string. Okay? Now let's run this again. So node embedding dogs back encompass, refresh, and look, we no longer have the author property. Now, as I told you before, these sub documents are similar to normal documents. So here we can enforce validation. We can enforce that. Every course should have an author. But here's the definition of our course schema. If you want to make this author property required, here we need to pass a schema type object. So we set the type to author schema and then require it to true exactly like what we have learned earlier in this section. Or if you want to make a specific property in this authorbdcument required, you apply that validation on authors subdcument itself. So here you pass a schema type object and set the required property to true. In the next lecture, I'm going to show you how to work with an array of subdcuments. 6. Arrays of Sub-documents in Mongoose: So in the last lecture, we added author as a sub document in this course document. In this lecture, I'm going to show you how to change this to an array of sub documents. So first, we rename this property to authors and then change its value to an array of authors schema. Okay. Now here, when creating a course, we also want to pass an array of authors. So I'm going to rename this parameter to authors. Okay. And finally, this is where we are creating a course. So instead of passing one author object, we pass an array. Here's the first author, and here is the second one. Let's say eater. Now, back in compass, I'm going to delete this collection, so we see all the new data without any confusion, okay? Now back in the terminal node embedding dot Gs. Okay, here's our new course document. You can see authors is set to an array with two objects. This is the first author and here's the second author. And if you look in Compass, let's refresh here. This is the course's collection. Here's our cross document with an array of authors. Beautiful. Every author is an object. When you expand that, you see two properties ID and name. Beautiful. Now we can always add an author to this array later on. Let me show you. So back in the code, let's create a new function, constant add author Async. So we pass a course ID here and an author object. Now here we need to find a pose first. So constant force. We set this to await forstFineBy ID. And here we pass this course ID argument. We have the course. Now, force authors, as you know, is an array. So we can call the push method to push this author object in this array. But our changes are only in memory. They are not saved to the database, so we need to call force.ca. Now let me delete this and call add author. Here, we need to pass the course ID. So I'm going to go back encompass and copy this course ID. So paste it here and then pass a new author object with the name, let's say, flash. Back in the terminal, let's run this again. Okay, our changes are saved to the database. So let's verify them. I'm going to refresh here. Here's the authors. Look, we have three objects in this array. And here's our new author. Removing an author is very similar. So back in the code, let's add a new function, constant remove author. Async here we need two parameters, purse ID and author ID. So first, we load the course just like before. Now we go to cours dot authors. Here we have a method called ID, and with that, we can look up a child object by its ID. So you pass this author ID that gives us the author object. Now we can call the Delete one method on this object. And finally, save the course. Okay, so let's call this new function, remove author. Let me duplicate this line, remove the second argument and change the name to remove author. Now we need an author ID. So back encompass, Here is the idea of flash. So I'm going to paste that here. Okay, let's run this. Node, embedding dot js. Beautiful. Let's take a look at our data. So I'm going to refresh this page. Authors, look, we have only two authors now. Flash is gone. So this is how we work with subdcuments. 7. Setting up MongoDB for Transactions: Welcome back. In this video, we are setting up MangaibRplica sets, which are essential for running transactions in Mangaib. This setup will ensure Mangaib is in replica set mode, allowing you to work with multi document transactions. In the next lecture, we will be diving into performing transactions using mongoose. So let's lay the groundwork today. So we'll go over two options. Number one is single node replica set on local Mangaib server. Now, this approach is great for testing and development. And number two is replica sets with Mangaib atlas, which is ideal for production since it's cloud based and fully managed. Let's start with the local setup. So we will configure Monger I B to run in replica set mode on your local machine with a single node. This setup allows us to develop and test transactions locally. So we need to configure Manga IB or Replica set mode, and for this, we need to edit Mongo DB's configuration. Locate the configuration file. By default, this file is located in the trip program files, Mongo DB server. Then the Virgin folder, which is 8.0 in my case, and then the bin folder, and here it is mongod dot conf. Now open mongod dot cfg with a text editor like notepad or Visual Studio code and locate or add replication section like this. Here it is replication RUPL that name. We set this to RS Zero. And save. You may get an error because you are not in administrator mode. So here is a prompt. Just try as an administrator and our file is saved. You can use any name for the replica set, but R zero is a common choice. Now, we will restart Mongo DB with the updated configuration. So open Power Shell as administrator and run lap service, ah, A Mongo DB. Now we'll restart it. So start service, h A Mongo DB. With Mongo I B now in replica set mode, we will initialize it. For that, we need to install Mongo DB shell. So Mongo B 6.0 and later versions provide Mongo Asch as a separate package. It is no longer bundled with the Mongo ib server installation. If you have it installed already, that's good. If not, then visit the Mongo Debe Shell download page. Select your operating system to download the latest version. I would prefer the MS installer for a cleaner installation. Although it's up to your choice, you can choose ZIP as well. Open the MSI and install it. Click next, and note that you have to copy the installation path. So copy it next and install. Once installed, add MongaSH directory to the system path. So let's open environment variables. You have to look for system environment variables. Here, click environment variables. In the system variable section, like path and click Edit. Add the path that you copied and click Okay to save changes. So one more time. Okay. Okay. Great. Now we will restart PowerShell to apply changes. After adding Mongo SH to the path, go to the PowerShell and run Mongo SH. By default, Mongo SH connects to Local host port 27017. If your Mongo DV server is running locally on the default port, it will connect automatically. Now, initialize the replica set by entering Rs dot, initiate. I'm getting this error message because I've already initiated replica set before. So you can ignore it. You'll get the correct message. Finally, we will check the status and ensure the replica set is running. So Rs dot status. Beautiful. Everything is set up correctly, and MongoDB will now be running in replica set mode, and we are ready to start working with transactions locally. But this is not then. For production environments, MongoDB Atlas is an amazing choice because it automatically sets up multinode replica sets, which means transaction support is ready to go right out of the box. So first of all, head over to MongoDB Atlas website. If you don't have an account already, sign up, it's quick and free. If you already have one, just login. Once you are logged in, click New Project in the Atlas dashboard. Now name the project. I'm going to choose Fair Wheels to fair Wheels and then click on next. Here, create the project. So our project is created. We have to create a cluster. So click on Create then here, if you're testing things out, the free zero tier is perfect. But for production, you'll want to pick a higher tier for more power and features. Once that's done, name your cluster. So the default is cluster zero, but I'm going to choose pair wheels. Now, pick a Cloud provider, AWS, Azure or Google Cloud and select a region that's closest to a users. This helps reduce latency and improves performance. I'm going to choose AWS and region as Mumbai and then create deployment. And here our cluster is created. So here we have to create a username and password, which we are going to use in our connection string to make sure that you copy the password and the user name. And then create database user. Next, you have to choose a connection method. So, here choose drivers. Make sure the driver is near Js. Version is 6.7 or later. And as you can see, fair wheels is provisioning. So our database is provisioning and it will take some time. In the meantime, let us secure our connection string. Go to Network Access and add your IP address. This allows your app to connect to the cluster. If you are on a dynamic IP, you can add a wide range or allow access from anywhere, but be careful with this in the production. No, that's it. I go back to clusters. It will take a few minutes. I'm going to fast forward, and here it is. So click on Connect, and then again, drivers, make sure near Jasn 617 or later is selected. And here's our connection string. It will look something like this. Manga V plus SRE Colen slaLsh and the username and the password. So this is not your password. You need to make sure that you replace DVPassword with a password for your user name. Copy this string and replace the connection string in our application with this one. So make sure that you name the database after the dart net, slash fair wheels. This will make sure that your database is connected, or if there is no database of named fair wheels, then it will create. Also make sure that you replace the DB password with a password that you copied that you generated randomly, and that's it. With Mongerib atlas, replica sets are enabled by default. So a cluster is ready to handle transactions without any extra steps. It's that simple. In the next lecture, we will build on this setup by diving into transactions using mongoose. 8. Performing Transactions using Mongoose: In Mon Vov, we have the concept of transactions, which basically means a group of operations that should be performed as a unit. So either all of these operations will complete or change the state of the database, or if something fails in the middle, all these operations that have been applied will be rolled back, and our data, this will go back in the initial state. MovadB transactions are really beyond the scope of this course. But if you want to learn more about it, let me show you the right page in the documentation. So search for Mongo DB transactions. Okay, so here transactions Mongo DB manual. This document clearly explains how to perform distributed transactions using a real wild example. Now, in this lecture, I'm going to show you transactions using mongoose. But internally, it implements this transaction using the Mongoiw transactions. Now back in the code here in rentals dot js on the top, first, we need to load Mongoose. So constant mongoose. We set this to require Mongoose. Now, it has a start session method that we need to call right here. So await Mongos dot start session and store it in session. So pons session. In our post method, this is where we create a rental object. Now we are no longer going to create this rental and update the explicit. Instead, we are going to start a transaction. Here, we initiate a transaction by calling session dot start transaction. All mangos queries related to the transaction pause a session option to associate them with the transaction. So we have two Mongoi operations here. Rental dot save and car dot CV. So you want to save this new rental to the rentals collection. So you call save and pass the session. Okay, so this is our first operation, saving the new rental. Now, as part of this unit, we also want to update the car's collection. So we pass session here and abate this operation as well. So these are two operations. After you chain all these operations, and finally, you need to call await session dot Comtransaction. And session dot session. If you don't code commit transaction, none of these operations will be performed. All right. Now it's possible that something fails during this transaction. So we need to wrap this in a try cache block. So here I'm going to add a trib block. I'm going to move all this code inside the tri block. So this is for our success scenario. Now if something fails, we should catch an exception here and return a 500 error with the clin. So response dot status 500, which means internal server error with a message lie, your vehicle is not bugged. Something went wrong. But before this, we need to abort the transaction. So await session dot aboard transaction and then session dot session. Now in a real world application, at this point, you want to log this exception. So later you can come back and see what went wrong. We are going to have a separate section in the course about error handling and logging. So for now, let's don't worry about this. So back in Mongo DB Campus, I'm going to delete the rentals collection rentals. Now back in the terminal, t's run the application. No more. Our application is running. Beautiful. Back in our database, look at our cars collection. So here we have a car and a number in stock is five. I'm going to create a new rental. Then we are going to come back here, and this number should be f. So let's go back to Postman. So I'm going to send a post request to our rentals endpoint, and here in the body of the request, I have a valid customer ID and car ID. So let's send us. Okay, here's our response. Beautiful. So this is our new renter object. You can see you rent it on ID, customer and car. Now back in the database here in the car's collection, I'm going to refresh this list. Look, number in the stock is now four. So this verifies that our transaction completed successfully. Now I've got a question for you. So here we create this rental object. We only set customer and car. Then we pass the session to the Mongo Di Boperations and send this rental object to the client. So in this code, we didn't set the ID or rent it on properties. But in the body of the response, you can see both these properties are set. So how did this happen? Nowhere in this code after we ran this transaction, we reset the rental object. So how did we get the ID and date properties? Perhaps you expect Mongo DB to set these values for us. But actually, no. In Mongo B, we don't have these default values. We define them in our Mongoose schema. So when we create a new rent or object, Mongoose knows the schema for this object. It looks at various properties and sets the default values. Same is true for the IE property. So more accurately, Mongo DB doesn't set this. This property is set before we save this document to the database. I didn't tell you this earlier because I didn't want to confuse you with too many details. There is more to ID that we are going to cover in the next lecture. 9. ObjectID in MongoDB: In this lecture, we are going to look at object IDs in Mongo Di Bi. So you have noticed that when you store a document in Mongadib, Mongo Debi sets the value of the ID property to a long string like this. So here we have 24 characters, and every two characters represent a byte. So essentially, we have 12 bytes to uniquely identify a document in Mongo Deb. Now out of these 12 bytes, the first four bytes represent a timestamp, and that is the time this document was created. Later, I'm going to show you how to extract this timestamp from this object ID. So with these four bytes, you don't have to create a separate property in your document like created at, because this timestamp is included in the object ID. By the same token, if you want to sort your documents based on their creation time, you can simply sort them by their ID property. The next three bytes represent a machine identifier. So two different machines will have two different identifiers. The next two bytes represent a process identifier. So if we generate two object IDs on the same machine, but in different processes, these two bytes will be different. And finally, the last three bytes represent a counter. If you are on the same machine in the same process at the same second, but generate two different documents, the canter Bites will be different. So with these 12 bytes, we can uniquely identify a document in Mongadib. There is a very, very, very low chance that we will generate two object IDs that are the same. Let me show you how that can happen. So you know that in one byte, we have eight bits. In each bit, we have either a zero or one. So how many numbers can we represent in one byte or eight bits? Well, that is two to the power of eight, which is 256. So with one byte, we can store 256 different numbers. Now I told you that the last three bytes represent a counter. This is like the counter that you have probably seen in the SQL server, MySQL, and other databases. So an auto incrementing number like one, two, three, four, and so on. So how many numbers can you store in three bytes? That is two to the power of 24. That is 16 million. So if at the same second on the same machine in the same process, we generate more than 16 million documents, this counter will overflow. And that's where we may end up with a two documents with the same object ID. But you can see that this is a very unlikely scenario for most applications out there. All I want you to know is that this object ID is almost unique, but not 100%. Now, you might be curious why we don't have a mechanism in Mongo Dew that guarantees uniqueness. For example, in database management systems like SQL server or MySQL, in E table, we have an auto incrementing number that guarantees uniqueness. Next time we want to store a course record in our database, the idea of that course will be the idea of the last course plus one. This approach guarantees the uniqueness of these identifiers, but it hurts scalability in Mangaib. This ID that we have here is not generated by Mongo DB itself. It's actually generated by Mongo DB driver. So we have this Mongo Dew driver that talks to Mongo Div. So this ID here is generated by the driver, and that means we don't have to wait for Mongo Dew to generate a new unique identifier. And that's why applications built on top of Mongo DB are highly scalable. We can have several instances of Mongo DB and we don't have to talk to a central place to get a unique identifier. Driver itself can generate an almost unique identifier using these 12 bytes. So when we build an application with Node and Express, we use mongoose. As I told you before, mongoose is an abstraction over Mongo DB driver. So when we create a new object ID and new document, Mongoose talks to Mongo DB driver to generate a new ID. Now, you can also explicitly generate an ID if you want to. Let me show you. I'm going to clear all this on the top. Let's load Mongos require mangos. Here we can create a new object ID. So let's set this to new Mongos dot types dot Object IDE. And then let's log this on the console. And here in terminal, let's run this program that is node object ID dot js. So look, we have a unique identifier, and we didn't store anything in Mongoib. We generated this object ID in memory. Now I told you that the first four bytes represent a timestamp. So this object ID has a method called GTs Stamp, save back in terminal. Let's run this again. Look, this is the time I generated this object ID. We also have a static method on this object ID class for validating object IDs. So mangos dot types dot Object ID dot is valid. So I can pass a string here, 12, three, four. Obviously, this is not a valid object ID. So when we log this on the console, we should get false. Console dot log is valid. Let's run this again. And here's the result. So now that you understand object IDs in more detail, in the next lecture, we are going to get back to our Fair Wheels application and make a few changes to make our application better. 10. Validating ObjectID using Joi: So back in our Fair Wheels application, I've got postman open, and I'm going to send a post request to our rentals endpoint. And here in the body of the request, we have a valid customer ID and car ID. Now, let me show you what happens if we change this object ID to a value like 1234. So this is not a valid object ID. Let's send this request. You can see we got the status 500, which is internal server error. However, this is not the case, as we have sent an invalid customer ID. As I told you before, we have a separate section about error handling and logging in the course. So let's not worry about this part. For now, let's log this error on the console to see what exactly happened there. So back in the code here in the cache block, console dot log per let's send the request again. Now, if you look in the terminal, cast to object ID field for value one, two, three, four, type string at path underline ID for model customer. This message may sound a little bit too technical or confusing, but I'm going to make it really simple for you. So here we are talking about this model customer or customer class. We are talking about the ID property. In Mongoose, you see the word path because path can represent a chain of properties. For example, a customer can have address, and address can have street. So that's why we use paths. Now here, Mangus is complaining that it could not cast the value 1234 to object ID. Obviously, because 1234 is not a valid object ID. But the issue we have in our implementation is that, first of all, we are getting a generic response here, and we are getting this error in terminal. In this situation, when we send an invalid customer ID, we should get a 400 error. That's a bad request because a server cannot fulfill this request. So back in RentosTrgs this is the handler for creating a new rental. On the top, we are validating the request, and this ensures that in our request, we have a customer ID and a car ID. But it doesn't care if these values are valid object IDs. That exception was thrown on this line on line 21. When we try to find a customer by ID, if you pass an invalid object ID, that's when we get that exception. So one way to fix this problem is like this. If Mangus dot types do object ID that is valid. So we pass request body dot customer ID. Now if this is not valid, we're going to return the 400 error. So status 400 with a message like invalid customer. And then we have to repeat the same for validating the car ID property. However, this is a bad implementation. This is a bad approach to solve this problem because earlier, we defined this function to validate our request. So this logic really belongs to this function. So in this function, we want to make sure that customer ID is a string. It has a value and it's a valid object ID. So if the input is in the right shape, then we go to the database to find that customer. Okay? So I'm going to delete these two lines from here. Let's go back to our validate function. So that's in rental dot JS. Here's our validate rental function. Now here we need to add custom validation because we need to talk to mongoose. We need to call is valid method of object ID type. Now, extending this validation is a little bit complex, and you don't want to repeat that every time you have a validate function. There is actually an NPM package for adding support to validating object IDs, enjoy. So back in the terminal, let's install Joy Object ID. So not that, the current version I'm using is version 4.0 0.2. Now, back in the code here in rental dot Gs, on the top, we need to load this module. So require Joy Object ID. Now, this returns a function. We need to call this function and pass a reference to this Joy module. So we pass Joy here, and by the way, you don't have to memorize this. You can simply look at the NPM documentation. Another result of this is a function, so we can set joy dot Object ID to this function. So Object ID is a method on this Joy object. Now, back to our validate function, so here, I'm going to change the definition of this customer ID from joy dot string to joy dot object ID. That's the method that we defined on the top of this module, right? And it's also required. We are going to make the same change for the car ID property. So object ID. Now, back in the terminal, let's run the application again. I'm going to send a request with this invalid customer ID. Same. Okay, look, we got a bad request. Customer ID with value 1234 fails to match the required pattern. And if you look in the terminal, C, we no longer get that cast to object ID failed error. Now, there is a much better implementation for this approach. So back in rental dot JS, it is likely that we are going to use this method in other places in our application, like in the car module or the customer module. We don't want to redefine this object ID method in every module. So on the top of the file, this is where we define the object ID method in the joy object. I'm going to move this from here to index dot Gs. So we load it once and we can reuse it everywhere in our application. So let's get this and go to index dot JS. On the top, let's add this line here. We should also load joy. So constant joy. We set this to require joy. Okay? So that's a better implementation. There is one more place we need to modify here. So in our models folder, look at car dot js. So when creating a car, we need to pass a valid company ID. So I'm going to replace this with joy dot object ID. Now, one last change before we finish this lecture. So in our routes folder, let's take a look at cars dot JS. Here's the handler for creating a new car. So here we create a new car object, save it to the database, and then return it. Now, in this implementation, I'm resetting this car after saving it to the database. This was purely to demonstrate that this save method returns a car document. And also, I didn't want to distract you with too much detail about how object IDs work. So now that you know, object IDs are actually generated by MongoDB driver, not Mongo DB database. You know that when we create a new car object, Mongoose talks to MongoDB driver and sets the ID right here before saving this car to the database. So technically, we don't need to reset this car in order to return that ID to the client. So we can remove that and change card from a variable to a constant. The same principle applies when creating a new customer and a new company.