R Programming Bootcamp for Data Science and Machine Learning

1. Introduction: Hello and welcome to the class R programming boot camp for data science and machine learning. In this class, I will be teaching you add programming, which is the second most popular programming language that is being used in machine learning and data science field. The first bin, Python. So if you want to learn R programming and you want to excel in your carrier in data science and machine learning, then this is the right class for you. I am Sunil, your teacher for this class. I have years of experience working software development companies and technology analysts for last 12 years and past six years I've been teaching people on data science and machine learning topics. So I will teach you programming from the very basics to the very advanced topic. And we'll see how we can use our programming lawless to analyze the data, to visualize the data and how we can use the R programming to make our data science and machine learning models to be used in machine learning and data science field. So if you're interested in learning and programming for data science and machine learning, then this is the right class for you. So if you're interested enrolling into this class, and I'll see you inside the next lecture. Thank you. 2. R Installation: Hello and welcome. So in this lecture, we are going to see art installation process. And at the end of this lecture, we will be able to run our programs inside our console. So let's get started to install R on our machine. We need to go to the website. Cran gland duct are hyphen project. Dark watching. Cnn dot are hyphen project or towards you, which is the official website up our programming. And you can see comprehensive R Archive. When you come here, you may see a different pace if we're coming at a different time after a few months. But more or less, you will see the Comprehensive R Archive Network here. And then here you will see download and install R. And here you will see the different operating system opsins that you can download R4. So here we are on the Windows machine. So I'll go with that download R for Windows. If you're on Linux or if you're using macOS, you can go to the corresponding operating systems. So we need to click on the download R for the respective Operating System from my kids. I'm clicking on the windows. Here. You can select install R for first-time. Okay? And here we will go with the download R for dot or dot for Windows. Click on that or not. Or hyphen four dot 0, n2 hyphen green dot EXE file will start downloading. It's the 83.6 MB file. And my internet is bit slow today, since morning. So it is taking some time. If you are on a high speed Internet, it may take few minutes and dot EXE file will be downloaded. Just wait for it to get downloaded. And once it is done, where we'll start the stylish and process. Now the dot EXE file has been downloaded. We need to double-click on that. And the prompt will come to install the art. We just need to click on Yes to allow the admin access to the installation process. And once you do that, the histone listen prompt will come like this. And here we need to select the language so you can select the language of yours and click on. Okay. Now we need to know Thompson condition like GNU General Public License Agreement. So you have to read it and then we need to click on the Next. Now you can select the directory. I'm keeping it in the C file by default directory. And we need to click on Next. Thing already exist because it was already there. So I just click to install in any way. Click on it and you can leave it as it is and click on Next. Click on Next. Here you can, if you want, you can select it to not to create the Start Menu folder, okay? If you want, I want to be created. So I'll click on Next. And here you have to select, you just select the data. Next drops are shortcuts so that you can launch the art easily. Whenever you want to write program, you can just click on that extra plot gut and it can get started. Click on Next. And now the odd is getting installed on our system. It may take a few minutes. Hardly. It will take two to 3 min at most. C here. Now the rpart, we're indoors for dot or dot to set of rigid. And it is saying it is finished. So just click on the finish and R is installed on your machine. To verify that odd installed or not, you need to click on the Windows Start and you just need to scroll down to check whether artists, they are not. See here. There's the art folder. And when you click it, click on the art folder, you'll see the two options are I1, I3, a T6, and add such T4. So if you are on Windows seven or Windows ten or higher operating systems, you can go with the four-bit version. Otherwise you can go with the 386. Okay, so I'm under 64 bit machines, so I'll click on this and r will be lost. So this is the R Do Re, and this is the R console. And here we can start writing the heart program. Suppose I love four plus five and it will give me nine. If you write the simple Hello World program for art. So we need to write print. And then in the single quote hello world, and it will print Hello world. So see you now. This is the simple Hello World program in R. So this is how our programs can run in the console. But for this course we're not going to use our console GUI. Instead we are going to use RStudio, which is and IDE for R. And that is more preferable and better option to proceed with the R programming. So in the next lecture, we will start downloading and installing RStudio. You see me in the next lecture. 3. Installing and Exploring RStudio: In the previous lecture, we have downloaded and installed on our machine. And we have seen how to work with the art console, right? Yeah. But we're not going to proceed with this. So we're going to download RStudio for this course. And throughout this course, we'll be using RStudio. Rstudio is an Integrated Development Environment IDE for R programming. And with that, we can do things easily and we can manage things in an organized manner and AD, they help a lot in programming. So we can coat with ease and we can see the results with it. So next week we are going with the RStudio. So first thing first, we need to download RStudio. So to download RStudio, we need to go to the art studio.com. This is the official website of RStudio. And when we reach RStudio website, you will see the various options like products, solutions, customers, resources, and many things. And at the top you can see the download will not click on that. Before that, we'll just see what are the products of level at RStudio. So there is our studio, the premier IDE for R. Then RStudio server is their sign is already there. And RStudio packages are also there. So we're going to use RStudio, the ID, and then we will be using R packages for various uses when we export that data science machine learning, all those things then will be using R packages. Okay, So RStudio and R packages we're going to use. So please click on RStudio. So RStudio is an integrated development environment for art. It includes console syntax highlighting editor that supports direct code execution, as well as tools for plotting, history, debugging and what management. Okay, So all these things we can do with the RStudio, we're going to use the latest release up RStudio one dot three. And there are two versions of RStudio are level that is RStudio Desktop and RStudio server. So we're not going to use the RStudio server. We're going to use RStudio on our deck stops, so we'll click on that. And when you see other storage extra, again, you'll see the two options, open source reason, which is basically a free thing, and RStudio Desktop pro, which is for commercial use and for this artist to read extra pro, we need to pay $995 body. We're not going to use this commercial one. We're going to use for this course, the RStudio free open source version, that is a GPL version three, free licensing. So I'll click on that. Download, RStudio extra. Click on that. And when you click on that, you will come to the harsh to do download phase. And here we can choose this free version and click on Download. And when you click on Download, it will take you to your respective operating system. We're done. So here I'll download RStudio for Windows. If you are on other operating system, you can click on that. Like Coben to macOS, Fedora, BBN, whatever operating system you have, you just select that file hand click on Download. So I will download RStudio for Windows and click on that. And artists, artist studio dot EXE file will be downloading. It will start downloading here. So I'll wait for that. So this RStudio dot EXE file has been downloaded, so I just need to double-click on it. So artists reduce setup wizard has started. So why just need to click on Next Next? You will just need to click on Next Next and it will be done. No need to do anything extra. Thus click on Next, Next, Next. And it will be done in couple of minutes. So the artist studio setup has been done. You just click on Finish and RStudio has stalled completely on your machines or just Now outage to-do app is there, just click conduct. An artist studio will be launched. So Cel ones are loaded. Rstudio, Well, look like this year you'll see the option to write your script. And this is the console where we can see the results of those scripts. And then on this right top corner you can see the environment, history. Whatever the command and scrapped we are running dorsal stream. You can see then that connection and then tutorial. So if you want to learn about particular package or something, you can go here and learn about that. And then here you can see the files on the working directory files will be shown here. Then the plots, when we use plots and all graphs, and if we run something landscape for plotting something as a graphical things, all those plots and all will be soon here. The packages which we use in our script. Those packages will be our level here. So all the packages that are installed on here. And you can just select, and if you want to remove that package, you can remove from here. If you want to learn more about the package, you can click on this link and you can learn more about the package. So here you can see the package name and then the sort description of the package and the wording of the package, and then explore and then delete the mob. The package option is there. If you want to install a new package, you would just need to click on Install and new package name you have to specify and it will be downloaded. Okay, and then here there is a help ops and you want to learn about R and RStudio, or do you want to go to the manuals are large to do manuals, you can go here and get the help on any topic, okay, Then these things we have seen connexins and tutorials and all will be here. And from here, the three dots. You can see when you click on this, you can select the working directory that you want to set for your RStudio. So suppose select our 2020 as a working directory. So I'll just select that. And then we need to click on this more opsin and Set as Working Directory option is that you just need to click on that. So senior set, WD said WE is the command to set our working directory. So C or D colon art printed grantee. This tree has been set to the working directory. So if you don't want to do from here, you can use this command to set the working directory, okay? And then here you can create a new folder, you can delete, you can rename all those options at our level here and here we can just write our script so much by plus five. And this run, you can select this line and you're going to click on Run and this script or this statement will be running and you will see the result here, phi plus pipe then simply you can, if you want to print something, hello, RStudio and you want to click Run, you'll see that there's LTL hello, RStudio. Okay? And in the history you will be seeing all the commands that are being calculated. Okay? So these are the things and this file you can save. If you want to save this file, you can click on Save, and it will be saved in your current working directory. So you can give any name flushed. It will be saved our file. Okay, similarly, if we want to create a new file, you just need to click here and you'll see the artist's group. You're going to create our script file or notebook or R markdown file. We will see what is R markdown file. We will be learning about this agile. Okay? And then if you click, you can create a new project or new directory, or you can go to the existing directory. And here you can see the version control. Okay? So these are the various options of level. If you click on file again, you will see the New File R Script. Mcdonald's, those things you can see here. And gold plots that we have seen here. Same thing. Build, debug profile tools are not. So we'll explore all the options of when and where needed. And we'll explore all those things for now. This is the script writing part, and here we'll see that a judge, and here we can see dark current working directory and plots and all. If we use, you can see your packages can be installed and are huge from here. And here you can see the history and all. Okay, So this is all about RStudio. And now we're ready to start with the R programming with this RStudio Ib. So from the next lecture onwards, we are going to learn our programming. So Steven type the next lecture. 4. Why Learn R: Hello and welcome. In this lecture, we are going to learn why are, why we should be learning R programming language? So to answer this question, we have to go through two things. First thing is, what is r and why we should learn are your regions behind learning our programming? So let's get started with what is r? So r is the most popular language in the world of data science. Data analytics are statics. So it is heavily used in analyzing data that is both structured and unstructured in nature. Nowadays we're getting huge amount of data that is called big data, which is mostly unstructured. And if you want to analyze those big data, you can easily do with the R programming or odd. Odd. Odd. Odd is a programming language and software and warming for statistical analysis and graphics. The presentation and reporting out what's created by Eros yamaka and Robert Jackson man and the University of Auckland, New Zealand. And it's currently they're blocked by our development core team. So R is invented by yamaka and Robert Gentleman. And that's why it's name is odd because in winter name start with our Roth and Robert and they named it based on their name. And it is called our programming from them. Regions to learn R. R is open source and freely available open source software you can plug and play. And if we want to contribute to the art that also you can do. And it's the GNU general public license. So no need to pay anything yourself freely are level. And that is the best reason to use our programming. Cross-platform compatible. So whether you run our program under Windows or Linux or macOS, doesn't matter. It will run seamlessly and it will give you the same result on any of these platforms. Are these highly flexible and evolving artists flexible in nature? And it very much evolving. And it is currently more than 2 million are much more than 2 million users are using our programming industries and domains. Widely used our programming. Like you name the industry and you will find the usage of our programming in that like financial domain they use to detect fraudulent transactions in the telecom domain. They are used the art programming. Too. Far there subscriber profiling. In the biology domain, you'll find the computational biology to perform genome analysis tool. Many, many domains. They're using R. And it's a huge community, as I told you, 2 million users and use developer community. And odd is having more than 10,000. The last packages and lacks of inbuilt functions cat into diverse need. So whether you want to perform a simple sum operation or you want to find the mean of some functions are some numbers. Or you want to do like some more graphical representations of your data. You can do easily with the audit packages and in-built functions. And our packages are great for data manipulation, data visualization, machine learning, data science and statistical modelling, imputation and whole lot of other packages that are labeled to play around with. R is great for visualization. And R packages like GG plot to Gibbs you create visualization so you can visualize data easily with our programming. And many major companies like Facebook and Google, they're using art for their various needs. Odd is equal to language part is that statistical analysis and data science to od is widely used in data science, machine learning, data analysis, data mining, all those things. So od is the go-to language for all these things. And if we want to learn, if you want to go into that data science and machine learning, I feel you start with the programming. Because if you know the basics about programming, you can easily go and learn data science and machine learning concept and you can implement those concepts, those algorithms with odd programming easily. 5. First R Program and Operators in R: Hello and welcome back. So in this lecture we are going to write our first R script file, or first our program, and we'll see some syntax of our programming. Okay, so first thing first, we need to create our file. So I created in the previous lecture first data, I'll just close it. And we just need to click here. And we'll select our script. Or you can, alternatively, you can use control shift and to create our script file. And here, what I'll do, I'll try to save this by clicking here. You can put controllers and give this name first. Broke down. Ok. Dot. Dot, dot is why elections and what programs. Okay, add a script. So I'll just click on Save. So now our first art programs script file is done. Now what I'll do, I'll just start with some variables. Suppose X, X 0s and valuable ear. And I want to assign the value seven or eight. So here we can assign a value using a less than sign hyphen, that's so x. And when we use less than eight, it means that this x object in our programming, everything is an object. So we can call x as n variable, or we can call it as an object. As an object or variable. So x is an object, and we're assigning it a value eight. How can we check that? It is having no value eight now? So you, when you click on Run here, you will see and see here on the global environment Environment tab here, you can see x values x. So x object is getting a while loop. It. We can simply use print command. And we can put x inside. And then when we run this, we'll get the value eight. Okay, we're getting there. Now, what I'll do, I'll simply write a program to add two numbers. So I'll use x equal to eight and I'll assign y equal to nine. And then, so see here what I did. I assigned to x, to y, and then I have used another variable or object yet. And I have assigned x plus y equal to jet. And what I did, I print detail so we will get the x plus y in dead. So jet will we carrying X plus Y, X plus Y plus 917. So we're getting value I, just 17. Similarly, what I'll do, we can use a string as well. The porch, I lose my spring. And I'll assign some string value to it. Suppose I leave my home. Anything, okay? This is the string I'm assigning to my string value. So now the myString object will be getting what my home, right? So what I'll do, I'll just try to print it my string value, and I'll try to run this. Why are getting adder? Because I have not executed this line and our executing this print statement. So we need to execute line by line, okay? And if we want to execute at once, you just need to click on the source. So click on source and everything will be equal to 17 for this print and then my home page for this string value. Okay? So this is the way you want. One more time. Suppose I'm using x. Equal to 910. Print x here. And I run this, I'll get eight. So because this statement I have and you created earlier, so I'm getting X4 eight. The new value, x equal to nine, x equal to ten is not executed. So to execute that, what I need to do, and it will run this line. And if I run this statement, again, I'll get the value n, right. If I run this whole source, what I'll get first, I'll get x plus y 17 because here, till here, x value is eight. But when we proceed in the program, X value is reassigned to ten. And that's why we are getting the X value as ten here. Okay? So the myString value is a string variable, U and X, Y, j, the other numbers, okay? And suppose something, if you want to support it, I don't want to use x equal to ten. I can comment it down by using Hess. Okay, so now this statement will not be executed. So now if I run this whole program, whole script, what I'll get, I'll get x equal to eight. Okay? I'm not getting the sun, the CEO. Now I'm getting x value is x value at 818910. If I remove this commenting, again, I will be getting ten. C x value is coming down. So if you want to come in down something, you can use Hess, that statement will be commented down. Similarly, we can perform all the mathematical operators on the variables. So suppose I want to use four into five. I'll get 25, 20. If I want to, like, use 25/5 and we get five. So all these things we can do, we can perform all the mathematical operators here. Suppose I want to use 25/2, I'll get 12.5. So all these things we can do here, we can also use our exponential things as well. Suppose I want to use three exponential to the exponent rule. We'll get nine, right? So run this and see here will be getting nine. Similarly, if I use 25 exponential 25, how much we get? Let's see. Okay. So this is the big value. I shouldn't have used. 25 exponential two will get 625, I guess. Yeah, it's x 25. So all these mathematical operators we can use here. If you want to use six exponential one, you'll still get six, right? If we make two, will get 36. So all these mathematical operators and calculations we can use in R as usual, okay? Similarly, we can perform subtraction as well. 78 minus three plus six will get 22. So we have seen like Edison subtraction multiplication, exponential, and we have special operator that is modulus. Also. Suppose I'll use 45 and model, Let's will use two times. So 45 modellers. Suppose I use five. So 45 modulus phi will give zero, right? Yeah. And if I used 45 modulus four, I'll get one, z. So if I use 25, borderless three, I'll get 41. Okay? So modulus will keep the remainder. So 3/20, 5/3 will give us 13 into 824 and then we'll get the one as a reminder. Okay, so when we perform division, the remainder will be given as a modulus. So these are the operators we have seen in the sense of Jackson multiplication exponential, and then the modulus. 6. Data Types in R: Hello and welcome back. In this lecture, we are going to learn about data types in R programming. So unlike in other programming languages like Java or C, or C plus plus, where we declared a variable, we declare the variable without datatype, right? Suppose, if we want to use a integer number variable, will declare that int x. Right? Now, float why our character the x right? String. Why string a, string it like that, right? But in our programming, we don't declare variables with datatype, right? Suppose I've wiped my probably, like in here, I have assigned x or ten, right? I have assigned ten to the x. So this object is assigned a value obtained. So the data type of this object will become datatype, datatype of this number ten, right? So this variable or this, suppose this is ten as an integer number. So this object data type will become number. If I assigned x equal to some string, this will become the character string variable. So we don't decide upfront whatever we assigned to the variable or object, that object will become object type. So it's four or less like our dynamic mirror light bright. So we don't put the data type upfront. It will be decided what kind of value going to that object. And that object will become that type of valuable object, right? So let me create another file where we will be seeing the datatypes up. So let me clear this edge. Well, so there are basically six types up our objects. And they're like, Oh, rectus, inferior. Rectus. Then lists. And then we have Alice. And then we have matrices. And then we have factors. Factors and then study. Okay? And then we have dataframes. Okay? So these are the six other object types that we have in our programming. So let me explain this. Data types one-by-one or object types one-by-one. Okay? So the first thing, we'll see the vector, so vector object with the simplest object in our programming. And death, I suppose it has six data types. So we'll see, suppose I use x equals to true. And if I print x, what we like it, I'll get to write and support that. If I want to know what kind of data is this x, what I can do, I can use print and then I can use glass inside the print age class X. Okay? And if I run this, I'm getting class of x is logical, so this is the exit logical datatype. Okay? Similarly, we have numeric support. If I assign x equal to 90, and let me just copy this to y. And I'll put plus of y here. And if I run the whole source, the class of why swing nomadic, right? So now we're not assigning otorrhea, not declaring the datatype of y. Whatever value we are assigning based on that. The variables are, the data type is being decided. Okay? So he had to rehab given a logical. So it is logical. And here we have given nomadic, so it is noetic. Next thing is India, right? Suppose that 90.9. So what will be the C? It isn't nomadic rate. Now the next thing is really thin integer. So suppose D, and I'll assign some values, of course 34. Okay? And then I'll use print gloss Plaza de. When I use print class up to you what tailgate? See, I'm getting nomadic. Sorry, I'm getting interior. No, class up this integer d is up integer datatype, right? Similarly, we have complex suppose and declared a complex number, three to plus five high. Right? And when I put cluster V, What we like it, see the class of I is complex, so this V, each complex number two plus phi. Similarly, suppose sine C as a ReLu program. If I bring the glass up, see what it would be here to be character, right? So let me run the whole source C. Now I'm getting the glass up sis character. Next thing is that at all supported, I'll use blue and I'll assign it a new character to draw and use Java. And if I print class of problem B, it would be, let's see. It is the raw. Okay? Next thing is the odd objects are called vectors, okay? So these are the vectors types like glass and w is a rod close up. C is character class of v is complex number plus or p is integer, class of y is numeric, and close-up x is logical. So these are the data types or object types, you can say in our programming. So let me save this file as well. Data types. In the next lecture, we'll see how we can create our vector, okay? 7. Creating Vectors in R: Hello and welcome back. So in the previous lecture we have seen the different data types in R programming. In this lecture, we are going to learn about vectors. How we can create a vector with most people element, more than one element or Control N. I'll create a new R script file. And in this, we're going to write down our steps for creating a vector with multiple element, more than one element. Okay? So what I'll do, I'll try to create and recta hockey. So support. I want to create a vector called g. And I want to assign it to multiple values. So in our programming, when we want to create a vector with multiple elements, we have to use a function called C function, right? So this C function will allow us to create and recta with multiple elements. Okay? So when you see and then close bracket, and then inside that we can write our elements support. I want to create a country's name. Suppose Alport, Australia, Britain. Okay. South Africa. Support digital elements. I'm creating inside this vector. Okay, So we can use the function to create and vector with multiple elements. Right? Now, I just use print and I put Ci. So this will print the values inside the director. So let me run this. Let me save this file. And I'll save this file as Rector dot. Okay? And all these files will be our level inside this current working directory that we have zipped. And I'll attest these files. So already you can download and you can walk on that. Okay, so let me run this source file. Now we are, let me clear this console and run and run it again. So see here now we're getting the value for t vector as Australia, Britain, South America, and Russia, right? So this says, and vector with multiple elements inside it. We can, we have seen how sorting. We have seen how we can create our vector with a single element like India. And it will give us the value in the brain. It will give us in the audit. So this way we can create and vector and object with a single element and we can use c function to create a vector with multiple element. And suppose, if you wish to print glass of G, what will be the output? Can you guess? Let's see. Character C, the null characters. So it is giving us the cluster t vector S character. Suppose if I change this t 0, sorry, let me first printed this class of t. So the cluster please character here. Now what I'll do, I'll just change it 200, a numeric value, okay? And I'll try to run, tried to run this. And then I'll try to run this last statement C. And now we're getting the glass up. B has a nomadic. Similarly, if I change here 120367900. And if I run this source file, what we get, we'll get the 12367900. And the class of G is now change to numeric. So whatever data you are going to use based on that data type of the vector will be decided not upfront like C or Java programming, okay? So this is the difference between C and Java n, right? So whatever you use based on that, it will be decided. 8. Sequence in R: In this lecture, we are going to learn about sequence. We are going to learn about the sequence function in r. So what do we do with the sequence function that we'll see? But before that, let me tell you one more important thing that I have forgotten to include in the previous lecture, and that is support. I'm creating a vector and using C function. And if like here, I've created a function using only numeric values. So the class of this will be numerically because all the elements are numeric. What if I gave one character and then I'll give numeric, and then I'll go through a logical and then I'll give I integer. And if I run this, and if I first let me bring this recta, see, you know, hello, 67 through everything to a quotation. Why? Because if we, if we create a vector z function and if one of the element is of character, all other elements with a numeric integer or logical, it will be converted to a string. So if I, let me just copy this, and if I put glass of wet, it will be coming into character. So all other elements will be converted to the character. If any one element is character, all other numeric, God, logical or integer, all other elements will be converted to the character and the class of that vector will be character. So remember this, if all numeric, it will be numeric. If all integer class will be integer, but if it is mixed of integer and character, the integer will be converted to the character. If anyone, any element is a character, everything will be converted to the character. Okay, so that is to clarify. The next thing is I'll create a new script file. I don't know why this is not going up. Okay. That is some kind of errors. I don't know why. So now we're going to learn about the sequence. So in R we can create a sequence of numbers like suppose I want to create one to ten. I want to print numbers one to ten. I can do that in two steps. In two ways. I can do that in two steps, but in two ways I can do that. I can use a colon operator or I can use the sequence function. So first thing, what I'll do, I'll try to create a sequence of numbers. I'll assign it to some value. Suppose I want to get the sequence. Suppose I'll say, see, I'm creating a ready, we'll see that we'll be having value. I want to assign it a value one to seven or one to obtain, or one-to-one hundred. So how I can do that? One colon, 100. If I do one color at 100 and I, if I run it, what it will do, it will create a sequence 1-100. Let me see. See here. 1200 numbers has been generators, so it will create a sequence of numbers, 1-100. So let me put the console on the right. So now it will be easy to understand and write the code here and the output will be here. And let me, okay. So now what we need to do to create any sequence, we need to give the first number from where we want to start. And then we have to give the ending number. Suppose if I get ten and it failed on this tool, it will create a sequence 1-10. So if we want to create a sequence, you can use this colon operator and you can create a sequence of number. Suppose I want to create numbers 1-20, or I want to create two to 20. How can we do that? If I run this, what we will get to see, we will get 2468 up to 20. So each and every element. A sequence numbers like one will be multiplied by 22 will be multiplied by two. Like that. We'll get, right. Similarly, I can create the sequence of the points. I wanted to create a sequence from 2.5 to four. I can do that. If I print f c Here, I'm getting 2.53, 0.5 because the next one will be for valley. So it will print the sequence from 2.52. And if I use here 40, it will be giving the ecto 0.53, 0.5 up to 39 point for him, right? In the same way, suppose I want to create a sequence from two colon, 20 minus one. And if I print D, can you guess what will be the answer? Let's see. Now we're getting one to 19. Why it is coming one to 19 because no one will be minus from this 1.20. So it will actually generating the number From one is to 19, to minus 11 and minus 119. So the boards will give us the same result right here. Okay? The next thing is, we can do it in a different way as well. I can use, sorry. I can use a variable a, I can assign a value eight. And then what I can do, I can, I want to generate numbers 1-8. So I can, if I run this tool, what I'll get, I'll get into sequence of numbers 1-8, right? So this way also we can do, we can assign a value to the variable or object e, and then we can put one is to it instead of punch to it, we can put one is to a. And this will also give us the same digit. One to eight numbers I've been generated. I can put one minus a as well and see what will the result zero to seven because it will, one minus 172027 sequence will be generated. If I put a minus one into bracket. And if I try to run this, now I get one to seven because this will be executed first, okay? So it will be 7.1 will be one because we are not subtracting this one from here. Because if it is in the bracket in and in, the bracket will be given the higher pre-prints. So this will execute first. One is to seven, means. It will generate numbers 1-7. Apart from this, we can use sequence to create a sequence of numbers to support that will give you finite SEQ and I'll give one comma, one comma five. What result I'll get? I'll get the sequence of numbers 1-5, right? Similarly, if I give a sequence of nine, and what digital target sequence of numbers 1-9. Here, I am giving one to five, means 1-5. And if I'm not giving the fourth digit, what it will take, it will generate 1-9 by default, okay? That, that is the default nature of sequence. Okay? Next thing is we can use this function in a different way as well. I can use from aqueous to suppose 8.2 to 32. And if I run it, what it will do, it will generate numbers from a 32, 32, okay? Similarly, we can also use sequence as I can keep the front number. Then I can give the two number, suppose eight. And then I can give to what these two will go here. It will. This is known as the step. So let me write it here. Sequence allocate to coma, support gravel comma, step to. This argument is step. So let me run this first and then I'll explain. See here what we are getting. We are getting. 24681012. What it is doing, instead of creating a sequence of 234 up to 12, what it is doing, it is creating, it is giving us adding two to four. There is a gap up to right, because we have given the step value as two. So two plus 244 plus 266 plus 28, then we'll difference up to between these two sequences. So that is what the user step argument, this is the step out when this is the step argument, right? We can also see another example. Suppose I want to create a sequence of ten, 10-25. And I want to increase the sequence by wine, seven flame. So let me run this. Saudi Cl, what we're getting, getting, the sequence of numbers are getting started from ten because you have given ten and it is going till 25. And what it is doing, it is adding 0.75 in the each values or ten, it will start, then it will add 0.75, then 10.75, what it will do, it will add another 0.75, 0.5, 12.25. So this byte, seven, five-minutes, it will increase the value by 0.75. Suppose if I do this by three, what it will do, it will add 101-316-1920, 2.25. If I increase this by value to five, what it will do, it will 1015, 2025 like this. Okay? So this way we can generate the sequence. There is another thing that is called length. So suppose I want to create numbers 25-50, and I want to keep length equal to six. What it will do here, it is giving us starting from the 25, then 630, then 35.40, 45, and then 50. So it will generate a sequence of six numbers between 25. And if I make it, I suppose, then what it will do, it will generate numbers 25, 27. So what it will do, 25-50, it will generate ten numbers with the equality, like 205-20-7303 point, something like that. Okay? If I gave it 100, it will generate hundreds of sequences, right? If I gave it to one, what it will do, it will generate only one number 25-50. If I give two, it will generate two numbers. 25.50. If I keep three, it will be in divide it by three, see a 2,537.5 and 50, okay? Like this. So it will try to generate three sequences, 25-50. 9. Replicate Function: In this lecture, we are going to learn about the replicate function. So what replicate function will do. So we'll use that app rep means that replicate and when we want to replicate something. So, so let me show you with an example. So suppose I want to create a sequence where I want to repeat 58 times. I'll use that function and I'll give five that I want to repeat. And here I'll give times three times 28. So what this Replicate function will do, it will repeat five times. So what you see here, 5558 times. Okay? So this Replicate function will replicate the same number. Replicate means repeat, it will repeat eight times. Similarly, we can use characters as well. Support that'll give us. And then I'll specify times equal to three or two. It will print three ds, ds, ds, ds. It will create a sequence of BSD SDS three times. Okay? So remember this, whatever we are giving here, it will be depleted by number of times, okay? In the same way we can use even, we can pass the object to the other application function. Suppose I wanted to, I have created to support art replicate function. I want to create a variable R. Well, I want to generate sequence 3-6. So what this three to three to six we'll do odd. If I print out it will be created 34568 support. I use the replicate function here. And what I'll do, I'll pass this art could replicate. And here you get times twice. What it will do this will repeat this sequence, 3-6 twice. Okay? So if we run this, what it will give, it will get 3456 and then again 32562 times. If I make it three, this sequence will be repeated three times. So 3456, again 3456, and again 3456. So number of times it will be repeated, the whole sequence will be repeated that many times, right? In the same way we have, sorry, we have another argument here. In that application function. That is, suppose I'll use our same object are, and I want to apply each to it. If I apply each year, I have applied times equal to three. Now I'm applying each equal to two. So what it will do, Let's see. Suppose I run this. What it is doing. It is the, each element 3-6 will be repeated twice, and then it will go to the next number of months, means three, twice 33, then it will put 44, then 55, then 663-344-5566. So each element will be repeated and then it will proceed to the next element. And earlier in time, what each element will be repeated. Here? Each element will be repeated, and then it will go to the next element. And here the whole sequence was repeated. 3456, then 3456 again. And yet 3344 supported our portrait. Three, what it will do, it will, each element will be the protector, time 333444, then 555666. So this is the difference between times and each in the replicate function. In the replicate function there are two arguments times then each times will repeat the entire sequence that many times. And each will, each element will be repeated for this many times. And then the sequence will be created like 333444, like this. 10. Accessing Vector Elements: Hello and welcome back. In this lecture, we are going to learn about accessing vector elements. We have seen how we can create vector in R. Now what I'll do, I'll create and vector Henry try to access the element of vectors. We'll see how we can access the vector elements in R. So for that, I need to create a file and I'll give it a name. Ric dot R. Okay? Now what I'll do, I'll create a rectangle with a name, month. And for the month vector, what Alice tall and short. Months. January, February, March, April like that. Okay? The c function, and I'll try to store the values Jan. What I'm doing here, I'm creating and vector, and I'm installing that vector into the object month, okay? And then how many storing the values? January, February, up to December. Okay. Now we have created and recta how we will access the elements of the vector. So for that, what I'll do, let me clear the console here so that we can see the result here. So if I run this and run month, January to December, okay? So now what if I want to access the app play? August or September like that, okay, for that, what we need to do, I can create I can create another vector support month tool. And for this, what I'll do, I'll huge month. And it was the capital bracket and you just see function. And inside the C function, I'll keep the index for the support this January 1, 2344. Comma seven. Comma nine. Okay. So, um, I want to access the month for 7.9 and a fray print month tool. So if I run these two, what we get now I'm getting April, July, and September because what the index is four, then 567 for July and for September it is nine. So this way we can call our access the elements of a vector function. A vector. So vector object we can access like the X would be if you want to access particular element. You can do that. If I change it to three, layer one and twice. And if I run these two statements, what we get to see, yeah, we'll get March and then normal and flip them is Monday, May. So like this, we can access the vector elements. We can also x is the vector elements using logical using logical indexing. How to do that. So suppose I'll create another object, month three and supported which month I want to access each month. And the capital bracket and inside the C function, what I'll do, first I'll give supports true, and then I'll give false. And then I'll get false. Then I'll get true. So suppose I'm passing these four values, true, false, false, true. And if I, if I print month three, what will happen? Let me run this and see the result. What we are getting, we are getting Jan. The first month is Jan dish true. So it is printing here. Then the next False, False. So February, March will be false, means it will not be, we're not accessing the February and March, right? So these two values will not be there too. It's February, March is not printed here. Then for the fourth one to this true. So we're getting to four. And then the rest of element we have not given. So it is coming either May, August, and September. It is repeating. So suppose what it is doing. It is again, doing the same fight on this. Janet Britain mayor gas September and December. So it is repeating the thing. Okay. Next thing is what we can do. We can use negative indexing. So suppose I'll create another object, month for and what I'll do, we'll assign for the month for alcohol month, and at least see function. And here's a projectile give minus two comma minus five. So what this will do and support print month four. And if I run these two, what will happen? I'm getting Jan March and then maybe not, There's February and May not printed because minus two means the second indexing minus two that each February, February not be we don't want to access minus two and -5 min the second month and the month we don't want to access except that everything else will be printed. Okay. So February and May will not be printed. See here, February and may not print it. Accept all others are printed. If I put -12, then December will not be printed as well. So now the norm they somebody is not there. So if you want to access elements and if we want to leave some elements, you can use dopamine for that index and it will not be printed. 11. Vector Manipulation in R: Hello and welcome back. So in this lecture we are going to learn about vector manipulation. So we'll see how we can form a lesion within the vector elements. How we can add two vectors, how we can subtract two vectors, how we can perform division within two actors. And we'll see how we can do multiplication of the vectors. So let's get started. So I'll create an R script file and I'll give it a name. Many foolish manipulation. Okay. So let me clear the console here so that we can see correctly. Okay, so first thing first, what I'll do, I'll perform automatic, often isn't within two with two vectors. So first thing, first, what we'll do, we'll create two vectors, will create, support our vector F1. And I'll create a vector, the vector F1. And what I'll do, I'll assign some values. Suppose they aren't yet, 95 something. Okay, so I'll create one vector, F1 and copy and paste. And I'll create another vector, F2. And we'll get somewhere loads like to type D and 20. Okay? That'll give, okay. So now we have two vectors, F1 and F2. I want to perform addition. Okay? What I'll do, I'll create another vector, another object a. And what I'll do, I'll assign F1, the F1 plus F2. So what I'm doing here, I'm adding these two vectors, F1 and F2 and assigning it to object a. Okay, so now what I'll do, I'll print a. So let me run this n. So you are done digitally. See you know, what I'm getting? I'm getting the result 24, 24, y plus 212. First element of these two vectors will be added, 12 plus 214. Then 78 plus eight is 690 plus 31, 25 plus 25, 30. So this way we can perform a region. Suppose I want to perform some tracks and what I'll do I'll just copy this. I really gave with tenants that day. Yes. And I'll bring tears and and what I'll do, I'll do F1 minus F2. So F1 minus F2. If I do what tailgate? See I'm getting 107060 and -20 y again, 12 minus 21078 -70, 90 -30, 65 -25, 20. So this way we can perform subtraction in the same way. If y Yours multiplication, I'll create over m and what a low F1 and F2. And I'll print, if I run this, what we'll get will get 12 when two to 24, 78 into 862490 into 3027005205125. Similarly, we can perform division. So suppose I create a vector d and what I'll do F1 divided by two. And I'll print deal. And if I run this, what I'll get you sick by two to 678/9, 0.759, 0.7, 5.90, 233 5/25 is 0.2. Okay? So this way we can perform a decent subtraction, multiplication and division. So let me write here, this is multi application. And then this is the region. So this way we can perform addition, subtraction, multiplication, and division on the rectus. And this is called vector manipulation. 12. Vector Elements Recycling: Hello and welcome back. So in this lecture, I'm going to tell you something that I have not totally, you have not taught you in the previous lecture. And that is a very unique question that you might have asked if it is a physical or offline class. And you must have come into, this question, might have coming to your mind as well. So suppose I have two vectors, F1 and F2. And F1 is having four elements, and F2 is having only two elements. And what if I will add these two vectors? Suppose I'll perform initial conditions to F1 plus F2. So here four elements and here two elements, one list. So how it will do. So in R, If we try to perform F1 plus F2, if we try to add two unequal vectors, then the second vector, which is of shorter length, it means the sorta vector will be, elements will be recycled to make it dark length of the first vector. So in this case, what will happen? Vector will become like this. Internally. What? F2 will be like two comma eight. Comma two, comma eight. Okay? So f two will become something like this, okay? You too, it will be repeated till it will make the number of elements in the F1. So there are four elements, so it will repeat to it, to it. So this will become something like this. And then it will perform the automatic operation. Let me run this hand soil that I dealt with that you'll get a better idea. Let me clear this and run this. See here. Now the result is 14, 86, 90 to 13. How to plus four, sorry, two plus two. Sorry, 12 plus 214. And then 78 plus 886, and then 90 plus two. Because the two, it will be repeated. So 90 plus 292.5 plus 813. So this will become like this. If I add one more element here, suppose 80. And if I run this, what will happen? Let's see. See here we're getting an error here. Longer object length is not a multiple of starter object length. It means now see yet to. This element is a fourt up to is having two elements. Two elements. And F1 is having fun is having phi variables, right? We're getting into debt. The second, the first element to this is having two elements and this is having five. So five is not a multiple of two, right? That's why we are getting at because two, if you divide five by two, you will get that remainder one. In this case, this reputation will not work. But if I add one more element, suppose seven. And if I tried to run this, this will work this time. Now we're getting the chart because now the number of element is six here. And second one is 22268/2, right? Six is Multiple up to the first L. Longer elements, longer vectors would have the mark. As many numbers. Number of elements would be multiple app does certain elements like to six, right? So six is the multipolar to this function, this multiplication or division or addition you can perform. But if it is five, then two will not be the phi will not be the multiple of two, and it will throw an error like this. Okay? So I hope you got it. Similarly, if you want to perform multiplication, that also you can do. Here, we are getting the multiplication division, Everything will be done. So the elements recycling will happen only if the softer the elements in the longer vector is the multiple of the elements in the sorted vector. So longer vectors, number of elements would be the multiple of those elements, okay? Number up sorta elements. Okay. I hope it is clear for you. 13. Sorting Vector Elements: Hello and welcome back. So in this lecture we are going to learn about who vector sorting. So let's see how we can sort vectors. Okay? So what we are going to learn, we're going to do sorting recta. Okay? So for this we have a function called sort. So we're going to use the sort function. So first thing first, let me create and vector 0 wreck. And for this, I'll give some random number, 349-082-7543, some random numbers. Okay? And suppose I want to sort this vector elements. So what I can do, I can simply create another objects are wreck. And what I'll do, I'll call the sort function bars this what is null vector to the sort function. And what this sort function will do. It will start this vector. And then what I'll do, I'll just simply print start. So here I'm printing the sorted vector and this is the additional recto. And I'm passing this vector to the sort function, and then I'm printing the sorted vector value. Let's run this and see that it's definitely clear this terminal and run it again. See, you know, we're getting a sorted vector. So 349, 273-043-6408. So it is sorted in increasing order. So by default it will sort in increasing order. So the lowest elements will move first and then it will keep on increasing. Okay? So this way we can sort and vector in R. What if I want to sort this bark? I want in a decreasing order. So for that, what I can do, I can use the same sort function. And here what I'll do, I will just keep decreasing, decreasing. It's the logical thing. And yeah, I'll give true. So decreasing, I'll keep drawback because by default it is false. And it will do that. Increasing order, right? Ascending order. By default, it will sort in ascending order. So what I'll do, I'll do they creating cultural true means it will sort the vector into decreasing order. So let's run this and see now this sorting into the cleansing or a 98 first, then 463-06-3420, 79.4. So this way we can sort in decreasing order. Next thing is we are going to see sorting of character, character, character rectus. So that also we can do suppose I have a rectangle, one, which is having some values like for some random things. And the red, blue country. Anything in a character form. Okay, so this is the character vector. It is having values in character. And suppose I want to sort this and I want to store the sorted vector value into the vet to function. I'll call the sort function and I'll pass this one recta to the sort function. And if I run these two lines, what algorithm? What do you like it? Let me bring to this print tool. So if I run this line, what we like it and you get blue, then country than four, then drown, dendrite. Why it is so because B, then C, then F, then r. Okay, So it will be in alphabetical order. And what if I want to sort in? But he was harder. I can give you decreasing call to draw and now see that deserved. It when we started flushed are then f, then C, and then B. So D must solve poverty cotton, it will be this way we can sort the vectors, both character and the number, okay? 14. Decision Making in R: Hello and welcome back. So in this lecture we are going to learn about decision-making in our, in our other programming languages. We have the if statement, we have if else, and we have switch statement. With that, we can perform decision-making. If a statement means if something is true, then the following statement will be executed. And it felt means if something is true and if you are giving a condition and that condition is not fulfilled, then the else statement will be executed. Switch will see how the variable will be tested for equality. Here is the list of values, so we'll see if else. And then we'll see the switch. So let me create our script file here. And I'll give this a name. Decision-making, dark art. Seaweed. Let me clear the console so that we can see that the gel. So first what we'll see, we'll see if sorry, I need to type here first. We'll see if statement. Okay. So first reality, if statement. And if it's, if it's stormwater, it consistent, consistent. Now Boolean expression and following statement. So if the Boolean expression is true, then the following statement will be executed. Support what I'll do here. I'll create an object a, and I'll assign a value 67. You can write anything, okay? And what I'll do, I'll write if statement, if ie less than 70, then what I'll do, I'll try to print is less than 17. Okay? Suppose this is the thing I want to perform. So if I run this, what will I get? See that result I'm getting is less than 70. Suppose if I change this value to 75, 75. And if I tried to run this, what will happen? I will not get any output because E is 75. And if I go inside this if statement, it will be sewing a less than 70 is greater than 70. So it will not enter into this statement. It will not fulfill dog if condition and this statement will not be printed, right? So this is called IF statement, right? Similarly, I can write like a dot integer. So the other part is integer, and then I'll pass E. And a light is. So let me run this because it's a numeric here. And what I'm testing yet, if is an integer, then I need to print is an integer, right? If I put, make this an integer, tending equal to 75 L. And if I run this, see, now a is an integer. It will be painting. This way. If this condition is fulfilled through, it will print this subsequent statement. Okay? Similarly, we can put an else statement here. If each integer, then it will print this. I can print some other thing. Lake is nomadic. Okay, So if I run, this is an integer. If I change this to numeric, and if I run this again, what we'll get, we'll get is numeric. Let me try. If I touched yet is numeric. And if I run this, what happened? Okay, I need to put this. And if I run this, string is an integer. I need to change the statement here, New Relic and here integer right? Now. Because a is Nomadic, this condition is getting true. So it is printing is nomadic, rate is numeric, is Nomadic. And this statement is not, it is not going to the else part. So this way we can put a for now we'll see switch statement. The switch statement. So let's switch statement will do suppose I have a vector D and I'll use switch. And then insights through what I'll do. I'll support four comma one. And I'll give 2345. And then I'll close the switch. Then if I put into d, what value we will get. Let's see. When you get four, because I'm passing for what's switch statement will do. It will go to the fourth value, and it will print that value. So a year for is there 1234, so it will print four. Suppose I'm writing here something else. Hello. So this hello will be printed. Okay? Let me show you this printing hello. And suppose if I do one or two, it will be printing to, it will go to the corresponding please. Okay. So far two, it will go to the 12. It will print these. For three, it will go to the hello for four, it will go to the hello. Five, it will print 55. So what it will do, a switch statement allows a valuable to be tested for equality against the list of values. So these are the list of values. And for them, indexing is 1,234.5. Whatever index you will give you, that index value will be printed. Suppose l gave three. So the index value three will be printed. 3 min, 1233. Will you buy a gift for? Then hello will be printed. C. So this way we can use switch statement in heart. 15. Loop Control using repeat and while loop: Hello and welcome back. So in this lecture we are going to learn about loop statement in our, what is a loop? A loop is a function. You can say a loop. With a loop, we can execute a statement or group of statements multiple times. So when we put loop, when we put a statement inside the loop, it will be executed multiple times till the condition is satisfied, right? Until unless it is satisfying the condition will keep on looping the same statement again and again. So a loop statement allows us to execute a statement or group by group of statements multiple times. And there are three types up. There are three types of loop statement in R. The first one is repute, the second one is a while loop. And then we have the very popular one loop. So what I'll do, I'll start with a repeat loop first. So let's see what is repeat loop in R. So suppose we have a recto which is having the railroads supports India, Burton, okay? Like this. So I'll say a bit later. So we have n vector which is having values India, US and Britain. And suppose I'm taking another route able, suppose I gave it a name count. And I'm giving it like support for Kant equal to four. And then what I'll do, it was dirty feet function here. And I want to print the vector of particular number of time. Okay? So what I'll do is print report that I gave. If you count here, if count is less than ten, then what I want to do, I want to print rec. Okay? So they'll say What letter. So if I run this, what will happen? Let's see. See you. It will keep on printing this because count is for and what condition I've given. I'm given if count is less than ten, so this Ford will always be less than ten, right? So what I'll do, I'll use here count, count plus one. So each dress and I'll increase the count value to one. Now, let me stop this. And let me run this code again. Now see what happen. It does print it 123456 times y is the extreme. First time it will come. Count is equal to four. So it will print whenever you go again. So count will become five. Then again it will print four. Then 54678 up to nine. Then extranet, the count will become ten. It will not print, it will print 45. It will print 445,678.9. So as to net current will become ten, it will come out of this loop. So this is the usual loop. Okay? So this way we can use the repeat loop, okay? In the same way, we'll try to use the while loop. So what I'll do, I'll huge wild. And again, Waterloo count is less than suppose. While count is less than eight, what I'll do, I will print senior counties for and what I'm giving you a count less than eight, then it will print wake and it will increase the count while loop well by one. So if we run this, we get India-U.S. four times, right? Because it will print 44, will print profile. It will print for six, it will print 47. And as soon count will become eight. It will come out of this while loop. Okay? So this way we can use the while loop. 16. For loop and next statement: In this lecture, we are going to learn about for loop in R. So suppose I'm creating a vector d rectally, which will be containing something like two to 20, okay? Boards, if I bring t will give me the numbers 2-20. Okay? Now what I'll do, I'll write here, I will do. I'll use a variable i in D. So for each value of I in the, for each value in this vector d, What I want to do, I want to print, right? And then I want to do I plus one. I equal to I plus one. Let me, okay. I will say with letter. So let me run this and see what we are getting. What I'm getting, I'm getting 234 up to 20. So what it is doing, it is checking whether this I, in the, for i, it will be 11 is not there, then it will increase it by one. So it will come and check here too, is that in D two, then it will again, I equal to three, then it will print three. Like that. It will print 20 and then it will come out of the loop. So this way we can use the for-loop, right? In the same way what I can do. If I put here a simple thing. I'll remove this. And yeah, What else do? I can put heap condition inside here. If I call too, Support 15, then I'll, what I'll do, I'll print, I write. If I equal to 15, I'll print. I changed here, I'll put next. And then I'll print. Print, right. So see you, what will happen? Let me run this for why we're getting at those. We've done this again. What, what is happening here? I want each value and I, it is printing this, but it is checking the value. If I call two plane, it will. What do the next. So here you see 2345678 up to 13, 14, it is printing correctly and then it is think, okay, I equal to 15. Next, Next means it will skip this titration. So 15 will not be printed here because next month it will skip this tracer. So I called a pre-print means this address and we'll be escaped and footprint will not be printed. And then it will go to the cysteine and then 161-718-1920, reprinted. So next month, it will skip that I dress. And so if I use here, suppose 17, then the 17th titrations will be skipped. Print 16, 17 will not be printed, and 18, okay, So if you want to skip one nitrogen you can use next. I hope you got the idea of how to use next. See you in the next lecture. 17. Functions in R: Hello and welcome back. In this lecture, we are going to learn functions in R. In R there are two types of functions. So the one is built-in functions, built-in functions, and the other one is user-defined functions. Defined. Okay? So there are two types of functions, built-in functions and user-defined functions. So first, let me tell you what is a function. Function is actually like if we want to execute or some group of statements and perform some calculations, we want to Our do something. So it's basically like a function is a keyword. And in functional V basically for fun group of statements, okay? Our group of functions of group up calculations. We will perform some calculation or we do something that is a function. Okay? So let me tell you the basic structure of a function. So suppose our function takes, suppose that this is a function keyword in R. And what it takes, it takes arguments, right? So it takes arguments, so it can take any number of augmented augments, one, comma two. So this will be a function, okay? And in this function we can do anything with these arguments, right? Suppose these arguments can have some values, so I can support, I can use some of these to augment, so I can perform augment one plus two. And I can like print some of these 12 months three. So this is one function where what I'm doing, I am performing some of these to augment, augmentation and augment to, and I'm finding some of these to augment one plus Segment tool. So this is one function and I can call this function by passing the argument. So suppose I will give some name to this function, some function, and assign this, this whole function. And I'll give it a name, some function. Okay? Now, what can I do? I can call this function right? Bossing the two arguments, argument one, argument two. So I can do some fun. And here I can perform three comma six. I can pass. If I run this, let me run the first under some fun this. And then I run this, see, Okay, yeah, hawks one, it is. So let me run it again. And if I call the function by passing, okay, this is also wrong Saudi for this mistake. And if I call this function by passing these two arguments, 3.6, I will get the result as nine, because three plus six to nine. So this is one user-defined function, right hand, we have already seen built-in function. So what are the built-in function we have seen, we have seen sequence like sequence one comma eight. And this will give us the sequence of numbers 1-8. So this sequence function is a built-in function that we have in our, so this is built-in function running it. We need not to write a program to print the numbers 1-8. This will do automatically because in this sequence function has been written behind, like in the coda far it is, it has been defined that when we use sequence and will provide two arguments. One is to aid, it will generate the numbers 1-8. So that is already been defined and that's why they are known as, known as huge, sorry, built-in functions, right? So these are built-in functions in the similar way we have seen some summer. One is 28. And this will give us the sum of numbers from one to it. It will give us the results statistics in the same way. We have the main function. So I'm typing, yeah, I mean, like suppose I will get to number 23, comma 90, 87. And if I didn't see me and we'll read 23 can also mean two comma nine. And the free dental, I'll get 5.5. So we can use doc mean some sequence all these built-in functions. So we have seen these ads are user-defined function, user-defined. And this sequence and some water. Built-in functions. Okay? Now what I'll do, we can see how we can call this sum function. Okay? So let me copy this sum. So we can call this function by providing the arguments here. And we can also call this as x, one to four comma two to nine. Now I'm getting 13 to this way also, we can, this is y position and this is vary the argument, name, argument, the argument one equal to. For an argument to equal to nine, like this we can provide. So this is another way of calling the function, okay? Now what I'm going to do, I am going to write our function to find the square up cities have nobody's of series of numbers. So what I'm going to do here, I will kill you. Quiet function. And what I'll pass to this and pass a number a hand. What I'll do, I'll create a sequence of numbers, so I'll use the for loop here. And what I'll do for I in a to ten, okay, so whatever number this will be from that number to ten. I want to create a number sequence of numbers. And then what I want to do, I want to assign, I quit. I want to square the I value and assign it to b. And then I want to bring to the b value, okay? So this is the function square root function that I'm defining here. And now what I'll do, I will call this square function by passing a number support for what it will do. It will first create numbers 4-10, and then it will require 45678 up to ten, and it will print the numbers. Okay, So let me run this. See here we're getting cysteine. First number will be 44 square 16, then it will print cysteine. Then the next number will be five. It will print 2055 squared 25, and then 630-67-4097 squared. 409849 is quadratically 1.10 squared hundred. So this way we can create a simple function that will find this quiet off series of numbers. So Syriza number is four to ten and we are getting here four to ten square. Each number will be squared and we'll be getting the next thing is what I'm going to do. I'm going to find the sum of even number. So I'm going to write a program to find some of even numbers between two numbers. Okay? So what I'll do, suppose one to ten. First I want to find one to even number, 1-10, sum of even numbers, okay? Not a squared sum of the even numbers. Okay? So what I'll do, I'll write a function sum of even function. And what I'll do, I will write here, first, I'll generate the even number. And how to generate even number. And you see alginate one to ten even number. So what I'll do, I'll multiply the number by two. So it will be sum up the numbers 1-10. And then what I'll do, I'll print and in print what I do the sum of these even number, so and foster even number, this sequence of numbers to sum, sum function and then it twists. So before that, let me bring to the given number. Okay? And then I'll come out and I'll call the even function. Okay? So let me run this. Okay, this is the lowercase c. C. You first time getting 246810, sum of even numbers 1-10. So these are some often even number 1-20, okay? Because I'm multiplying by two. If I make it five, it will be the earlier what we have written. It wouldn't be that, okay. 246810. Okay. So this way we can tender sum of even numbers, sum of numbers. And then when you get done with this function, some of even number. So it will sum everything and it will give us the result. Okay? So when you sum these numbers will get to 110. So this way we are getting that some of even numbers 2-2001 to 20. Okay? So this way we can do it. Next thing is I'm going to do calling function without document. This is very simple, calling function without arguments. So what I'll do here, I will simply create a function, hello, h function keyword here. And inside the function keyword, what I do see even here also we have called us some of even, some of even function without passing any argument. Okay? The same thing we are going to do here as well. What I'll do here, I will simply use a print and I'll say Hello. How are you? Then? Oh, tenancy. Say hello. I just call it. Okay. So before that I have to run this. So let me run this and see, yeah, we'll get hello and how are you this way we can call a function without augment. 18. Matrices in R: Hello and welcome back. In this lecture, we are going to learn about metrics in our matrix analysis, very important concept and we need to understand it. Because when we go further beyond this class, when you go and try to implement the art concept to analyze the data you, when you try to use the R4 data manipulation, data visualization, or even the data mining, data analysis and in data science projects in machine learning. So matrices are going to be very full and we're going to use it in many places. In our matrix. Is an object or an object. In art, everything is an object. And in that way, matrices are also our objects, right? So matrices are the art objects in which the elements of the same atomic type Arrange. So in matrix, matrix we are going to add into same type of elements, same atomic types of elements in two-dimensional rectangular layout. So it will be to diamond stone rectangular matrices. Matrix is a function. So how we are going to create the matrix? We are going to use the matrix function that is inbuilt in R. Create metrics and art. Syntax would be matrix. And then we provide the data, determines the matrix elements we are going to provide here. And then we're going to provide number of rows and rho is number of rows in the matrix and then n columns, number of columns in the matrix. And then we're going to say by row, we want to add in the elements by row, by column that we are going to define by this argument. And then we are going to give the diamonds sons named him named means. We're going to keep the name of the dimensions. Okay? So data is input vector, which becomes the data elements of the matrix you can draw is the number of rows to be created and college number of columns to be created by rho is logical clue if true, input vector elements are arranged by row. So if you want to create a matrix by assigning the elements by row, we have to give by rho equal to true. Otherwise it will be, by default, it will be in a column way, okay? Columnar values, like if we are given element one to ten, it will be 12 or three. Column wise it will be, if you give true, then it will go by Viral hockey. And dim name is name assigned to the rows and the columns. Okay? So this is the fundamental we should be knowing. And now what I'm going to do, I'm going to deviate matrix. Okay? So for that, I have already created our file where I have written few programs, are few functions to create metrics so that we don't waste our time in writing the code and again and again. And while writing it takes lot of time. So I have already created these two matrices dot our files. Explain what I'm going to do in each and every steps. Okay? So first thing, we're going to create a matrix where elements are arranged sequentially by column, by default. Okay? So suppose I'm creating a matrix and I'm giving it a name M1. So one will be an object in NADH which is going to contain this matrix. So I'm going to use matrix function. And here what I'm passing, this is the data that I'm passing. 12 to 35 means it will create numbers 12-351213, 14 up to 35. So this will be done data and then what I'm giving the next thing, Andrew, number of rows six. So I'm just passing the data and giving the row number of rows. I want to create a matrix which is having number of Rogers six. That's it. I'm not giving any column value or anything. I'm just passing the data drive to 35 numbers and number of Rogers six. I'm not giving by row or column that argument. I'm not passing here. So I'm just creating a matrix which is having, which will be created with the six rows and it will refill with the data to 35. So let me create it. If I run this. See ya, M1 is one is to six, so one is to 412 to 36 data will be provided. And if I print this M1 matrix C here, now we have six rows, 123456 rows total six roads. And in this six rows are data 12, 13, 14. See here first the column one will refill the year column 112, 131-415-1617. Then it will go to the column to then 24 to 29 and then 30 to 35. This way, so data is being filled by column wise, right? To wash the column one, column two, then column three. Okay? Because I have not given anything, yeah, I have not specified by rho equal to something here. Okay? Next thing is, if I use by rho equal to false, what it will do, it will do the same thing. So if I run this as well, again, I'll get the same output, same matrix will be printed again. But if I provide here the same code by rho equal to true, now the matrix will be created by filling the rows first. So flushed 12131415, first row will be failed. Then 16, 17, 18, second row, then 2021, 22, 24th row, like that. Okay, so fast, the rules will refilled. And then the second row, then third row like this. And here it was first row, then second row, first column, second column, third column, like that it was filling here. First row, second row, third row row-wise, or column-wise or row-wise. So if you want to fill the data by row, you have to give by rho equal to true. Okay? Understood. Next thing is giving the column and row names. Suppose I want to give the column name here, and I want to give that row name also. How I can do that. I can use C function and I can give name to the column rows. So suppose I want to then a six rows. I want to give each row name row one, row two row, row six. So I'll use the c function and I'll create an object of row names, and I'll create another object column names rather than keep column one, column two, column three, column four, whatever name you want to give, You can give you. Then while creating the, while creating the matrix, what I'll do, I'll first pass down data matrix inside the matrix functional forest bird the data. And then I'll give number of rows, six. Then I'll give by row through this, you can give true or false whatever you want, okay? And then here, the next argument will be deemed named dimensions name. And here what I'll do, I'll create a list. And inside the list, what I'll do, I'll pass the row names and row columns as a two argument. So this will create the dimensions for the matrix. Rows and column name will be fetched from here and pass on to them list. And it will create the row and column named for the matrix. So let me run these two lines. Okay, Sorry, I need to run this whole thing. See here. Now we're getting a matrix with the column name, column one, column two, column three, column four. And row names as row one, row two, row three, row four, row five, rows six. This way, first we need to create a column list up column name and column row names and columns name. And we need to provide those to dim names argument okay, through our list, okay? And then it will be created a row and column names will be given in that matrix. Next thing is accessing the elements from the matrix. So suppose I want to access the element from the M5, this matrix, first column and third row. So how can I do that? The third row, three comma one, m phi, and then the bracket and three comma one. It means I want to fetch the third column, third row, and first column. The first argument is for the row and the second one is what column? Third row, first element will be printed. Third row means this one and the column one-month, plenty. So if I run this, I get, I should get 20 years, 32. I'll see here we're getting 20. So m5 is the matrix. And I want to finish the third column, third row, first column, third row, first column. Similarly, I want to first off value of the fifth row, second column flip through. This is the flip through antecedent the second column, so 29th would be printed here. Yeah, we're getting 29. Similarly, suppose I want to first row six. Okay? So I can give m phi and rho six and column value. I'll leave it. So if you give like this, what will get? You will get the row six value c here, 323-033-4305. So this is the row six. You will get that. Okay? Similarly, if I want to access the second column, I can give, I can leave the row and I can give one liter column value. And if I run this, I'll get the column to see you. 13, 17. This is the column 213-17-2120, 5.20, 9.33, 29, 30. So this way we can feature column value. Next thing is matrix addition and subtraction. We can do multiplication, addition, subtraction and division with duck matrix. So suppose I have M1 matrix, let me print this. See this is the M1 and I have another matrix that is M2. So both are same matrix, but what are the different, different metrics but the values are same. Okay? So support, I want to add m1 and m2, so I can simply do m1 plus m2. And I'll assign this value to some object. So if I run this, and if I print the sum value c here, I'll get the guarantee for 36 like they're stored 2012 +12, 241-313-2614 plus 14, 20th like that. Okay, similarly we can do m1 minus m2 and we'll get 000 because both the metrics are same, right? M1 into m2. We can do multiplication, sorry for this. 12 into 12, 24, 30 and entertain once it's turned like that. Okay? And in the same way we can perform the matrix multiplication where the, each element will be divided by each element, okay? Well, by 121, like that. Okay? So this way we can use matrix and we can perform it isn't subtraction and we can access the matrix elements. 19. Factors in R: Hello and welcome. In this lecture we are going to learn about Dustin R. So what does factor into? Factors are the data objects, like we have learned so far that everything is an object. And so factors are also data objects in R which are used to categorize the data. So basically factors are used to categorize the data and then store those, categorize data into the levels. So first it will categorize the data, and then it will create labeled Spot, dodge categorize data. And then it will store that labels, okay? It will store that data into those levels, right? And factors are used to represent the categorical data. And they can store both strings and integers. And they did an integer vector having a level. Suppose integer vector having a level, something like that. Okay? So basically suppose we have data, suppose a male or female. So we can store that. It is better to store that as a 0.1, right? So what it will do it suppose we have a data where male and female, some data population where we have our data for male and female, both. What factors will do it will create a category of data with male and female, and it will store that data in the category of male and female. And then easily you can find how many males are there and how many females are there, something like that. Okay. Let me open the file that I have created four vector. So I have already created a file factor in R dot R. And here I'll also how to create a factor in R. The first step to create a factor in R is creating a vector. So here what I'm doing, I'm creating a vector for facial profession. And here inside the profession vector there will be Dr. Engineer Carpenter, Dr. mechanic, pilot, DR. carpenter in Kenya like that. Okay. So there are a few professions I have created here. So this is the professor vector where I have kept the professor of the persons. Okay? So if I run these two lines, what it will do, it will create and vector for France and right. And it will have Dr. Engineer Carpenter, Dr. so see adopter is repeated the right. Dr. so many doctors are there, right? So suppose this is a vector that I've created. If I want to check whether this vector is our factor or not, I can use a function here called each factor. So anything, if we want to check if a factor or not, you can use each factor function and pass that vector or object to the each factor function and it will give you that it is a factor or not. Okay, so let's run this. See here it is showing us false. It means this vector is not an factor. This is a vector not a factor. So how to convert this professor and vector into a factor? That's what we're going to do. The next step. Next step is apply the factor function. So support. Now I want to convert this proficient with a factor. So I'm creating another, creating another object that it's factored underscore. Professor. Okay, so now I'll use the factor function and I'll pass this profession vector to the factor function so that this profession vector will be converted into the factor. Okay? Now, if I run this, and if I run this, what allocate, see here. Now, this step we'll create the factor from this professor vector. So now this factor, professor nasal factor, so I can check here is factor. Here it is showing us through. And when I'm printing this factor professional, which is a factor, what I'm getting I'm getting I'm getting the same result. Dr. interior carpenter, dr. Like that, the same data. But here I'm getting another output that is labels. And labels are carpenter teacher, Dr. so all those things will be no labels, right? The other levels are carpenter, Dr. driver in junior, mechanic, pilot, and teacher. If I use table function and pass this factor, the table function, what it will do, it will give us the same thing, labels, okay? And if I use summary, what we like it. If you use somebody, you will be getting how many carpenters are there in this data? So to carpenter, three doctors, one driver, to engineer, to mechanic, to pilot, and to teachers. Okay? So this way we can create and vector into the factor. Factor will see it has created a factor, carpenter Dr. and it is giving us the data that there are two doctors. There are two carpenter, three doctors, one driver like that. So it will categorize the data. Now, the factor, what factor has done? It has categorized this data, this data into the category of profession like carpenter, Dr. and it is giving us the numbers like these many carpenters or their main doctors when engineers are there in our data. So the factor will be useful while analyzing the data which are categorical data, right? So we'll see how we can do categorical analysis using factors in R. So for now, it is enough to understand how to create a factor in R from the vector, okay? So we can apply factor function to the vector and that vector will be converted into the factor. And the condition is this professor vectors to be having some categorical data. Okay? And then we can check each factor function. With the, each factor function, we can check whether a vector is a factor or not, okay? And we can use table to see the levels of the factor and we can use the summary function to check what are the numbers are there like how many are Dr. how many engineers are there? So somebody up the factor will be seen by using summary function, okay? 20. Data Frames in R: Hello and welcome back. So in this lecture we are going to learn about DataFrames in our programming and how we can use DataFrames. That what the topic of this lecture. So let me first tell you what is DataFrame. Dataframe is a table or a two-dimensional array-like structure in which each column contains values of one variable and each row contains one set of values from each column. You understand what I said. Dataframe is a table-like structure or two-dimensional array, you can say, in which each column contains values of one variable. Each column will be containing values from values of one variable, and each row contains the set of values from each column. Okay? And it is a special case of list in which each component of each component will have the equal length and each component form the column and contents of the components on the road. Understood. Each component form the column. Each component will form the column and the content of the component will form the roads. You'll understand when we do the hands-on, you'll understand it better. So a DataFrame in R will have following features. The first thing is row names must be unique. So row name should always be unique. So row names, so it'd be unique. You cannot have the same row names in a DataFrame, okay? So each row name must be unique and then column name not be empty. So column name, so not be empty. It should be a nonempty. So all the column names would have entries, okay? And data stored in a DataFrame can be off numeric factor are character types. So any of these type data you can store in DataFrame, numeric factor or character type. And each column, so it contains same number of data items. So each column contains the same number of data items. So DataFrame as a table or a two-dimensional array like structure in R, in which each column contains the value of one variable and each row contains the set of values from each column. Okay? So let's get started with the practical. Okay, so what I have done, I have already written program and the filename is DataFrame dot r. What I'm doing here, I'm creating a DataFrame. So DataFrame can be created as follows to what, how we create a DataFrame. We use. Data.frame. Data.frame function will use to create dataframe. And this is student is an object to which I'll assign this DataFrame. Okay? So data.frame. And then what I'm doing here, first thing, I'm creating a serial number. Okay? So serial number, I'm creating 1-5. And then eight, I'm giving, I'm using c function to create the age, okay? I'm creating a vector here, see 201-15-1030, 5.45. So A's will have these entries. Then I'm creating name. Name will have the many entries. Okay. So five entries, shaggy, Ronnie's today's John and Tom, and then I'm closing this. Okay? So this way we can create a DataFrame. So let me run this line first so you get a DataFrame. See here. Then let me print this. Student CEO. Now, the student is a DataFrame, right? And what it is containing, it is containing a row or column that serial number, age and name, because we have given serial number, then eight, and then name. And each column. Serial number and name will contain the values. What serial number will contain the value one to five, so 12345 serial numbers and then ages 201-15-1030, 5.40. And name will have these values, okay? So each column will have the values from name, name, variable. That's what we learned in the theory part. So this way we can create our DataFrame student, which will have the column serial number is a name, and it will have the row values. From serial number 1-5, ages 21. And this five is, we have given a name, we have a specified in the name variable. So these variable values will become the entry into this table, right? So it's a table-like structure. Okay? So this is what known as DataFrame in R. And we can see the structure up our DataFrame by using STR function. We can use STR and then we can pass the DataFrame. It will give us the structure of the DataFrame. The DataFrame had structure and we'll say dataframe. And it is having five rows from three valuable. See here 12345. So five objects, or five rows and three variables. Three variables are serial, number is a name. These are the three variables. The column names are known as variables. So three variables and five rows, five objects of three variables. And these variables are serial number, is and name. So serial number is integer, is is numeric, and navies of character, type and senior, each variable or each column will have same number of items like serial number is one-to-five. Age is also having five entries and name is also having five entries, right? So all should have the same number of entries, right? Okay. Suppose if I give six year, what will happen? Let me run this editing DataFrame. Because the serial number is having one to 6.8 is having only five entries, and name is also having five entries, right? So here, what error we are getting error in dataframe and it is suing arguments imply differing number of rows six and 6.5. So one variable is having six rows, six items and all others are having 51 leaf. So that's why it is showing us the error. So it should be 55 for each. Okay, So here you can put five and then when we run it, we will not be getting any error. And it will be running successfully. With the STR. We can get the structure of the DataFrame. Next thing what we are going to learn, DataFrame components can be accessed like a list or like a matrix. So first we'll see accessing Leica list. So we can use, if you want to access like lists, we can use either of these three. We can use our dollar operator or we can use the double bracket, or we can use the single bracket, okay? To access the data from the DataFrame. Okay, So suppose I want to access the name from the DataFrame is student, one of the names. So if I can do that with this student, and then inside the column, I can pass the variable name or column name. Okay? And if I run this, I'll get all the names from the DataFrame. Similarly, I can do the same thing with this dollar symbol, right? Dollar operators student, that is the DataFrame name, and then dollar, and then the column name or a variable name. So here, if I run this, I'll get the same result senior student dollar name. It will give you all the students name. Okay? And similarly, we can do this as well, is to rent. And in the bracket. Again, inside the bracket, we can pass the name. These three things will give you the same result. Okay? Now, we can do like here we have given name. Name is that third column right inside this DataFrame. So we can pass the column number as well, student three, so it will give you the third column. The name. Name is the third, third column in this DataFrame. See, if I get to, it will give us the age. Ca, it will give us the Is. This way we can pass the column number and fetch the data from the DataFrame. Now next thing is modifying the DataFrame elements. We can modify the DataFrame also how C you will give that student, that is a DataFrame name. And then here we'll keep the 11 means the first row, and then I'll give H. And I want to modify the first row is 291. Okay? So let me run this and let me print the student. See here. Now the first row is, is modified to 91. Earlier it was 21, now it is 91. So this way we can. Modify the frame DataFrame elements. We can add rows to the Roche component to the DataFrame. Suppose I want to add another row. Suppose here five rows are there, right? I want to add one more rows to the DataFrame Halloween, I can do, I can use the rbind function. Our function is used to add a row into the DataFrame. Inside the rbind, I need to pass the DataFrame name and then up to use a list. And inside the list I have to pass the serial number six, then age one-twenty, and then the names would be there are a variety, so nothing to be empty. Everything we have to give, there are three columns, so we have to give three columns here. Serial number is n, name. And if I run this, one more, rows, row will be added. Seizure six, one-twenty remedy has been added to the student DataFrame. Similarly, we can add column component to the column component to the DataFrame as well. And how we can do that, we can do that with the C1. C1 means Columbine are vitamins row bind. Okay, so cbind function, we used to add a column into the DataFrame. So inside the same thing, we need to pass the DataFrame name. And then we have to suppose a column I want to add as a country. So I have to give the column name country and then after use the c function and have to pass the country number of countries. So here I need to pass the six countries in them, right? 12345, okay? And if I run this, because the sixth value, we have not committed. So it is not coming year. Or in a DataFrame is having only five rows to, it is adding the Phi one, okay? So this way we can add another column, like contrary to our DataFrame. Next thing is we can assign the country in a list like manner, like lists like assignment, we can use student dollar country. It means in the DataFrame, we are going to add another column. That is, that will be a name country. And we're going to add the entries like these countries, okay? Six IN india, neutral and US, Japan and China. Okay? And if I run these two statements, I'll get the country has been added here and the country names are here. This way we can add a column in a list like assignment. Okay? Next thing is we can delete a component from DataFrame. So we can delete an entire column by using this thing. Okay, student, that is DataFrame name, dollar here column name, name, and I can assign null. So if I assign null, the entire column will be deleted. So let me run this and see you. Now. Serial number is and countries their name column has been deleted because we have made the name column as null. Okay, So this way we can delete the entire column. In the same way we can delete an entire row. So to delete the entire row, what we can do, we can use student and minus two, and then we can delete the entire row. So let me run this. Here. The row two has been deleted. Two is two footprint New Zealand that has been deleted. So if you want to delete a particular row, you can provide here minus two means the second row will be deleted from the student DataFrame. So this way we can delete an entire row from a DataFrame. So this is how we can create a DataFrame using data.frame. And we can provide the number of columns or number of variables. And then we have to provide the number of values for those variables is to be called number of items here five. So all A's and names would be 55, then only it will create a DataFrame. Then we can get the structure of a DataFrame by using STR function. Then we can phase the particular column using by providing the column name using this bracket or the dollar symbol. And we can buy column number. We can modify DataFrame like this, DataFrame elements like this and all those things, cbind, Irvine, all those things we have seen, right? So I hope you got the better understanding of DataFrames in R. And I hope I make you understand what is DataFrame and how we can walk with the DataFrames. See you inside the next lecture. 21. Combining Data Frames: Hello and welcome back. In this lecture, we are going to learn about how we can combine vectors into DataFrames. It means we suppose we have three or four vectors. And I want to create a DataFrame from those vectors. So how we can do that? And then secondly, what we'll try to do, we will try to combine the DataFrames as well. Okay, so let's get started. So first thing first, to combine the vectors, we need to create the vector. So here what I'm doing, I'm creating for vectors, names, city, zip code, and salary. So these four will be the four vectors that I'm creating are four objects I'm creating. And I'll, what I'll do, I'll combine these four to create a DataFrame. So these four vectors will create one DataFrame for me. Okay, so first rector is names. And in the name vector what I'm giving in the names vector what I'm giving, I'm using the C function and I'm giving the names of the Parsons lactamase, Rockies, Henry and monkey. And then the second vector is city vector. And inside this I'm giving this particular their respective cities named like Bangalore, London, New York, and Mumbai. Okay, and then I'm giving the gift code for these cities in the third vector. And then the fourth vector is salary, in which I'm installing their respective salary. So this will have the dermis salary, racket salary and recently and mantissa. Mantissa salary. These four vectors we can create by this that we have learned in the vector chapter in the lecture of vectors. So let me run this. So this will create these four vectors, okay? Now I want to combine these four vectors and create one DataFrame. I want to create one DataFrame by using these four vectors. So I want to combine these four vectors and create one DataFrame. So for that, what I'm doing, I'm giving a DataFrame name here imply details. So EMP dot details will be the DataFrame name. What I'm going to do, I'm going to use cbind function to combine these vectors. Okay? So because this name city deep code will be, that will be the columns team. So naturally I'm using C bind here to combine the column names, city, zip code. And sadly these are the four, this will be the four columns in the DataFrame. So I'm using c wine and I'm providing the columns name, like names, vector, city vector, zip code, and salary. Okay? So let me run this. Okay. Now let me print the employee details. Okay. So let me see the details of employers. See here. Now the employee detail is DataFrame. And we can see the columns name, city and zip code and salary. And the entries are dummies, bank load and zip code and salary, Rockies city, London, you've got this and the salary. Then Henry. See here from these four vectors we have created for, starting with these four vectors, we have created one DataFrames, right? Suppose if I put the names and if I run this name, what I'll get, I'll get one leader named similarly, city and the currents alley. So I have combined the vectors into a data frame. So now we have a table-like structure here that is a DataFrame. So now we have one DataFrame. Similarly, what will I do? I will read, you can use the C-A-T cat function to print something. Okay, so here I'm just printing the file DataFrame from four-vectors. And this, then it will come like this. Okay, so the first DataFrame from four-vectors is this imply details, EMP dot details. Okay? So if you want to print a headline, you can use the chat function, okay? And then print the employee details I have already printed here. And you can print it here as well. And then what I'm doing now, that next task is I want to combine two DataFrames into one. So we have one DataFrame imply details that we have created from the four vectors. Now what I'll do. Create another vector, sorry, I'll create another DataFrame, employee details to EMP dot details too, with the Hale-Bopp DataFrame function. And here, manually. Inside the DataFrame I'll give the name vector c t vector, zip code, salary. Okay? So this way also we can create, in the first step what we did. We have created the name city, zip code, salary vector separately, and then we have paused it to cbind function to create a DataFrame. And what we're doing now, we are directly passing this vectors. We are creating the vectors inside the DataFrame function. We're not aging C Y, and here we are using data.frame, DataFrame function to create a DataFrame. And inside the DataFrame function we're passing, we're creating names. Vector, density vector, then zip code vector and entries also we are giving here only, okay? Salary and a string edge factor false. Okay? And then if we run this, we will be creating one DataFrame. That is employee details too. Okay? And if I run this tool, what will get, we will get the second DataFrame that is employed. It tends to name strategic code and there are 3.2 lowercase RAM and push pop. And their city zip code and respective salaries printed here. Now we have the two DataFrames, employee details and imply details to what I wanted to do. I want to combine the rows from both the DataFrames and create another DataFrame that will be all employee details, that will contain all imply details from imply one and employed to DataFrame. So I'm creating another object, all dot employee dot details. Okay, this will be the DataFrame which will combine the rows from employee DataFrame, DataFrame and imply details to DataFrame. So for this, what I'm going to use, I'm going to use are buying out-of-band because we want to combine the roads here, three rows and here four rows to this four rows. And these three rows, I want to combine and create another DataFrame. That's why I'm using out-of-band to combine the rows. And then inside the Audubon, I'll provide the first DataFrame and then the second DataFrame. And then we can use the chat function to print the headlines that combined imply details. And then I can print. Okay, so let me run this. See you. Now we have all imply details DataFrame, which will be containing all seven rows, the four from the first DataFrame, and the last third is from the second DataFrame. So in this way we can combine two DataFrames. Okay? So what are the things we have learned? First, we have learned how to create a DataFrame from vectors. And then we have seen how we can combine the two DataFrames into one dataframe. Okay, So this way we can combine the DataFrames and we can create a DataFrame from the vectors as well. See you inside the next lecture. 22. Analysing Data in R from CSV file: Hello and welcome back. So in this lecture we're going to learn the very important concept, and that is data analysis in R. So what we're going to do in this lecture, first, we will try to read a CSV file, CSV comma separated files. And it's like actual seat you can think of. And it will contain the sum implied details. And then what we'll do, we'll try to read that CSV file with our programming. And then after we read the data, we will try to analyze the data that is there inside the employee detail CSV file. And we'll try to gain some insights from that data. Okay, so let me show you the CSV file first. So this is the CSV file that I have created. And we will see here this is the employee details dot CSV file and it is a comma separated file. So the first employee ID, employee name, salary, data up joining and department. These are the four column. These will be the column of the table or Excel seat, you can see. And then this will be the first row. The comma separated values. The one will be the employee ID, then roster will be that employee name. Salary will be for 3,000. They tapped joining this. And then department will be CSE or something. Okay. So these are the some data I have kept inside this CSV file. And now what I'll do, I will try to fetch this data, these columns and row values through our programming. And then we'll try to analyse the data. So first type is reading a CSV file through our, okay, so let me go to the code. The first thing we need to do, we need to set our working directory. So to set the working directory, you need to go to the files here, this section, and see when you click on these three dots here, it will take you to the you can browse the files on your computer. And here what I'll do, I'll go to the Art 2020 because this is my working directory and I want to set this place as our working directory. I'll select that. And now we are inside this art 2020 directory. Now I want to make this tax rate has a working directory, so I can do that. I can go to this more often. And aptitudes year, Set as Working Directory, so I'll click on Set as Working Directory. Okay, So this way we can make this our 2020 directory as our working directory. Alternatively, you can simply use the command set WD. Set WD is a function that will set the working directory. Wd means working directory and segments setting up the working directory. So setting work set, WD is the function inside that you can provide the path for your directory. And this path will be your working directory when you execute this. Okay? So now we have set the working directory. What I have done, I have kept the employee details dot CSV file here so that it will be easy for us to who read that. We need not to pass the complete path here since we have inside, we are inside the working directory and our employee tilde dot CSV file is also in the same directory, okay? And if it is not there, then we have to give the complete path here and complete part for this file. Okay? So first step is we want to read the CSV file throughout. So what I'll do, I'll first create a variable you can say or object in our employ data, EMP dot data. This is an invariable where I want to store whatever I'm going to read from this CSV file. To read a CSV file what we need, we need a function called read.csv. So read.csv will be used to read the CSV file. And inside this function, read.csv, we need to provide the dot CSV file name so that our file name is employee details dot CSV. That's it. When we execute this this function, it will read the data from this employee dot CSV and it will store inside this employee data. So let me run this first. See here. Now when you see here, it is showing us is showing us employee EMP dot data eight objects of five variables. It means there are eight rows and five columns in the CSV file. Employee id, one to eight imply name. These are the imply name, salary rate up joining. And it has, you can see the details here. So now we have read the CSV file and we have stored the data inside this EMP dot data. Now I can print this EMP dot data and see what detail it is containing. Just run it and see you. Let me run this again. Now we are getting the table I like structure, right? Emp ID, employee name, salary, date of joining, and department. These are the column names and these are the values, right? Rows. For the corresponding columns, right? There are eight rows and five columns. Five variables means five columns. So can you relate what kind of data type this is in art? This is called DataFrame. Dataframe will have the table-like structure, right? So whatever we read from the CSV file, it will be stored as a data frame in R. And we can check whether this is a DataFrame or not by using each data. Each dot data.frame function, right, is dot data.frame function. And we can pass this employee data object. So let me run this. See here it is giving the result as true. It means this employee EMP dot data is our DataFrame. Whatever we read through this, read.csv will get a data frame. So now we have a DataFrame here. The next thing is now we have our DataFrame table-like structure here. Now I can find the number of columns using an called function. So I can use an coal and then I can pass the DataFrame name here. And it will give us the number of columns in the DataFrame. So this CSV file is having five columns. Similarly, we can use and rho, and rho is number of rows. When we run this, we'll get the number of rows, eight rows in this DataFrame. See here how easily we have read the CSV file and we have created a table or a DataFrame in R. And now we are finding with n colon and row number of columns and number of rows in this DataFrame. Now, we are going to get some interesting information with R. So now I want to find the maximum salary. What is the maximum salary of an employee? So I can use max function and I can pass the employee data. Dollar salary. It means I want to pass the salary column, this salary column to the max function, the salary column, EMP dot data. Dollar salary means this column I'm fetching and I'm passing it to the max function. So that max function will work internally and find the maximum of these salary. So it will give us, it gives us the result 95,200. Let me print this max salary. See here we're getting the maximum salary of an employee is 95,200. Okay? So this way we can find the maximum salary. We can also find the average salary by using mean function. So inside the main function, what we will do will provide this salary column. And it will give us the average salary of the employees. Your average salary of the plaza. Every salary of him places 53,009, 24. And if we use sum of all the employees salary and divide by eight, because there are eight employees will get the same rate as CEO. So this way we can find the average salary of the employees. In the same way we can find the maximum. We can find the employee detail having maximum salary, we have found that 95,200 is maximum salary. One implies getting. Now we can find that employee with this detail. So we can use the subset function and we can pass this implies data. And then we can put another argument here, salary equal to max salary. So what we're going to do, we're going to find that employee who is having maximum salary. So it will give us the employee detail who is having with getting the maximum salary from the imply. Let me run this. See here this employee names deep is getting the maximum salary and these are his details. So this way we can put dot can be sunniest salary employee detail who is getting maximum salary. So this way we can find the employee who is getting maximum salary, highest salary. In the same way. We can also run this. Get all the employees working in finance, working in finance department and getting salary more than 85,000. See here, this is the department, so this is the department finance. There are two employees and both are getting selling more than 85,000. So what condition we are giving? We are giving calling the subset function and providing the employee data. And then we are giving a condition like department equal to finance and salaries would be above 85,000. So it will give us all the employee detailed whose department is finance and salaries, more than 85,000. Let me print this. Lets me run this. Here. We are getting imply. Let me run it again. So here we are getting the employee ID 3.8 means to the and run. These are the two employees who belongs to the finance department and their salary is more than 85,000. So these two employees hiring department, finance, finance department, and their salaries more than 85,000. Next thing is, I want to find the implies who's joined on or after 2000. So we have a date of joining here also, right? So I want to find all the employees who joined after 2000. Honor, after 2000. And so I can provide the employee data and I can give Ashdod date. And inside that I'll give the date of joining. So data up drowning I'm providing to this date function. And this should be greater than adds date to login. First January 2000. Okay. So all the employees whose data up Joanne is greater than this tutorial, First Gen to Tarjan will get those employees details here. So let me run this. Wait on it again, see here. So now we're getting the implies Priyanka Mahesh. So they meet us when these five implies they have joined in 2000 and after that or not after 2000, they have joined the company and we're getting the employee details. Who joined on, uh, on are often 2000s. So this way, we can analyze the CSV file and the data that we have inside the CSV file in Excel. So this is pretty cool analysis we can perform with the simple functions. Right? Now what I want to do, we have read a CSV file, right? We have read the CSV file and we have performed all these analysis. Now what I want to do, I want to write a CSV file. I want to get some data. I want to generate some data, and I want that data to be published into a CSV file. How I can do that? I can do that with this. Write.csv. Read.csv is reading the CSV file. And write.csv means reading or writing a CSV file. We want to write a CSV file. Whatever data, this data I have found as implies who's joined after 2000, right? So I want to write this table or this data into a CSV file. So I will pass this object joined after 2000 to this write.csv. And here I'll give the CSV file name. So I'm given CSV file name as employee dot twined after tooth origin dot csv. So all this data will be written into this CSV file. New CSP file will be created. Okay, So let me, let me see whether this file is already, see here, this file is already there. So what I'll do, I'll delete this. Okay? And then I'll go back and then run this. Okay, so let me run this. And let me go to the folder here. See here now we have a new file created. Let me show you the data. Here. We have 12345 rows. Has told me ran through Eddie that data up to n is 2000 2018, 2,914.2018. All these implies have joined after 2000. So similar data is here. And now that data has been put inside this new CSP file, employed dot joint afterthought and see how easy it is to get the result here and write that result into a CSV file that we have created just now through our throughout, we can write the results into a CSV file as well. And now I can read, use the read dot csv to read this CSV file again and print. So let me run this. Here. We are getting the same result here as well. So this way we can write us via CSV file and we can read a CSV file. And we can perform all these analysis like finding the maximum salary, finding the employee who is having, who is getting the highest salary. We can find the employees joining after a particular date. We can find the average salary of the employees. We can find the mean of the salary. We can find many other things, whatever required for your business. If you want to. Come to a particular conclusion by enlarging the data, you can do with the, with the odd functions and you can analyze your data. So I hope you got to know the strength of our programming and how to enlight data in the R programming and reading and reading and writing, reading and writing a CSV file. See you inside the next lecture. 23. Creating Pie chart in R: Hello and welcome back. So in this lecture, we are going to learn about pie chart. So let me first year these consoles and objects and let me close this file. And I have created one file that is called byte char Tatar. So I have written the program, I have written the functions to create pie chart using data. So we suppose we have some data and we want to plot that data into a pie chart. And we want to analyse the data so that we can do in our, very easily, an ad is popular for this feature as well, because we can visualize the data using various charts. So one of the guitar is pie chart and that we are going to learn in this lecture. So first thing, first, first thing, we need to create the data for the chart. So I'm creating a vector n and I'm giving values like 307-82-3309, something. You can give whatever you want. And then I'm creating labels a, B, C, D for this value. So value will be 30, 78, c, 23, and D 39, like that. Okay? Now I can get the data file name for the chat, whatever chart I'm getting I'm creating, I can give it a name like abc dot PNG or a pie chart dot PNG. So suppose I'll give it a name by Jarred dot PNG. Anything you can give. Okay. So I'm giving it a name. I can use the PMT function and I can give file equal to the filename. So I want to create a pie chart dot PNG image file. Whatever grabbed that'll create, that will be stored in a PNG file image format. Okay, So that also can do with this line. So this is optionally, if you want PNG file on one image file, then you can create with this. Otherwise you can directly come here and you can use the pie function and you can pass the data. And so this will go into the data and then you can pass the labels. This is the levels. So when pie chart will be created with this data, and this data will be labeled with a, B, C, D. Okay, so and then dev.off means this file will be written off on the, on the system. Okay? So let me run this code here. So let me run. Okay. So CEO, now we have but we're not seeing that chart here. I don't know why. So let me run this part again. See here. Now we have a pie chart with the labels a, B, C, D. So E is tardy. V is the most like 78. That's right, the bigger area for b and then d and C. This way, with this simple data, we have created this graphical representation. And let me go to the art grantee and CEO, there must be one file created. See here pie chart. That PNG file has been created with that graphical representation. So see here, this image file also created and see here the a, B, C, D. These are the labels and this is the data value is 78, is something that we have given. A value is 30, and C10 is 23, 29 like that. Okay, So see, we believe we have created a graphical representation of our data and we can present our data like this. Okay, so let me go back to the core. So this is the command, this is the function PNG to create a PNG file from our graph. Okay, so this is our graph and this is our PNG file for that graph. Okay? So if you want to send across the chart, you can create a PNG file and send it over the email. Okay? So this way we can create a pie chart. Okay, next thing is, now I want to create another pie chart. And for that I'm creating like implies and their salary. So I'm creating a salary vector and I'm giving those salaries impulsively to it and it supports but our salary and the names of the employees every sick but one date and all those things. Okay? So these are the two vectors I'm creating salary and the names. So obviously it will have this elite wonder it, but been 800 like that. Okay? And the same with B and D. I'm trying to create a PNG file, salary dot PNG for the graph that will draw and the by salary and names I'm passing and passing the salary as a data and name such as labels. So the next pie chart that we are going to create, it will be created based on the salary and the names will be labels like here, ABCD did in these levels will be there and be saved. But BUN and those things. Okay, so let me create this deal here. Let me run this. Okay? This again. See here. Now we have a pie chart with the names like obviously the park j tan. And what is the usage of graphical data are plotting. Because with this, we can see that, okay, titin is Aaron less salary of music is having less salary, but one N far oak is having fairly good salary. The buck and Ellen is Erin every salary. So with this graphics, we can easily analyze the data without looking at the data in a deeper we can see this and we can come to conclusion that music is having the less salary after that, Satan is having little more than every sick. And then the buck and Ellen, and then bobbin and fabric and fabric is having the most, the most of the cellular elements of our lucas having photocopies getting the highest salary, rs1,000 estate. So we can do this analysis by seeing the pie chart and see here, there must be a salary file has been created here, see here. So this you can send over the email and you can put like that. Okay. So this is the huge pie chart. And with this simple pipe function, we can create a pie chart by and we have to write the data and we have to provide the labels. This name is a label that we're good. Thank you. Okay. We'll see the rest of the things like how we can analyze the CSV file detail in the next lecture. So I'll see you inside the next one. 24. Analyzing Employee Data: Hello and welcome back. So in this lecture we're going to analyze the employee details dot CSV file that we have seen in the previous lectures. And we're going to get the data from this CSV file. Read the data from the CSV file, and we're going to create a pie chart. And we'll see what are the things we can do with the pie chart. What are the graphical things we can analyze with the pie chart? How we can enlighten the employee details. From the pie chart. To read a CSV file, we use the read.csv and we pass the CSV file here. And I'm going to store that into the EMP dot data. This we have already seen. So I run this and we get the data into the EMP dot data object. Now, this will be a DataFrame rate. Remember, right? So if I print this employee data, EMP dot data, it will get also data frame. They will like structure right here. This data we are getting from this CSV file. Now, I want to draw, I want to get a pie chart like this for this implies, okay? So to do that, I can use the pipe function and I can create a pie chart. So I want to create a pie chart based on the salary. So what I'll do, I'll use EMP dot data, dollars salad, all those elements. Get the salary column up this DataFrame. And it will create a pie chart and label what I'll give, I'll give the label as employee name. So EMP dot data, dollar, E&P underscore name, employee name. So it will create a pie chart like this, and this will be the salary of the employee and the levels will be that implies name. So let me run this and see here we are getting pie chart where these blocks will represent the salary and the levels of implies theme. So with this pie chart, we can clearly see that. See that. So D is drying this month's salary. Randolph was drawing this must salary, right? So deep salaries maximum rate. And we see here, so deep salaries maximum 95,200. And my hair is drying the less salary, right? So if we see the salary, 7,800, okay. So this way we can get pie chart based on the salary and employee name and the labels. Okay, next thing is I want to plot the chart, title and rainbow color palette, so I want to make it more colorful. So here what I'll do, I'll use the pipe function. Pass the salary, then imply name. So this is the data, then this is the label and mean, mean equal to employee salary. This will be the heading for this this chart. Okay? So this will be the heading For this chart. And then what I'll use, I'll use call equal to rainbow colored is called rainbow. And for that, what I'll give employee dot data length, okay? Then tap the employee data. Okay, so let me run this. Now. We are getting a much colourful graph or a pie chart where the heading is employee's salary and the names are the labels and the salary. These blocks are representing the. Implies salary. Okay? So this way we can make a colorful pie chart, which is looking much better than the earlier one, so you can use this as well. Next thing is I want to create the labels. So what I will do, see, yeah, we're giving the levels as employing them that is already there in this DataFrame right? Now I want to create my own levels. So what I'm doing here, I'm going to create percentage of the salary, other labels, okay? So what I'm doing here, I'm creating an object five per cent, and I'm using the round function. And what I'm doing here, hundred into employee salary divided by some of those salary of all the employees. Okay. So basically what I'm doing, I'm finding the percentage of births and deaths salary of each employee based on the total salary. Total salary will sum all the salaries. And then we'll try to find how much Susanna is getting out. What is the Susanna's salary in percentage? The total salary. Okay. So we're getting this pipe percentage value. And then what I'm going to do, I'm going to plot the dentist salary in pie chart. So for that, I'll use the pipe function. And here data will be the same salary of the employees, this column, salary column. And then labels will be the pipe per cent. So the label instead of name now I'm providing five per cent, okay? So it will be 1%, 2% like that. Okay? And then the main heading, I'm giving, parsing this salary by chat, okay? And then the Collette, I'm keeping the same scheme, rainbow. And here I'm giving the lenses and data, employee data, salary, salary and providing as a length. Okay. So let me, and then another thing I'm giving, I'm giving the ligands, ligands or something will be printed here to specify what does it mean, R, which color will OXO, which imply. So it will be posted on the top right corner. And then here I'm giving employee data. Dollar, employee name means the employee name. And then I'm giving the cex means that it will be the value of the length or breadth of that thing. Once I draw this, you will be understood. So let me, you will understand one star graph is coming C here. Okay? So let me run this as well. And now let me see. Yeah, Now we stood up the name. We are getting the percentage numbers like the sum of total salaries of all employees. This person, the green one who is so deep, who's getting 22.1% of the total salary. And then run our eyes getting 19 pointed. So this, this legion, this is called lesions. This, this is the allegiance. And based on the color green blocks, those are the bread belongs to the prion. So with this graph, you can understand that Priyanka salaries learned 0.6 per cent of the total salary, right? In the same way in this belongs to the Michael, Michael cell is 25.6. And the lowest salaries of this person who is riba you are seeing is having less salary, 0.2 per cent. So this way we can create our own labels and we can put the legends for each level. And this is more specific like how much 1% is getting in percentage, percentage salary. Next thing is we can draw the 3D pie chart. And for that we need to download this library plot matrix. Okay? So if we're not, if it is not there for your RStudio, you can come to the file. You can go to the packages and you can click on the install packages. Here you can give your name, started library name, which you want to download and click on the install. A party is not there. You can click on Install and this library will be installed on your machine, RStudio inside Data Studio and you can use that. So we need this library, plot tricks to create 3D pie chart. And inside that library we have the pie 3D function. And with that we can pass the data, that is employee's salary. Labels will give the employee name. And then explored point to and main is placentas salary in 3D. Okay, let me run this to see, and now we are getting 3D pie chart. Right here. We are getting a 3D pie chart like this is the summary of Sudan. This is certainly up meat antibody Michael traversing C, very less salary for fewer things or the mass like this. So this is the 3D representation of the employee salaries. And so we can see here, and this will look much better than the 2D pie chart. So this way we can create a pie chart and we can analyze the data in art. Okay? So you can also try with your own data. You try to create your own vectors and then try to draw the pie chart. You can analyze the data from a CSV file, create your own CSV file, and do all those things. So I want you guys to create your own project like this, and both those graphs and these pie charts into the project section of this class. And we will see that we can provide our feedbacks and all. So it will be shared between us, all the students and with me also. So you try to create or create your own data like this, and try to visualize your database by creating the pie chart or a 3D pie chart are like finding the mean, finding the average salary, all those things. Okay. So and pushed into the projects excellent. 25. Reading excel file in R: Hello and welcome back. So in this lecture we are going to learn how we can read an Excel file through art. It is pretty simple and we need to install one package, and then we need to load that package. And then we need to read the Excel file through a simple one-line code, and then we can print the data. So let's see how we can do that. So what I have done, I have created a same employed detail, Excel file, the same retails I have kept into this Excel file that we had in our CSV file. So it is same data, but I have created an Excel file through that CSV file. And what I have done for that, I apps simply open that CSV file. What I'll do is I do not have the Microsoft Excel here. So what I'll do, I'll just go to the and here we have employee details dot CSV file, right? I'll just open that here in Google seat. And we will see that it will be created at x. And if I see yes, so now we have the column name plate, employee ID, employee name, salary, date of joining and department, and all the details in the Excel format. Now what I'll do, I'll just download this file in Excel ethics format. So click on that Microsoft Excel format, download it. So now it has been downloaded and I have kept that. I kept that file here in our working directory. So next thing is now we have this employee details dot Excel as x, y naught. That means that Microsoft Excel file here in our working directory, that is our 2020. So what I need to do, first thing we need to install this package. And the package name is XLSX. So to install a package in R, we can use this syntax install.packages, and we can provide the package name that we want to install. When you run this command, it will install the excel ethics bracket. Alternatively, we can also go to this package tab here. And we need to click on the Install. And here we need to provide X as X, okay? And you can click here Install and it will be installed. So we can do either way, and the XLSX packers will be installed. This package is required because we need to read that Excel file through our programming. And we need the libraries that are built-in into excel package. So just click on Run and this will be done. Okay. So what I'll do, I'll just cancel it because I have already installed this package. You just click on this and it will be installed, okay? And you can go here and you can do that also. It will take hardly a minute up for your time and this package will be installed. Next thing is an opera that we need to move this thing because it's still not be there in our script file. After you install the library or the backend, you have to delete that line. It's still not going to the liquor Other Script fight. So next thing is we want to read or Excel ethics files. And there is a function in this library. So we need to load the library, we need to load the package, so we will use library and then we'll keep the library name XLSX. And then what I need to, I need to use the function v dot XLSX. It means that we want to read an Excel file, Microsoft Excel file. And here, first argument, we need to give money to give the file name. So the final image, employee details dot XLSX. Okay? And then we need to give the seat index, index equal to one. Okay? And this, whatever we are reading here through this red dot Excel, we are storing in sight or object inside an object. Employee underscore data, E&P underscore data. So this will have all the data that we're reading from this Excel file. And then we will simply print that. So let me run this file and see we are getting the result here, employee ID, employee name, salary, they end up joining and department. So this way is pretty simple. We can read through the we can get the older access it, we can read that right? So pretty simple. Load the library. First install the package, then load the library. Then we have to read the Excel file by simply providing the Excel file name. And we have to store this data in object into an object. And then we simply print that object to get to see what that object is storing. It is basically storing all the details from the Excel sheet. Okay, I hope you got to know how we can read an Excel file. So we can read an Excel file by using read dot XLSX function. And we need to provide the Excel file name. And then we will simply get all the data inside the Excel file. So the package is XLSX that we need to install. And we need to load the library XLSX before we use the XLSX function. So this is the way we can read an Excel file, Microsoft Excel file through our programming. You inside the next lecture. 26. Reading xml file in R: Hello and welcome back. So in this lecture we are going to learn how we can read an XML file through our programming. So we will write a code and through which we will read XML file. So first thing first, let me tell you what piece. Xml file. So an XML file is an XML. Xml means XML Extensible Markup Language and insert we call it XML Extensive Markup Language. And it is similar to like rewrite HTML, hypertext markup language, you know, for writing web pages and all in the same way, XML Extensible Markup Language. It's a data file. And in this file we keep data is like a database, like we store data in the Excel sheet as well. We store data in a text file as well, and we stored it in the database as well. In the same way, we store data in XML files, that is Extensible Markup Language. It is a formatted like an HTML document, like in HTML document, we use the markups and all to store the document in the same way. Xml also, we use the markup to keep the data inside the file. But here we use custom tags. In HTML, everything is predefined. But in XML we can create our own tags like HTML. We have to use the older predefined language tax that is specific to the HTML, but in XML we can write our own tax, whatever we want. We can create a tag for that. And that's why you select custom tag based language or a file extension that weekend. I'll file markup language that we can use here custom tags to define objects and the data within the each objects. So we can define the custom object and we can put the data according to that, whatever data we want to port for the tags we can do. Xml files can be thought of as a text-based database like MySQL. We use the youth to keep the data in a table and raw format and we decide what kind of column we want to insert in the same way It's text-based database. So now what I'll do, I don't want to create an XML file from the scratch. And I want to use the same our details, same details that we had in our CSV file, employee details dot CSV, and then we have clear to the employee detail dot Excel file. So I want to have the same kind of data, same data in XML agile. So for that, we can go and create manually an XML file. Otherwise, we can also use this conversion tools dot IO website, where we can just browse the Excel file and we can convert that to an XML file. So let me show you. So when you come to this website conversion tools dot edu slash convert slash Excel to XML. So this phase, when you come here and we, when you browse and just put your employee details Excel file here. So now I'm using this employee details XLSX file to create an XML file based on this file data. So you just need to select the file. And then we have to click on the Run converge on this Excel file will be converted to an XML file. And it will take few seconds and your XML file will automatically get ready. So what I have done, I have already downloaded this XML file here, employee details dot xml file. And we will try to read this XML file. It is having the same details as employee details, right? So okay, so what I have done, I have modified it and I have kept only for records here. And if you see the file which I have just created inside downloads and this XML file, you will see data. Eight employee ID it up to it. Is that okay? So like that, we can do that. So let me copy paste these two hour. Yeah. Okay. So to Lake rehab installed the package XL XX for reading the Excel file. In the same way to read the XML file, we need to install the package XML. And for that, we need to use install.packages and we need to provide the package name XML. Alternatively, we can go to the install package, and here we can provide the package name XML. And we need to click on Install and it will be installed. So I'm not going to re-install it because I've already installed that. And if you haven't installed it, you just run this line or you can come here and provide the XML here, and you can click on Install, and it will take few seconds to install the package, or hardly a minute to install the package. Once the package installed, we need to remove this line because package is already installed. Next thing is we need to load the XML package or library. So we need to use the command or we need to use the script library. And we need to provide the package name. So we need to load the package, okay, so library. And we need to provide the package name XML. So all the functions and required to read XML file will be written inside this XML library. And that all the inbuilt functions will be our level for us. Next thing is we need to and the library methods as well. This is required for this XML package. Okay, and then next thing what we need to do, we need to use the function XML parse. Xml files is a function that is required for reading, reading an XML file. So we'll use XML parse and we need to provide the file name for the XML file. So file equal to the file name. So our final image, employee details dot xml. And we need to, whatever it will fetch from this XML file, we need to store that inside this employee details object. Because in R we need to store everything inside of an object like variables you can say. So imply details, variable or object will keep whatever we are. We get from this file, XML file through this XML parse function. So now all the details from this implied details dot xml file will be stored into this object. Now we need to print that object. So let me run till here. Okay, let me see here. It is printing the XML file like an accordion Bly imply ID1 and Glassdoor salary date of joining department. Okay, So see here these are the user-defined tags that we have created for our XML file. Okay, See you all the details we are getting so we can read the XML file like this. Xml parses the function to read the XML file in C. Or we can use the function XML food node, the root node up the XML file. So we are urging the root node and we are passing the employee details, E&P underscore details object to the root node and then we will print this root node. So let me run these two lines. If I put root node one, it will give me the first node details lattice imply ID1. If I put one of two, it will give me the name. The first row to see you employ name priyanka Rostow from the first node. If I put two up to what I'll get, I'll get the name from the second implies that is the employee ID. Employee name is. So like that we can access each node. I can put three of one. Suppose three of three will get salary of the tarred him play, right. And in the same way we can find how many nodes are there, how many Norths are there in our XML file for that, we can use XML size and we can provide the root node object that we got from the XML root. So when we run this, we'll get done number of north. So let me print this. Number four because this Excel file I have kept only for employer it is. So it is showing us that there are four Norths are there. We have four nodes when implied, one in bright to employ D3 and employee ID for. So it is giving us the what up north in the XML file. Now the main thing is once we get this data, XML data we can read through R. We need to convert that to the DataFrame because it is very easy to read data or manipulate data when it is in the DataFrame format, right? So in Are there is a function called XML to DataFrame. It means whatever we have in this employee details dot xml file, we can directly convert that to work DataFrame by using XML to DataFrame function and all the details. We'll store it in the employee underscore DataFrame and then we can print it and we'll see that it will come like DataFrame. So let me run this and see now all the details are in a tabular format or in a DataFrame format in our senior employee ID, employee name and salary data planning and department. So with this, with this single line of code, we can convert an XML file toward DataFrame in R programming. And this is pretty handy when we do our exploratory data analysis and machine learning and data science. So we can easily convert XML file to a DataFrame and then we can proceed further to analyze the data or the way we want to create graphs and all plotting and all. So all those kinds of analytics we can do on this DataFrame. So it's pretty easy to do all those things in our programming. I hope you got to understand how we can read an XML file. How we come to know that, how, how many nodes are there with this XML size? And then how we can convert an XML file to our DataFrame in R using XML to DataFrame, right? So that's it for this lecture. See you inside the next one. 27. Reading JSON file in R: Hello and welcome back. So in this lecture, we are going to learn how to read JSON file in our programming. So what we'll do, we will learn what is JSON file, then we'll create a JSON file. And then finally we'll try to read that JSON file through our programming. So let's get started. So first thing we should be knowing what is a sulfite. So Jason stands for JavaScript Object Notation, and it is an open source standard file format. So it is open standard file format and data interchange format. So basically Easter file format as well, and data interchange format as well. That means we can store the data and we use it for the exchanging the data. Also, that you just human-readable text. In this, we write human-readable text to store and transmit data object consisting of attribute value pairs. It means it will be like We will first key value pairs, all those things site. If you know the MongoDB or any no sequel databases, you must be knowing there is a document database where we keep, we store data in the form of key value pairs. So there will be a key, and for that key, there will be a corresponding value in the same way just on also you just the attribute value pairs. It means there will be an attribute and that attribute, there will be an attribute, value pairs, and added data types. Basically any other values as well, and basically edited that type. So I, I hope you got to know what is Jason in theory, let me know how we can create a JSON file. So see you soon. File is a very simple. We need to put data inside these curly braces. And inside the curly braces, what we will do, we will use that key value pair to store data. So suppose what I'll do, I'll store the same this employee data only. We have seen how we store this in a CSV file like here. And then we have seen how we can store it in an Excel file. And then we have seen how we can store that in an XML file. And now we'll see how we can store the same employee detail in a JSON file. So for that, like I said, you saw the best file format trick. Here, employee ID. All the employee IDs, I'll store in an array one to eight. So all the employee IDs from one to it, like we have eight employee ID. So we have, we have restored all those in. And we have given attribute name is employee ID, and the values are one to eight, right? And this colon is the left side of the colon. These colonists separating the key and value are attribute and value. So left side of the colonies, employee ID and the right side is corresponding value. In the same way we have employee underscore name, that is an attribute and all the employees names, this Priyanka will be corresponding to this imply anyone or to sit last two are I like React there are still parts one through four to my haste for for what you are seeing like that. Okay, then we're storing the salary array, then we're storing the data training at it and then department at it. This way, in the department, we can put data into the JSON file and I have saved this Azure employee details dot JSON. Now we know how we store data in JSON file. So next thing is we will try to read this, gets on file data through our programming. So let's do it. So I'll open, I have already written the code. So I'll open that reading JSON file that I have written. And for this, we need to install the package. Our guests on this, on this back-end Minister installed. Or you can install it by running this line of code. And you can come to the install package. And you can put here or this one. And you can click on the Install and it will be installed. So you can use this command script as well. Then we need to use the library or the axon. And then we need to provide the JSON file name. And JSON file name is employee details towards the Sun. And here we are going to use a function from this library are Jason, That is from Jason. It means we are going to read the objects from our details, from the employee details dot js on this JSON file, file equal to the final name you need to provide. And this from JSON function will read the data from this JSON file and whatever it reads really stored at in this object E&P underscore details. Then we'll simply print it. So let me show you here. Let me run this and see here it is reading the data like employee id, one to eight employee names. These are the employee named than the salary, and then comes the date of joining and then the departments. So see here how we live with one line of code from just one simple function. And just we are providing the filename and it is reading all the JSON file data. Next thing is now with this weekend. Print the data. So we can see here it is printing the data from the JSON file. Now, as we know in our, it is pretty easy and it is pretty recommended to get the data in a DataFrame format. So we can simply now the JSON file data is in this object in E&P underscore details, we can convert that into the DataFrame by using agile dark data dot frame. So when we use this function as.data.frame and pass this object with just getting older File documents or the data. This object is getting all the details from this JSON file. And when we pass this employee retail object to the as.data.frame, it will convert this data. Now the format is like this. It will take this data and it will convert that into a DataFrame in R. And we are restoring that details our DataFrame in E&P underscore data underscore frame. And when we print this will get the data in a DataFrame format. So let me run this. See here now we are getting data in our DataFrame format in R. So it is pretty simple that we can read that JSON file with the from JSON function. And then we simply pass this object, which is getting all the details from the employee details or the sun and convert that into the DataFrame by using this simple function as.data.frame and passing this object. What this JSON object, or a Python object which is carrying the data from the JSON file, a DataFrame in R. And simply we can print that hand CEO, how beautiful it is transforming into a DataFrame. So this way we can, we can read our JSON file and convert that JSON file data into a DataFrame in R programming. And for that, we can use this DataFrame to analyze the data and do further processing, are creating plots and all whatever you want to do you can do with this DataFrame. So this is how we can read JSON file using our programming. See you inside the next lecture. 28. Creating Bar plot: Hello and welcome back. So in this lecture we are going to learn about another visualisation chart or graph that we can create using R. And that is bar plot or a bar chart, as you can see, bar graph. Bar plot or a bar chart or a bar graph is one of the most common type of graphical visualization that you must have seen in your office or in your projects and all. Whenever we try to visualize something forecasting, we use it bar chart. And when you see this chart, this is a bar chart. And you'll realize that you must have used it for many, many times, right? We could get same kind of bar chart in our Microsoft Excel as well. And when we try to visualize something, even when we draw something on a pen and paper. To visualize something, we most probably we draw bar plot. So barplot is one of the most common type of graphical visualization. And it's towards the relationship between a numeric and categorical variable. So what does it mean? It means that the CEO, the a, B, C, D, this x-axis will be like something that will be very categorical like. Now, what is the like weeks or days in a week? Like Sunday, Monday, Tuesday, our January, February, March, April, May, June, like that. And this y-axis will be the numeric representation for these months, like revenue of an organization or salary of employees. So this x-axis, ABC will be the employees, and this will be that their respective salary. So this excess will read the numerical part and this will be the categorical part, right? So each entity of the category variable is represented as a bar. So this is the bar. And that's why it's called bar chart, because the numeric value is being shown as a bar. So that's great, is known as the bar chart and size of the battery present the numeric value. So this size is represented like this is somewhere around flipping. This is somewhere around 13, This is somewhere around seven, and this is somewhere alone, nine, and this is some decision. So E value is 15, b value is seven, c value is nine, the value is eight. And this e value is 13. So like that. This body size will represent that numeric value. If ABC or employee, this is their respective salaries, right? So this way we can use the bar plot and in R we can draw bar plot by simply using bar plot function. So bar plot function. And inside the function we'll just provide some, but amateurs and our data. And it will create the bar plot, this function bar plot to create bar charts. It can be both vertical and horizontal bars. And the syntax is barplot and it will take the argument th. These edges will be the vector, data vector. The vector which will contain all the data. And this xlab is X axis, Y is y-axis, like this, a, B, a, a, B, C, D will be the x-axis, xlab and y will be the y-axis, this main. And then the name ox will be the, will see what is this? This is, this will be the like naming like this. 123, like let me tell you this. This zero to 14, like the employee's salary will be the data. And these names that are will be the name of the implies. Okay? So this edge and named start out both the vectors to have those same number of data, right? Same number of data number operators would be same in the edge named start. Ok, OK. We'll see that HE, the vector or matrix containing the numeric value of the bar chart. As I said, xlab is the level of x-axis. Why love is level of y-axis? And mainly the title of the bus. So here with this will give the title of the bar chart like employee's salary. And then named start argues a vector up names appearing another each part, like I said, for this bar, this bar b like that. Okay? And the call is used to keep the colors to the bars like here we are given different colors. So if you want to make a chart colorful, we use the call. Argument, Okay, Henry, past that. Okay. So here's a simple example. What I'm doing here. I'm creating a new vector and it will contain this, these many numbers. Okay? So this vector will contain these many numbers, okay? And I want to create a chart for this. So what I'll do, I'll just simply create this Q vector and I'll pass this Q vector two bar plot function. And it will create the bar chart for us. And this one, PNG file equal to a b, not a PNG equal to the file name. You can create an image file up the chart or bar chart, and you can save it on your system. And for that, we use the PNG function. And inside that we'll use the file Col2 and whatever name of the file you want to give that you can give here. So based on this data or charter will be created and the chart data will be saved in this image file. And then we'll use the bar plot function to draw the by bar chart for us. And then dev.off means we'll save the file and it will be written on our local machine. So let's go to the RStudio and run this code. So here I'm creating a vector for data input. So based on this data, the bar chart will be created. Now I'm creating a file to write the bar chart on that. I'm giving PNG file equal to and here you can give any name. You can give abc, abcd, not PNG, whatever name you want, you can, okay? So I'll give a, b, c, d bar chart, okay? And then we'll use the bar plot to create a bar chart, and then we'll save it, right, dev.off, save the file. Now I'll run the whole thing and see if for some weird reason the plot is not being sold here. But as you see here, now, our bar chart has been created. The file has been created here. See you. So okay, 12345678 entries, right? And here we have 12345678. The longest is 9,001.99 thousand. So this way we can give you a double bar chart. So this is a simple bar chart that we have created based on this data, right? Next thing what we can do, we can go a little further and try to create little more interesting. I'm going to have, so what I have done here, I have created a data like B, a vector or a data which is containing the babies born in each month like B will contain V vector will contain the babies born in month like January 5,600, February to March, 7,800 babies born like that up to the summer. Okay. So this is the 12 month data for the babies born in a particular month. And then, so this is the numeric value. And, and what I'm doing, I'm creating another vector M, which will contain the month name, right, for each value, the corresponding month name vector I'm creating that is January, February, December it will contain, okay, So this is clear. I'm creating the numeric value here and the name for the each bar I'm creating here for January factorial 600 like that. Okay? Then I'm creating a file, babies born dot PNG. And I'm passing that final aim to dot PNG function. Okay, collecting duct chart, image file name. And then what I'm doing, I'm simply plotting the bar chart based on this data. Okay? So what I'm doing, I'm forced bussing the V value. So I'm inputting the data like based on this data, I want to create a bar chart, okay? And then names.org means for naming the each bar I'm passing this m vector. M vector is scattering January, February. So the x-axis will be January, February, and y-axis will be the number of babies born in a particular month. Then xlab, I'm giving them month. And y lab, I'm giving babies bond in that month. Then color I'm giving green. And main means the hitting of the chart. I'm giving babies bond chart. And then for the each bar. I'm giving a low, okay, and then I'm saving the file. So let me run this. Okay, so I have done it correctly. Let us see that the senior, now we have another file. Baby is born dot PNG. Now. We have this January, February, March, April, May, June, July. It was like this. And then here we have the maybes bond data lake January, somewhere around 5,000 something babies born February somewhere for dilute, something like that. Okay. So most number of babies born in November and then July. That is the, by seeing this bar chart, we can come to know that norm. What is the month when most number of babies born and after that? July. So July and number, first number and then second is July. See here for July 7,304. Now on buddy 9,800. I started July 9 thousand and number 9,800. So this way we can give me the support. I want to change this color to red. And if I run this, see now our chart is converted to read, right? You can see there is a little border with yellow color. Let me change it to green so that we can see that can be early or make it clean and run it again. This file, see, now it is coming clean. The Barnard is green. So this way we can create a bar plot or a bar chart using odd. So I hope it is clear for the numeric and for the name, name.org. I'm passing this m vector, this m vector and B and bossing other main input vector. And then xlab, month. And why Lab babies born and color red may uncharged. Hitting is baby born chart and the bottle is green. Then dev.off will save the file to our local machine. So I hope it is clear that how we can create a simple bar chart and how we can create this kind of bar charts. So we have seen like we have created this. And then we have James's to read, write, and we have created this simple bar chart as well. So this way we can create a bar chart, right? See you inside the next lecture. 29. Stacked Bar Chart in R: Hello and welcome back. So in this lecture we're going to draw out, we are going to visualize a very interesting bar plot that will give us a very good real wealth. What is going on in an organization like quarterly, they charged region wise? How we can plot using bar plot. That's what we are going to see. So to do that, what I have done, the quarterly, the child, what each month actually, what I have done, I have created a matrix, okay? So I have all this revenue I haven't kept inside the matrix. So I've created a matrix with the revenue for four months. Quarter one, quarter, two, quarter three quarter fourth, quarter, one quarter, two, quarter, three, quarter four. So these niche what each quarter and four quarters are there, right? So for each quarter, these are the revenues. Okay? So I've created a matrix, and what I'll do, I will pass this matrix as a input data to our bar plot. So I'm passing the M here and then the main total revenue. And then named start on quarter, I'm passing quarter. Quarter. Quarter is a vector which is containing the value Q1, Q2, Q3, Q4. Okay? So this L bars as well, names.org, and then xlab will be quarter, the name and the white legs. Wildlife will be the revenue. And then colored. I'll pass. Another vector. Colors I want to draw for. The bottom will be like colorful. Once it is drawn, I'll explain you four colors. I've created a vector and it is continuing value, blue, pink, yellow, and green. And then the quarter. And then what I've done, I've created the regions, regions like east, west, south, north. Okay? So that also I have okay, and the matrix, what type created four rows and four columns. And I'm arranging developed by rock and figure out what I'm doing for legion. I'm adding the legend also top-left. And what I'm pretty lenient I'm giving their region is filled colors. Region sees this region south, east, west, south, north. Okay. And I'm feeling with the colors, each color as well. Yeah. Okay. So let me run this first. So see here, this is the matrix and see what quarterly revenue. So this kind of bar chart or bar plot we are getting here for quarter one. This is the quarter TO, this is the result or driver new quadrat three this and quarter four, this thing here, this blue, pink, yellow, and green for these colors, I have created a legend theorem and saying that blue is for South region, yellowish for each region. And green is for West region and not pink region. And South is blue, is this yellow like that? Okay, so by seeing this graph or the bar plot, we can easily find that digital quarterly results for each region. South. This is the revenue for quarter one for in the same way for the North, this is the revenue for the east. This is the revenue for quarter two, quarter, three, quarter, four like that. So it is pretty easy to find and visualize things using bar plot, right? And how we then that first we have created our color vector, then quarter vector, and then region vectors and then reagents I've used in the legend. And metrics upgraded four-by-four. Okay. And here I'm creating a quarterly revenue dot PNG file. And here I'm passing the quarter, this quarter as a named and then New, and I'm filling the colors bypassing the colors vector. So this way we can get it Plate which layers and using our bar plot. Okay? So you can also play around with your data and try to create some beautiful, good-looking, colorful barplot. Stevens cited the next lecture. 30. Boxplot in R: Hello and welcome back. So in this lecture, we are going to learn about box plot. Box plot is a method for graphically depicting groups of numerical data through their quiet tiles. I'll tell you what are these quartiles? So it's basically a graphical son of numerical data. Group up numerical data like we do in a bar plot and all in the same way. Boxplot is also there. But in this, we represent a group of data in a box. So it will be tropical, but it will be a box while you're liking bar chart. We have bars of the data and group a beta. Here. We'll have the lab, the box of the Grubhub data and boxplots are measure of how well distributed in the data, in the dataset. So it, it will be like a major four. The thing that how will the data is distributed in that dataset, right? So we'll see when we draw the boxplot. Let me tell you one more thing. Boxplots are measure of how well distributed if the data in the dataset, it divides the data into three quartiles. So what are these quartiles? As you can see in this picture, that there are three quartiles. One is quartile one and quartile two, and this is quartile three. And this is the inter-quartile range. This graph to represent the minimum, maximum median, first quartile, and third quartiles. So this part C, this is the box SAP data, okay? And these winds are outliers this point, and this point is they're are class, this is the minimum outlier. And this is the maximum or glare, right? And in between this box and the blue line here is the median. This is the median of the whole dataset. So this is the median, median value of the whole dataset. And these are the old class minimum value and the maximum value. And this blue line is the median. And this is the maximum data that is near to the median. So this, these are other useful data actually. So this is minimum, this is maximum. And this is the, this range from year to year, this interquartile data, and this one is known as Q1. Q1 quartile, first quartile, and this is known as third quartile. The first quartile, and the third quartile and first quartile, me grantee, percentile, and third quartile, or 75th quartile. And in R, we use boxplot function to draw or to draw our box plot. And in this way provide the like augments like X data, not what with names and Maine. So I'll tell you what are these things? So X is a vector or formula. So here x is a vector of formulas. So we'll see this is basically we keep Bu, things to release that are relational formula on which the box plot is going to be drawn. And then the data, obviously the data from which we will draw this relationship, edX, write the formula x. So this is the formula or relationship on which we are going to draw the data and this draw the graph. And this is the data, actual data from which we will get this formula or a vector. And notch is a logical value. Set as true to draw naught will see what is not in a bit. First, let me tell you the meaning of this. Augments. What it is. Logical values set S2 to draw width of the box proportionate to the sample size. So water with, if it is set through this box size will be proportion to the whole dataset, size of the whole dataset, okay? So if it is not true, it will be not proportional to the whole dataset. Okay? Next is name. Names is the group of labels which will be printed under each boxplot. So this is one boxplot. And for this, if you want to give some name, you can give that through the names argument. And Maine obviously is the name of the Graph. Okay, so title of the graph we can give with the mean. Now comes the what is not. The notches on the sides of the box plot can be interpreted as a company then interval around the median value. And the height of the Nazis dumb median plus or -1.7 into IQR divided by square root of n, where IQR is the inter-quartile range. We have seen what is inter-quartile range. So this is the inter-quartile range between the 25th and 75th percentile. This is the inter-quartile range. So basically, this is the, that is the n value, okay? Where IQR is the inter-quartile range defined by the 25th and 75th percentile. And Yan is the number of the data points in the dataset. Total number of data points in the dataset is N. And you can see here this is the box plot and this is the outlier, maximum outlet. This is the minimum outlier, and this is the median value, right? And this is the 25th percentile or Q1, and this is the 75th percentile, that is Q3. And this value, this thing, the ninth and the not, this is known as the not this value from here to here is known as notch. Notch is the 795 confidence interval of the median. Okay? So from median, this is the median and this, and this value will be known as notch. Okay? So when you put not equal to true, you can see this curve on this thing. If you put notch not equal to two, you will be seeing a straight line here. This not, will not be there. Okay? So this way, see here in this notch is false. So you will see like this boxplot, and if you put not equal to true, you will be getting this notch. Well, it means that the most of the data here will be, will be near to the median and it will be very confident interval of the data like median value is this and this plus and minus of them medium like here. The concentrated data points will be near to the median and it will represent that true data, right? So now we have the basic understanding of what is boxplot, what is quiet title, what is outliers? And what is the minimum quartile and what is maximum quartiles? What is interquartile range? And what is Q1? 25th percentile, and Q3 is the 75th quartile, okay? And this is the median value. So we will see in the next lecture how we can draw a boxplot based on the data that we have in our Estelle Blake, we're going to use mt cars, which is a real dataset that is available inside the car package or the ADA inside the art dish to reset itself. And we'll use that empty cars dataset to draw the box plot based on the MPT and the number of cylinders. So I'll see you inside the next lecture. 31. Boxlot using mtcars dataset: Hello and welcome back. In this lecture we are going to draw a box plot, okay? And we're going to use empty car dataset that is already available in our distribution. So we know need to download this dataset. It is already inbuilt into the our environment or our distribution. So we can directly use that and tried to present that empty cars data into a boxplot. Okay, so let's see how we can do that. So first, let me show you what is there in that box. What is that in the empty cars. So first let me pay to access the I'll create one data input and empty cars that is already available inside the inside though. Odd and one meant. And what I'll do, I'll try to run this and see what is there. Let me print it. Here. The empty car, our dataset and disease having like a laptop guards that Martha export dash 1017 ordinate for Dr. Riley and desktop research team must cities and all these cards, details are there like miles per gallon. What is the myelin? Each cars miles per gallon. And then the number of cylinders, the engine engines will have the number of cylinders, like got 2468, number of cylinders in the engine. And then displacement, HP. What is the hotspot and drag weight of the cars? All those parameters are given here in this data that is empty cars. So what I'm going to use, I'm not going to use the entire dataset. I'm going to use C, D, and E. I get one Lee miles per gallon and number of cylinders. Okay. So for that, what I'll do. Okay, So let's use these 2 mi per gallon n number of cylinder. Okay? And what I'll do, I'll print and then I'll huge head. And then I pass this data inputs so that we can see what comes. Let me run these two lines and see. Now we're getting the cars and their mileage, miles per gallon and number of cylinder into that. They're in jail. So these two details I'm getting with this, okay, so now I'm going to use this data input, which will carry the miles per gallon and number up cylinder. And I'll try to draw our box plot. Okay, so first thing what I'll do, I'll create a PNG file to store the graph on a file and had huge file equal to Allen gave some name like empty cars, box plot dot PNG. Okay. So I'll give the file name as empty cars box plot dot PNG. And then now what I'll do, I'll try to draw the boxplot and for diet and lose boxplot. And here what I'll do, I'll give two MPG miles per gallon with the number of cylinders. Okay. So I'm going to draw I'm going to Create a box plot between these mpg and number of cylinders. Okay? And then what I'll give, I'll give data according to this data input, our empty cars. So I'll keep data equal to empty cars. Right? Then. What we need to keep Next, we need to get the xlab. What will it be done? What do we want to write? Will write X Lab number of cylinders. And why lab will be white? Lab will be mpg miles per gallon. Okay. Then the next thing, what we'll keep, we'll keep main equal to my little data. Data. Harder will give cards myelinated. Okay. Then we'll save the file. Okay, dev.off. Okay. So what we are doing here we are, we are creating a box plot, mpg and number epsilon n to the car, and we are using the dataset empty cars and x-axis will be numbered up cylinder and y-axis will be miles per gallon. And the name of the graph will be cars, mileage data. Okay? And let me run this. Okay. Okay. So this is our plot, the box plot that we have drawn. It will come here. Now the heading is the name of the chart is car's mileage data. And this is number of slender 468, and this is miles per gallon. And the data box plot. Okay? So this way we can draw though, we can make box plots. If you want to understand one more time, I'll tell you. What we have done is simple. We have, we are using empty cars data, which is already available in our, our distribution, so we need not to create it or download it. It's already inbuilt into the environment. So we are using this, they take or two empty cars means we're going to use this empty cars dataset, which is having all this information about the cars, different cars. And then what I'm doing, I'm using a boxplot and I'm drawing the box plot between this number up cylinder and mileage per gallon for each cars. Okay? And I'm using data and pre-comps. And for x, x is, I'm using number epsilon there and y-axis mpg, Okay? And then the name of the graph I'm giving my live data and then I'm writing that file due to some suits in my audits to do. I'm not able to see the plots here. And that's why what I have done. I have written it to the file empty car, boxplot dot PNG. And we can see the file here. So this is the boxplot and this is the median. This is the median, right? And this is the minimum outlet. And that is the maximum or minimum. And it's Adam. No median value for the number of sellers. So four-cylinder engine. My lenses here and median mileage is something approx 26, 27 mi per gallon or number of cylinders six. It is coming around 20, 20 mi per gallon and number of cylinder, if it is eight, the myelin around 15 gallons per 15 mi per gallon, right? So this way we can draw a boxplot from the empty cars. See you inside the next lecture. 32. Boxplot with notch: Hello and welcome back. So in the previous lecture we have seen how we can draw our box plot using empty cars dataset. And based on this number up cylinder and mileage per gallon. So based on these two parameters are non dish two features of this dataset. We have drawn the boxplot and we have seen how the boxplot looks. So cars, mileage data here, mpg and number up cylinder 46 8 mi per gallon. Yeah, 15 2025. And this black line is the median of each group. Okay? So four cylinder cars, six cylinder cars and a cylinder cars. Now, we can actually draw the same box plot with not. So we have seen what is not. Now. We'll see how not. If we put notch called a true how this boxplot will change and NADH is used to draw, like it will tell you how the median of each group is related to each other, right? How medians are different groups match with each other. Okay, so let's do that. And also we'll see, we'll also try to put some colors into this graph boxplot so that it will look good, okay? And we'll also try to name this x axis. Okay, so let's do that. So first thing first, what I'll do, I will change this name to boxplot width. Not. Okay. And then what I'll do, I'll simply put not equal to, sorry, not equal to true. And now, let me run this. Now let me go here. See here, we have, so our graph was like this. Now, when I put not equal to true, our graph has changed to this. And see, you know, this is the median of each of the graph, right? Each of the data set on each of the group, like four-cylinder, six cylinder. And how these medians are related are different from each other, how they match with each other. See the median though. How would these things are coming, the Nazis coming for each of these groups. Now, let me put some color into debts. So what I'll do, I'll huge. And then I put rector and hear what algorithm you give. Red, green, yellow. Now I'm giving three colors. Red, green, yellow, yellow. And let me run this. Let me put some name Maria Callas hockey. So that will have the different, different graphs, images. Can now we have different colors for different group or box. And earlier it was like this, without color and without notch width, notch, notch width colors. Okay. Now, let me give some names here. Okay. So here, the mileages. High rate for four cylinder cars, mileage is high for six cylinder, medium, and for the cylinder, this law will give the names high, medium, and low. So that when we see now boxplot, we understood, we will understand that this is the high average car and this is medium and this is low mileage gas. Okay? So that's cool. Names hotel give high, medium, low. Okay. Let me run this. So see you and now instead of 46.8, we have high, medium, and low. So this way we can, right? And if you want to more specific, you can put like porcelain that high, something like that so that it will be more clear. Excellent, medium, low, like this. Okay. Okay. So this way also you can put it. So I hope you got to understand how you can put boxplots. So you also try and see you in the next lecture. 33. Histogram and distribution of Histogram: Hello and welcome back. In this lecture we are going to learn about histograms. So we'll see what is our histogram. And we will also see the types of histograms and how we can draw on how we can put histogram based on our data in our tactical season. Okay, So here we'll see the theory part and we'll learn what is histogram. So let me tell you. A histogram isn't appropriate presentation of the distribution of numerical data. So it is basically a graph like bar chart or bar chart that we have seen. It just seemed kind of distribution represent this distribution of numerical data. So basically, if you have numerical data, you can draw out a chart or a graph or Brad and histogram is a very appropriate representation of the distribution of numerical data. So it is basically used what? Numerical data, and that numerical data should be continuous in nature most of the times, it was first introduced by Karl Pearson. So Karl Pearson has introduced the histogram. Another way to say histogram, like another definition of histogram is a graphical display of Brita using parts of different heights. In a bar chart, we have seen the bus died in the same way. Histogram is also a graphical display of data using bar sub different heights. It is similar to a bar chart or histogram groups numbers into ranges. So if you see bar chart, it is not grouped the data into the range like ten to 2020 to 30. It will not arrange. It will not group the data, but in the histogram, it will group the data into the ranges. And then it will put the bar chart, bar chart with numbers into the ranges. Grouping up numbers into the ranges will give you a histogram. So I hope the picture is getting clear. We'll see the images as well. We'll see the real representation of how bar chart is different from the histogram. So here it is good to note that bar chart plus grouping off numerical data into ranges, like you have data 10-100. So it will create the bars. And with that, it will also arrange the data 10-2020 to 30 in the groups. And it will create paths. So it will be a histogram, the height of each bar sold, how many fall into that range. So it will basically give you an idea that 10-20, how many numbers are there? How many support? If you are looking at the salary of the people. So it will tell you like ten to 20, how many people are there? How many people are there like that? Okay. Creating a historic sunny, creating a histogram provides a visual representation of data distribution. Histogram can display large amount of data and the frequency of data values. So like what it does, it will group the data into the ranges. So it will give you the frequency like ten to 20. How many? So it will give you the frequency as well, support a particular data value. It will tell you how many times it is occurring, the frequency of the data values, the median, and the distribution of data can be determined by your histogram. So median and the distribution of data can also be determined by histogram. In addition, it can solve any outliers are gaps in the data supports. We have the data 10-140 to 50. We don't have any values. So it will tell you that 40 to 50, it will not show you the bar. So with the graphical representation, you can find it, okay, 40 to 50, we don't have any employees. So it will show you the gap in the data, and it will also show you the outlier. Suppose you have like ten to 100 and then you have another bar coming from 17180. So all the data blocks will tend to 100 and another graph is far away. It is showing 171-81-7180 ranges and an outlier that easily we can identify by looking at the histogram. So histogram, or great way to solve this jolts up continuous, continuous data that I told you earlier, such as height and weight if you want to. So the histogram is best suited for that kind of radar. So here I have. So I'm just showing you how you can differentiate the bar chart and histograms. So see you in the histogram there is no gap. It's a very continuous, like 468 up to 24. Bar start coming right in. Here you can see the difference between histograms and bar charts. So here you can see the histogram, it very continuous like that. Ignore gaps between the bars right here, the bar chart you can see it is showing you the different like January, February, March, and there is a gap between the bars. So this is the main different than like in a histogram. The bars will be there will be no gap between the bars and the bar chart will have some gaps between the bars. Okay? So that is one pictorial difference that you can find between the histograms and bar charts. Now come the distribution histogram or how many types of histograms are there? Normal distribution, you can see like this. So in a normal distribution, points on one side of the average are likely to occur as on the other side of the evidence to see you data this side, left side, and on the right side are almost equal, right? So that is the normal distribution. And if we go to the bimodal distribution, it will throw you in a bimodal distribution. There are two peaks, C-O, there is one peak, and there is another peak. There are two peaks in the data. So this will be bimodal in a bimodal distribution data. So we separated and analyzed as separate normal distribution. So this is one normal distribution and this will be another normal distribution. And when two normal distributions come together, it will create bimodal distribution. This is the normal distribution and this is bimodal distribution. The third type of distribution or histogram is a right skewed distribution. What is right skewed distribution or histogram, or right skewed distribution is also called a positively skewed distribution. Why it is called positively skewed distribution because see the skewed values are coming on the right side, zero to infinity, right? So these are the positive values. So when it is right-side, the push two values are skewed. It will, we call a right skewed distribution. In a right skewed distribution, a large number of data values occur on the left side, with a fewer number of number of data values on the right side. See here, on the left side, more data values are there, and on the right side, the number is decreasing, okay? Alright, it's skewed distribution evenly across when data has a range boundary on the left-hand side of the histogram, e.g. boundary of G. And the next one is left skewed distribution. Here. Left skewed distribution is also called negatively. Why negatively? Because on the negative side it is getting skewed. In a left skewed distribution are large. Number of data values occur on the right side. So the number is increasing from left to right. So when we are moving right, the number is increasing right? And fewer number of data values under left side or left side is lesser value than right-side. More values is a left-skewed distribution. And a right skewed distribution usually occurs when the data has a range boundary. On the right hand side of this diagram, e.g. boundary, such as hundreds. Okay? So these are the four types of histograms. One is left skewed distribution, then we have seen right skewed distribution, then the bimodal distribution, and then the first one is normal distribution. Histogram. Now, are you just hist function at JIST hist function to create histogram. And it will take few parameters or arguments to draw a histogram. Okay, so eax is, what is X? X is a vector of values for which histogram is a chart. So this x is a vector for which we want to draw the histogram, the main, xlab and ylab. Main. Main. Main is the title for the histogram, and xlab is the axis labels, okay. Like frequency or any other thing, if you want to port. And then x, lima and wildly are the ranges of X and Y values. Okay? And then breaks Brexit one-off vector giving the break points between the histogram cells are function of computer vector breakpoints in a single number giving the sales of a histogram. Okay, we'll see what it breaks in a detail. Then we have the coal that is color and then we have the border that you know. So in the next lecture we'll see the example for how we can use the hist function to draw our histogram. So we'll draw a histogram using hash function in that next lecture. 34. Drawing Histogram using hist function: Hello and welcome back. So in this lecture we are going to write our first program for histogram. So what we'll do, we'll create and vector that will contain our data. And then with that vector, we are going to make a histogram. We are going to plot that data has a histogram. So let's get started with this. So what I have done, I have already created a dot R file that is histogram dot R. And I have written the code. So I'll show you what is the course. So I have written the code already so that we can save time in writing, okay, so forth. What I'm doing, I'm creating vector data for the graph. So I'm creating the data here. So I'm creating, I'm creating a vector and I'm assigning that vector to x. So x is n vector that is containing data, lake for printouts and 17,000, 3,006 plus external support. The salary few implies I'm storing in this vector x. So this vector X is our data and that is containing support, the salary of employees. Okay? So this is the salary of the implies propane Tao Qian, like that. Okay, Then at some ranges like preppy to 2050 to 60, 20 to 30, 40 P2P like that. Okay, so now what next thing, what this is going to be the very simple histogram example. Okay, so now we have the data. I want to plot a histogram. So what I'll do, I'll create foster histogram image file. So I'll use PNG function and we'll give file equal to histogram dot PNG. And then what I'll do, I'll use the hist function that I have told you in the previous lecture. So here I'm creating the histogram, okay? So I'll use hist function and I'll pass this x. What is x? X is the data for which we are going to plot the histogram. Okay, so these values will go here with X. So X is a data vector, vector data. Then what I'm giving xlab equals to salary and then color I'm giving as a green and the border I'm giving yellow. So I'm not giving too many parameters yet. I'm not passing too many parameters. Yes. Only xlab that I'll give salary, then up color, I'll give green and the border, I'll give Slovakia. And then I'll save this graph image file to our system that is depth dot off. Okay, so before we move, let me set our working directory. So to do that, what I'll do, and yet what I'll do, I'll save that. Go into this directory. And next thing what I'll do, I'll go to more. And yet what I'll do, I'll set working directory. Okay, so now I'll save this file. So I hope the steps are pretty simple. I'm creating a vector X with the data. Then I'm giving using PNG function to create an image file for all of our graph so that we can see and we can utilize. And then I'm storing that on our local file by Deborah off. Okay. And I'm using hist function to kidney does histogram with the data and passing x vector here and then the x lab, I'm giving the name salary and color green and bartered will be yellow color. So now I've saved, now I done thus this whole source file. Click on Source, and this is done. Next thing, I need to go to the D Drive, go to our 20 and see here our firewall with the histone. Let me check the file name. The final image, this top graph dot PNG. And if he's too, I think this is the file name histogram graph PNG. Okay, So this is the way that I have my delete all others that I've created earlier. So now we have the histogram of x. And here what we're creating, I'm giving the, let me remove this, delete this. Let me run the program again. Let me change this to red. And border will be yellow hockey. Let me run this again. Let me see whether the file in general to see your histogram. This is salary and here frequency. And then here are the ranges like ten to 2020 to 30,030 to 40,040, 2050 to 60,000. So now you can see how histogram is being dark. So ten to 20,000, how many implies are trying salary 10-2123. Let's verify this with the data. Okay. So let's go to here, go here. 22 tend to 2010 to 23, so ten to 2012. And then we have another, you have one lead two. White is doing. Okay. And then we have $20 or 2,000,017,000.15 thousand. So these are the three implies drawing sadly 10-20. So it is showing tend to 2310 to 23 employees. So this way it will divide the data into the ranges and make them our groups or ten to 23 employees, then 20 to 32 implies less 25 data as well. 20 to 31. They started to see 20 to 30. We have two employees. Okay. 25,020, 2000. That is also true. Totally. Then 30 to 42. Again, 30 to 42, 34, t1, and t2 one. Okay. 30 to 40 also to next is 40 to 50, only one employees there. So let's see for T2, T4, T5. And see this is more than perfectly okay. So 40 to 50 only one implies during next is last one, T2s t We have four implies. Let's see that more than 5012, then three, and then four. So we have four, right? See, if you have to trust the city, we have four implies. So this way it will group the data and it will tell the frequency. Like if you see this, you can easily tell that cryptosystem more than 50,000 salary for employees or drawing ten to 23 employees, 20 to 32 implies and 40 to 50, only one in price there who is getting salary for P2P. So this way histogram will be drawn and it will group the data based on the frequency. Okay? It will define a range or Beta, and then it will tell you the frequency of occurrence of the data into that range. So ten to 23 occurs, 20 to 32 workers, 30 to 42 workers, 40 to 52 occurrence and then put Petri dish t for agonist. So it will, what it will do, it will group the data into few ranges and then it will tell you the occurrence of data into that trend. So ten to 20, there are three data, 310 to 20 and it is Jacqueline three times. Okay? So this way, so this way we can say three employees are drying, sadly 10-20. So this is the significance of Histogram. Ok. Next is what I have done. I have taken our different data here. I'm what I'm using, 114-567-8910. Okay. So then the same thing, I'm giving a different file embellished program graph, one dot PNG. And here what I'm using, x sin x, I'm passing year. Only the data points are different. Data values are different here. Okay. And pretty simple later, 114-567-8910. Okay. And then I'm saving it to okay. Let's run this and see what we are getting. So let me find histogram graph one here. Now. It is defining the data. I like zero to two. How it is defining the range is zero to 22 to 44 to 64,628.8 to ten. So g rho 22, how many occurrences? Two occurrences that so G row two to 11, these two ones are less than two, right? So zero to two to correct. Next thing is to two for one. So two to four. Only four Is there one. Then four to six to four to 65.6 to write than six to 8782 again, and then 9108 to ten to 9.10. So this way it is dividing the data into two ranges, zero to two, or two to 44 to 66 to 8.8 to ten. And then it is giving us the frequency of occurrence of data 0-22 twice two to four, once four to six. Repeating price, okay? To occurrence of values 4-6, right? Yeah, 5.6. So this way also, you can change the color from here. You can make it black. And know the graph will be in the black. So this way we can change the color up the histogram. I'll see you in the next lecture. We'll be seeing some other examples of histogram. 35. Using breaks xlim ylim in histogram: Hello and welcome back. So in the previous lecture, we have seen how to draw a histogram. We have seen two examples, two different data. So we have taken one vector. And then in the second example, we have taken the very simple data. And now what we're going to do, we're going to learn how to use break, how to use limit, y, limit and break bonds, these three parameters to the hist function to draw a histogram with break and x limit. Okay, So I have written the code. So first thing, I'll use the same vector where we have the subtly of your blinds reading from pre-print origin to 60,000. So I'm creating an x vector that we have done in the previous lecture. And then I'm creating a destination file where I store this histogram. I'm using p and g function and I'm giving finally Mises to Brex dot PNG To with Brex dot PNG. This will be the final name that will be created in the name of the file that will be created on which our plot will be, our histogram will be on drawn, okay, and then here with the hist function, I'm creating the histogram. So I'm passing the x vector here at the data. And then I'm giving xlab name adds weight and color. I'm giving blue, hand the border. I'm giving that green. And then x limit I'm giving. Here, I'm using an argument that is excellent. The x limits. For the x-axis, I'm giving the limit zero to 40,000. And see our data is containing the values 15-60 tolerance are covered. Data is going beyond for t, But here I'm looping, limiting that x-axis values to zero to 40,000. Okay? And why limb is GTO to ten. Okay, so good, why y-axis will be the frequency of occurrence of the data like zero to ten or zero to 20, whatever we create. Our opening implies a drawing salary 0-2020 to 30 or 30 to 40, like that. Okay. So that I'm lifting toe. So and then I'm using another parameter here, breaks equal to two. So let me first make it one. Okay? So what I'm doing, I'm giving Brexit equal to one. We'll see what impact this Brexit equal to one is putting on our histogram. And then I'm saving the file. The histogram that will be generated will saving on this his toe with Brex dot PNG file to save that on our local, seeing them using db dot off. With this, it will be saved on our machine. So now this is the code, simple one-line code that we're passing some parameters with bricks. Xlim. Xlim, xlim is limiting the data values on x-axis, y limits for the y-axis and break will understand when I run this code. So you will be understanding better when you see the output. And with that visualization, we will understand what Brexit actually doing. Okay? So let me run this source hold Source, click on the source and this will be so let me open the output files. So here is two with Brexit file has been created. Let it open. Now the file, our historic numbers. So you'll see here, since I've given breaks equal to one, we are seeing only one big bar, right? Zero to 40, and it is showing 0248 employers are trying salary between it to 40. Let me verify that. See here, we have total 10.12 3456, 1234, 5670, 2408. But we have one list, 702, 40, I guess 123,456.7. All other are more than 40, right? 1234. Okay, so now we can see this graph. Let me change the bricks to two. So now you will see there will be two partition. Okay? So let me, when the file again, see here now we have zero to 40. 40 salary has been divided into two parts, zero to 20.22, 40, and then 42 status showing separately, right? So now the whole dataset has been divided into three, but zero to 40 since here. Since here we have begun when zero to 40. So zero to 40 data, zero to 40 data has been divided into two parts. Cheeto to 20.0 to 60, c, zero to 20.0 to 22 400,220.02, 400,223.22, 44 data and 40 to 65. Okay. So let me show you the difference. If I put t zero to t here, what will happen? Zero to 60 data will be divided into two parts. Let me So you open the file again. Now, G zero to 60 has been divided into three parts because we don't have data beyond safety. It is not showing the other one but zero to 60 web debt. We have data. So Quito toxicity data has been divided into three parts. So this stage, what breaks does so x lim zero to 60. If I make it 50, what will happen? Zero to 50 data will be divided into two parts. Okay, let me run this. And so you saw that you can, let me click on Source and let me go to the file and open it again. Now, J22 few pre-data has been divided into three parts. Okay. Got it. Zero to 2020 to 40, then 40 to 60 comes together, right? So it is showing up to 50. Okay. If I make it three, what will happen? Let's see. Let me open it again. See you cater to 2020 to 40.40 to 60. Let me Let's keep on experimenting. If I make it four, what will happen? See, now it is showing ten to 2020 to 30, 30 to 40, 40 to 50, and then 50 to 60 will be separate. So 12340250 data has been divided into four, right? That's what we have asked for R2P, so we divide it into four. If I make it zero to suppose 32, 30,000 data will be divided into four parts. So let me open the file again. Ten to 20.20 to 30, because we do not have that many data, right? So ten to 2020 to 30 and then 30 and beyond has been divided. Because we if I make it too, Let's see. 30 data has been divided into two parts. The first is 0220 and then 20 to 40 and beyond, right? So let me get my portrait like G naught to T and let it make for and see. The output. Gate A2 safety has been divided into 1234 parts. Let me open the file again to see the object file C and G, zero to 6110 to 2020 to 30, 30 to 40, 40 to 50 to 60. Okay, So this is what we do with the breaks. Okay? So this is what x Lim and wildly, wildly moles. So you can Our CEO, while image doing it is restricting the y-axis frequency to zero to ten. So suppose if I change that, if I change this to support your only six, what will happen? Let me run this and open that current file. See here now we are seeing frequency zero to six. Okay? So this way we can restrict x-axis and y-axis where the xylem and widely supported. If I make it For what will happen. It's good to experiment with the code to see the output. And when you see the output, you will know the actual use of the parameter C. You get a two for, one little for it is okay. If I make it supports, I'll make it zero to two. What will happen? Now I'm changing the y-axis, the frequency. I'll see you see, you know, it is going like this one. Allele frequency occurrence of total to it is going right. So this way, we can play around with the code in any programming language, bit R or Python or anything. If you want to learn, you have to start playing with the code and start playing with the parameters. And if you change the parameter, you will see the exact impact of that parameter. And in that sense, you will learn better and you'll get the implications, exact impact of that particular argument, particular parameter in a particular function. Like in his function, we know what is x, what is x lab, what is color, what is border? What is x Lim. We have seen how we xlim is impacting the histogram, how wild limb parameter is impacting the histogram and how bricks is impacting the, impacting the histogram. So when you experiment, when you play around with the code and their data, you will get better insights and better understanding of the code. So I hope you got to understand what is X-linked, Y-linked, and breaks and how they impact our histogram. So with that, let's see you inside the next lecture. 36. Basic line chart for time series with ggplot2: Hello and welcome back. So in the previous lecture we have seen how to draw a histogram. Now we are moving to some very interesting graph. You can see our basic line chart. And this is very important because this is going to be important when you learn more about time-series or problems in your data science journey. And this is, in this lecture we are going to like see how we can draw a simple line chart for our time series using GG plot, which is a package in R. So what is time since it? First, we need to understand that. So let me take you to the basic, very basic definitions from Wikipedia about time series. A time series is a series of data points. So basically your CD job data points indexed are listed or grabbed. So it can be indexing of the series of data points, or it can be lift up series of data points, or it can be a graphical representations series of data points in time order. So it can be anything but our time order. So a series of data points indexed in time order. That means the data points are based on the time, so it should be built under time. So when we have data which are based on, which are based on the time for a particular time period or particular time in Teach based on the time then that series a Brita. We can, when we plot, we get a time series plot. So most common Lee, or time series and sequence taken at a successive, equally spaced point in time. What does it mean? Means are time series is a sequence that we have locked right? Time series or sequence of data points indexed in time order. And most commonly at time series is a sequence taken at successive, equally spaced appointments on a time. Suppose on our timeline, we are going to take successive, equally spaced points in times of all 0220 4 h, then we will take zero to 3 h than three to 6 h 32, 9 h nine to 12. Early success, two equally spaced point like three, 3 h interval we're taking. So that will be a data 0.326, 6299 to 12, like that. Okay. So I'm giving an example. It's not the exact thing. But you can understand like what timeline, zero to 24 h and few things that are happening each, on each second, each minute, right? So when we document those data points, like on a four-star something, this is the data on the second hour. This is the data on the tada. This is the data. So when we define that data successively equally spaced point in time, it will give you the time series, sequence of time series. Thus it is a sequence of discrete time data. It's not continuous, it's a discrete in nature because we are taking a equally spaced time points from that data. So we are making a discrete. So this is the definition. Now what we will do, we will try to understand this by plotting it. When we plot, we will understand more about it. Okay, So before we plot, we asked to install the library GG plot two, so you can go to the package and install that GG plot two. Okay? So for this we need Library GG plot two and the player, okay? So these two libraries is required. And then what we do, we'll create a dummy data. So what I'm going to do, I'm going to create a DataFrame that will contain day. They will be agitate as dot. That means it will take data in this format like first-gen total 15. And then what I'll do, I want from here to zero too. Research to 4 min, 365 days. So from here to 365 days, up each day like first-gen then 31st December 2014 like that, I'll take three to five days. And for particular date on what I'm taking the value, I'm using the runif function. You know, what is run a function running fox and it will create what, what it will do. It will create a sequence of numbers straight, 4365. So it will create a sequence of numbers. So what are running function will do? It generates the random deviates of uniform distribution and is written as runif. So what it will do, it will gender easily generate the n number of random numbers it will render. So what we're doing here for value, we're generating initiative research to wipe random numbers for each day. So we'll assign the first random generator number to date, one, like this one. And then the second randomly generated number will assign to the second date. Okay? So we're not just, we're only using the randomly generated numbers from this standard function. Okay? But we're also adding into that by, we are also genetic sequence from minus 142 to 40. And that sequence we are what we're doing. We're taking a square root of that and then we are dividing with that 10,000. Okay? So basically we want the output from this. We will add into what we get from this runif function. Okay? So runif will give the random deviates and uniform distribution. Okay? So what we get from this to date, day and value that we will store in this DataFrame, we are creating a DataFrame which will, which will contain two values, t and particular value from these two functions that we'll get, we'll see how we have, what are the values we are getting, okay? And then what we'll do, I'll just print the data to show you what data we are getting. And then with this DataFrame data, what I'll do, I'll use the GG plot to plot this data. So I'll pass this data frame. And then what this will do, I'll provide x axis as a t and y axis as a value. Plus. I'll use the geom line, function two and then xlab values, okay? And then I'll print the print, I'll print the product. Okay. Next thing, what I'll do, what this geom line function will do here. You can see your Zoom nine fonts on what it will do. It will connect the points in order of the variable on the x-axis and G. And it will actually, it will create dust like stair step plot. Ok, so basically what it will do, it will join the points and it will create the graph. Okay? So here we will be using few symbols like percentage, d is d as a number zero to 31. Percentage will be created as a weekday percentage capitally is like an abbreviated weekday, like Monday. And this will be turmoil and percentage jammies. Month zero to 12 and presented b and capital B will be abbreviated and an abbreviated month like Jan will be abbreviated and full January will be non aggravated. And then percentage Y and percentage capital Y will be Tool, percentage vitamins two digit yet, and percentages capital Y will be four digit yet. Okay? So this is the basic abbreviation that we are going to use. So let's go to the RStudio and try to run this code. So we are using GG plot and the player library here. And then what I'm doing here, I'm using data. I'm creating a DataFrame here, dummy data with like acetate. And then we are creating random value using this runif and sequence. And then I'm printing the, whatever we are getting from the runif that I'll show you what we get from the runif function and what we get as a data. Okay? So let me run this code here. Okay? So let me run this tail here. So now I'll show you what we are getting. Okay? So see here, for the runif function, we are getting this value, okay, 0.3, 44.4. Like that, it will generate 365. Winds. Okay, and then for the data we're getting, d, like January 1st, John, total and far between and we are getting this value. Okay. And then for the 31st, December 3rd, and 4th protein we're getting, so we're getting this data and value in our data frame. And now what I'll do, I'll pass the data frame to the GG plot data and then exit call today Y equal to value. And I'll use the geom line to plot the data and then what will be there or what I'm using here. I'm using the average deviation scale, underscore, underscore date. And here I am giving the debt levels that x v. Okay? So let me run the whole code. Click on the source and see the plot here. So let me print the floor. See here. Now we are getting a time-series data yet Jan 2014, 2014, July 2014, October 2014, and Jan 2015. Okay. And if I run this, we will get we will get abbreviated January, July to what n. Then it is going like this. If I use capital Y, b, small b, and d, what we get, we'll get the full year like 2014, then Jan one tours on 14 apparel one like this. Okay. And this is the week, right? So it will show you the weeks, the late. 13th, week, 26th, week 3093, and so on. Okay. If a huge mountain, why it will give us the month, why? Month and YC. So dismal January totaling 14 approval to tunnel protein like that. So this is the here we're seeing the month on the x-axis. Here we're seeing the mountain together. We are seeing the particular value at that point in time. Okay? So this is the way we can draw a simple time series. Here. We can modify this a little bit. And here I'm using SLB, our teams library in that you can, if you do not have, you can go to the Packages, click on Package, click on Install and you can install it, and then you can use it. So I'm using the same data that we have created in the previous. Just now. And here what I'm doing, everything, I'm using theme in the geom line. I'm providing the color as a green. Hesiod is the black line. For now, I'm changing that to green. So here geom line, I'll provide color, the color to green. So it will be that graph will be in the green color. And then theme underscore Ipsum, I'm using blast for the team. What I'm given x dx t dot x element dot TXT underscore t Actually angle. I'm giving 45 and just I'm giving one. So what this angle 45 will do, it will turn this graph at 45 degree. So let me run this whole code. See here. Now the graph is coming like this. And the 45 degrees, so it is converted at 45-degree. So this way we can use this angle equal to 45 before I suppose t. Let's see what difference we're getting. An artist slightly change. Let me change it to support 160. Can you see any difference if I put 45? And now I'll change it. Doe 145. Let's see if we are making any difference. Now it's coming like January 2020, October 2019. Like that. Graph is changing rate. If I do it 45. It is coming like this. And when I am doing it like suppose 90 degree, it will be coming in on different way. It is. This representation is changing, right? January. This, this thing is changing, right? If I make it ten degree. And if I plot and see here how this whatever we are writing here, January 2019, that angle is getting changed, right? If I make it, suppose that he sees TDD, let's see how this gendering no change. Let me make it one entity. When you play with the data, keep on. Can you give to understand what actually are different? You are getting it right. The ear. It's like middle, late January 2019. Now it is showing as a mirrored image of January 2019. So this is the difference. Okay. So this way we can in that level, okay? So x-axis level orientation, weekend chains like this, okay? If you put 90 degree, it will come at an IT degree. So I hope now you are clear what different this angle is making right? Now it is at the 90 degree. So this way we can use the element texts and we can change the orientation of the texts like January, like this. Okay? So this way we can do that. 37. Scatter Plot and plot matrices in R: Hello and welcome back. In this lecture we are going to learn about scatterplot. So what is scatterplot? A scatterplot is a type of plot. Our mathematical diagram using Cartesian coordinates to display values what typically two variables setup better? So it means it's the simple. If you take our x, y plane, we need to put points on the plane, x and y, x comma y point we need to plot so we will not draw lines are safe or anything, but only the points where we'll port so support the Majlis and the number of cylinders, so forth. Cylinder. The number epsilon on the x-axis and my legend the y-axis. So it will be like four comma. The number of cylinders is four and the myelin is tripling still four. Come up your print one point on the Cartesian plane we will put. So it's a pretty simple one. If the points are coded, one, it is still valuable, can be displayed. Okay, so what I'm doing here, I'm going to draw one dataset that is empty cars, that is the input data. Set. Our level with that we have already used in our earlier examples. So I'm going to use empty cars dataset that is readily available with our, our environment. So what this empty cars, WHO Lab it will have. It will have to data regarding the cars like number epsilon, dark myelin, and weight of the car and other things. Okay, so from this dataset, what I'm doing, I'm going to pick the number up cylinder and myelin, and that means number of cylinders and the Majlis of the particular cost that I'm going to fetch from this dataset. So I'm using recta. See cylinder myelin. Myelin, that means cylinder myelin. And I'm storing that into this object. Okay? So if I run these two, what will I get here? So I'm getting details for various cars like Mazda, Datsun, and I'm getting number epsilon, that is six, formaldehyde export. And the mileage is one-to-one. In the similar way, Lotus Europa, number epsilon, that is food and myelin is 34 for Bandera. It cylinder, and the myelin is for pinpointed, while Woof 142 e number of cylinders is four and the myelin is 21. So this way we are getting The tales for many cars, various cars and their mileage. Okay. So now what I'll do, I will create a scatter plot showing the gods and then my list. So for that, what I'm using, I'm using a PNG function and I'm giving the scatterplot file name as my list plot dot PNG. And then I'm using the plot function. Simple plot function will be used to draw this scatterplot. And yet I'm providing that x equal to input. So x is basically the input vector. So you get x equal to what I'm providing. I'm providing input as a number of cylinder. X, x axis will be input cylinder. It means I'm getting the number of cylinders, okay? And y-axis will be important. Mpg input dollar mpg means I'm getting this mpg data, mpg. So that is my list TO x-axis will be cylinder number of cylinder and y-axis will be mileage, miles per gallon. Okay. And xlab I'm giving x-axis x-axis level. I'm given number up cylinder because I'm putting number of cylinders there. And for y-axis I'm giving name as myelin and x limit I'm giving four to eight because number of cylinders are reading 4-8 or ten. Okay? So I'm giving four to eight for x-axis limit. And the y-axis limit, I'm giving ten to 35. Okay? And the graph name or the scatterplot name, I'm giving number up cylinder, cylinder versus mindless. My laser dot cost, okay. And then I'm using kept dark or off to save the fight. So it is pretty clear here. Now, let's save this file and use this name as empty cars. Ca will cylinder minus okay. And then save it. Okay, now let's run this. So let's run this source file. So now it is successful later on. Now, see the output file is to be scattered, plot my list product. So see what name we have given. We are given mileage plot dot PNG. So this is the plot. Okay? So see ya know, the name of scatter plot team is number of cylinder for this myelin and x axis, I'm sewing number up cylinder that each 45678 and y-axis is myelin, 10, 152-025-3035. Okay? And here with each point, suppose this is the point. This point is for number epsilon, that is four, because x axis value is four and y axis value is somewhere around 21 or 22. So four cylinder and myelogenous Frontier 14 cylinder, another point, 4.23 or 2023, then 2044, comma 24, and then four comma 26, 27, like that for about 30143034 comma four. So number of cylinder for, and these are the mileages for the four cylinder cars. And see you soon as a point, x comma y point on this Cartesian plane in the two-dimensional x-y plane. And we are putting the points like number of cylinder four and my religious 21 point something, okay? In the same way, X axis six means number of cylinders, six point this number six. The cars which are having no cylinders as six, they had having this many myelin sheath. This myelin like a footprint, 16171516 to 2021 milestone. This, we can know that six cylinders guards start giving less mileage compared to the four cylinder cars and come to the air cylinder cars. They are my lenses for that loop. So the eight cylinders cars are giving the myelin the lowest. So the best myelin is giving Garza the number of cylinders, four and then six greens. With this scatterplot, we can come to the conclusion that the number of cylinders are deciding the mileage of the car, the less than number of cylinders in the car, more will be the myelin. That means if the number of cylinder is increasing, the mileage is decreasing. Seizure for the mileage is more six myelin, reducing. The number of cylinders is coming to eight. The myelin is further reducing and lowest among the 46.8 cylinders cars. So number a, myelin sheath. Inversely proportional to the number of cylinder, the cylinder, cylinder number of cylinders in the car engine is less, mileage will be more, and the number of cylinders in the car will be more, mileage will be less. Okay? So with this type of scatter plot, we can easily come to the conclusion by just looking at the graph, okay? And it is the, one of the simplest graph that you can see and it is pretty easy to analyze and get the details from the chart itself. Next thing what I'm doing, I will just give the file name as empty cars and discuss scatter plot or dusk or mattress. So what I'm going to do here, I'm going to use the pair function in R. And with this pair function we can maker like mattresses up the chart. And how we can do that. We can take the data points from the empty cars dataset. And at each rate. Mileage per gallon displacement cylinder. From this dataset, empty cars data, I'm getting data equal to m precast means we are using this empty cars dataset. And from this, we are using these four variables are four columns, like Wait, MPG, miles per gallon, displacement and number of cylinder. And F4 column. These four parameters will be used to make graphs like weight will be taken and the scatter plot will be drawn based on weight. And my list, weight and displacement, weight and number epsilon.in the same way, myelin will be taken and then the graphs will be don't like Mileage Plus weight minus, plus displacement, then Mileage Plus cylinder. So mileage versus cylinder, mileage versus displacement, mileage versus weight. So this way, one variable will be taken and The graph will be drawn with respect to the three variables. Here I'm giving empty cars scatterplot matrix as the name. So first let me commend the main one. And first I'll use, I'll start with the simple one. I'll just try to draw the MP. My list was this number Epsilon diagram that we have drawn here without pair. So what I do, I'll try to draw a dark beers only using MPG and cylinder. And let's see what will be the output. Okay? So let's run this. And the filename is this C here. See you. So the same output. But here what we're seeing, we're seeing in a matrix form. So here it is throwing MPG miles per gallon and swim number epsilon doc. So you can see here MPG miles per gallon and here number of cylinders. So you can the number epsilon that is for all the four cylinder cars are your 46.8. They are clustered here for respective mileages or heal the four cylinder myelin here, then six cylinder myelitis, and now it's cylinder my list. So if you combine these two will get this scatter plot. If we combine these two, we will get this one. Ceo, we aren't getting the earlier, we're getting this number of cylinder into my list. This graph we are getting, I'm going up cylinder on my list. Now we are seeing the y-axis dislike. This will be the mileage one. Okay. And the number of cylinders being Sonia and the myelin is being Sonya separately in a matrix form, right? The same thing but in a different representations. Ok. Now let me go to the code and let me comment this line and then uncomment this line. Now I'm using DUF for variables. And let me run this. And now see the graph how this coming. So open this. Now see the four variables, weight, MPG, displacement and cylinder. So then here the weight, the Majlis displacement and cylinder helping Sonya. All right. So this wastage thin. 123 456-789-1011, 12.4 cysteine has been drawn. Okay, let me plot actually. And with these four things, so four by four matrix husband, right? So this way we can use these beers to create the plot matrix for that dataset. So this will create the scatter plot matrix so we can use to create block matrices. Okay. See you inside the next lecture. 38. Finding mean in R: Hello and welcome back. In the coming next few lectures, we are going to learn about statistics in R. So we will see what are the functions that have inbuilt in our programming that we can use for statistical analysis in R. And that will be very helpful in our body. That will be very useful in machine learning, artificial intelligence, deep learning, and all those things. So statistics is the main thing if we want to get information from the data. And article analysis is the fundamental for the machine-learning algorithms. So we should be knowing what are the statistical analysis and the dorms. So in the coming few lectures, we're going to learn about these things, statistical analysis in our programming. So let's get started. There are many inbuilt functions through which we can do statistical analysis. And these functions are very useful tools and just unit to use the function name and you have to pass the data vector, took it with some arguments, and your work will be done. So it's pretty easy to do all the analysis in our programming. So now we are going to see what is mean and how we can calculate mean. Then we'll also see what is median. And then we'll see more. So we're basically going to learn in this lecture, mean, median and mode. Okay, so let's get started. So first thing, what is mean? Mean is calculated by taking the sum of all the values divided by number of values. So it's like average. So you know how to calculate average. I support you have numbers 12345 and you want to get the average of this. So you'll do one plus two plus three plus four plus five divided by number of values. So number of arrows, yellow, 5/5, so you'll get the average. So average is mean. Mean is also mean is equivalent to the average. So mean is sum of values in a vector, our data, some values in the data divided by the number of data. Okay, so let me show you how we can do that in R. And for calculating mean, there's a function in R called main function. Okay? So let me write a program for that. First, let me tell you what is the syntax part mean, and then we'll go and write this. For me. We use this function mean and what are the things that we'll take it, we'll take x and that will be the data vector. And then it will take another argument that is equal to zero. And then dark. So what that will do and what the stream is, we'll see when we do our practicals. So it's just and then dot RM equal to false. And that's it. Okay? So this is the basic main function in art where x is the datavector trim, and then an auto. So what is x? X is the input vector. So let me tell you here, x is the input vector, which will contain the date data. And then our dream here we are using the trim here is used to drop some values from both the ends. So if you give dream equal to zero, it will not drop any values. But if you give one, it will drop one values from each side. So it is If you wish to drop values from both ends, both the ends of this x vector. Okay, we'll see, and we'll understand better. And dark Adam will, what it will do. It will just remove them missing values. So it is to remove the missing values. So x is the input vector, data vector. Trim is drought values from both the ends and NMDA armies. And remove that. Missing values. Suppose you have a dataset and larger values are missing. So if you want to remove those values, you can use any that aren't equal to true. So when this ended up our image called the false, it means it will not remove the loads which are not having any. It will not remove the indexes which are not having values. When you use this as a true, it will remove all the missing values. Okay, Got it. You can put it true or false based on your requirement. So let me comment this out. So now what we'll do, we'll create a simple vector. So what I'm doing here, I'm going to create a rectangle that is input vector. Okay? So I'm going to create an input vector. So what I'll do, I'll use suppose that data. Suppose a huge one as the input vector, or a simply a huge. And i'll, I'll, I'll give some random values here. Okay? Suppose I have 789, 671-250-6304 to 405-80-9907, minus two, -34, 32 -21, all those, all the dwellers. So we have our input vector which is containing the values. Okay? And now suppose I want to find mean. So, so what I can do, I can simply use suppose mean of a that I'll create a variable here. Mean underscore a, that means mean of a. Okay? And I'll simply use the mean function and I'll pass this one a, I'll pass the EBITDA here. So what this mean function will do, it will take this, did this vector as input and it will calculate mean of these values. So let me this mean underscore. So it will give us the mean of these values. Okay? Let me run this. Here. We are getting mean value as 33.7, 8571. So this is the average of these values are mean of these values. Okay? So this is how we can find them. Mean of input vector from the input data mean of the data. So these mean of this data is 33.78. Okay? So next thing, what we are going to learn is how to apply and deduct atoms. So suppose I'll create another vector here. Let me put this terminal or console on the left, that will be EG. So I'll go to the paints and I'll put this on the right so that we can see that right here. Okay. So mean is here, okay. Now, what I'll do, I'll, sorry. We are going to remove the missing values. Okay? So suppose I have the same vector with some missing values, okay? So these are the missing value. Suppose one is here. Okay? So for these, we don't have Beta is NA not applicable. Okay? So how to remove these missing values? So we can simply, if I use mean, underscore a, sorry. And I'll use mean or what we get the result as. Let's see. So let me. See, we're not seeing any results because it has the NA values. Okay? So now if I put a mean of a comma and dark them equal to true, what will happen? It will give us, let's see. Sorry I have used and that is wrong. So let me run this again. See we are getting the value now. Okay? If I remove any that are altered through from here, and if I run it again, what will happen? To see? See, we aren't getting any. Okay? So if you want to get the mean of this vector dropping the NA value, then you have to use any that autumn is called true and this will give you the okay. So next thing, what I'm going to do, I'm going to use trim function, so using Trim option here, okay? So what I have done, I have created a vector C, which is having ReLu, simple values, 1234567. Okay? And if I run this and find the mean of this will get some mean value. That is four. Okay? So and how we are getting four, because one +2, 361-015-2120 8/7 is equal to four. That is the simple average. Okay? Now, what I'll do, I'll simply for seven here. Then. I'll put one here, I put four here, and three here. So simply, I'm just putting the values that a random place. Okay? So this is not a sorted array, right? So if I find the mean of this, again, I'll get the photo. Okay, we've got the same value, right? If I lose trim equal to zero, what will happen? Let's see. We will get the four right? Now, if I use 0.13 equal to 0.1, what it will do, it will sort this Addie. It will sort this in ascending order, and then it will drop the one where one value from each side. So let's see what mean we're getting here. We are getting for Y because it will drop 1.7 and the rest of value for two plus 244 plus 36 plus 399 plus 514, and then plus 62020/5 is four, right? If I dream Coulter to what it will do, it will drop the two Alice from his side. And again we are getting for. So let me put this the original, the sorted one. So I'll just comment on here and I'll put 123,456.7. So what this equal to zero to 0.2 will do, it will remove 1.2 from this side and 67 from this side. So what will lead? 345. So it will relapse with the C equal to three comma four comma five. So seven plus three plus 47 plus 51212 divided by three-fourths. So that's where we're getting four again. What will happen if I put ten? We call 2.3 and run this. It will sort and remove three values from each side. And again, we'll get for y, we call if we remove 123 from this side and five-sixths n from this side, again, we'll get the 41 of the four will be left, right? Only four will be left and mean of four will be four. Okay? So that's how it works. Let me, I'll put some more values here, 910910 here. And let's run this. See now we're getting the mean as phi y because it will remove three values, 1,234.78, 7910 from this search. So what we're left with, left with 45.6. So 4561, what will be the mean? It will be 15, right? Some of the for 5 s per 15/3. That means number of values three now, so footprint divided by three, we get, we will get five. So this is how we can use the trim function. So first, what it will do, it will sort this at a resort, this vector, and it will make it like this. And then it will remove three who I lose from east side, from the, from beginning three and from the ending three. And the remaining it will get you the mean. Okay, so this is how this dream works. Okay? So if you want to remove the values from beginning and the end, both the end and we want to find the average, or you can use the trim and the value just Texas 0.1. You don't want one means one wireless from his side. 0.2 means two letters from me, say 0.3 min, three values from each side. Okay? So this is how it works. And if you want to remove the missing values, you can use any dot Arab equal to true. It will remove these NA values from the dataset and it will give you the remaining values, and it will find the mean of those remaining, remaining values. Okay? So this is how nn dot autumn and trim works. See you inside the next lecture where we'll be learning about median in how to find median in art. 39. Finding median and mode in R: Hello and welcome back. In this lecture we are going to see how we can calculate median of the data, okay? So how we can find the median value? So first thing first, we will be knowing what is the median. So median is the middlemost value in a data series. So suppose we have this data series. So we want to find the middle value of this data series. So let's get started. Okay, so medial, most relevant, the value which comes in the middle, It's not like me, which is the average value of the data series. But this is the mean, which if we plot this data on some x's, so what value will come in the middle? That is, the value, we are going to find, the median. So median is the middle most value in a dataset is okay. And to find the median, we use the function median in r. So we'll use function median in art. So not me DnaA, it's median. So we'll use this function, median, okay? Me DIN median to find the median. The dataset. So this is the dataset I have created and this is the median function. And what are the values it will take. It will take the E and the input vector. And then it will take any dark autumn call to false or true, whatever you want you can give. Okay? If we want to remove values, you can put Adam dot. If you want to remove the N A values, you can put it at Aram equal to false. We have seen how to use NADH in the previous lecture where we have discussed about the mean. So at the same functionality and also to any data item. According to false means it will not remove the missing values. And if n is not equal to true, it will remove the missing values. Okay? So now if I run this, what we like it, I'll get, I'll get the median of this series. I'll put yours median underscored a and assign this median to this. But **** hard this well, this variable, the median underscore age. So let me print this. So if we run this, we get the median as like some middle value of this data. Seizes series is nine, right? Suppose I'll use, I'll put some more random values like 6745, 2245, 4722, 9979. And now if I tried to find, we'll get 33. Okay? Similarly, it will, it will try to find the middle most value of the dataset. I will do it for free and this will get 44, okay? So it will try to find the middle most value of the dataset is okay. This is how we find the median of a data series, our dataset. Now, next thing is we are going to learn that is finding mode. Okay? So now we'll use to try to find the mode. What we have learned here, we have not finding median. So now we'll learn how to find more. So what is the mode? Mode is the value that has the highest number of alkalis inner city. Suppose we have this. Let me copy this. And let me create the dataset for this finding more problem, okay? And this is our dataset where we have support this 45, 45 is occurring many times, so 45 than yours, I'll remove this pseudo 45, and then I will get for diverse. So t Here, 45 is repeating 1234 times, right? Support. So this finding mode means mod Easter. Number of finding the maximum number of occurrence or for value, right? So mode is the value that has highest number of documents in our dataset. Unlike mean and median mode can have both numeric and character. So first we'll see how to find mode of this numeric dataset. And then we'll say try to find for the character dataset also. So if I use mod because there is no inbuilt function to find more in R. Okay? So what we will do, we'll create our own custom function to find them maximum number of occurrence of data value in a dataset. Okay, So for that, what I'll do, I'll create function. So what I can here, you will get for the sum, okay? So I'll create region function mode, okay? And what I'll do, I'll give this name as they've done. Let's go mode. Okay. And for this, what I'll do is I'll use function and then pass this. Okay, so what is the dataset? Okay? And under this what I'll use, find unique, unique underscore age. So unique value in that what I want to find. So for this, what I'll use, I'll use a unique function that is in art. And what I'll do, I'll pass this a here. So this unique function will give me the unique value in this dataset. Okay? So let me simply bring this may come to school. They know what I'll do. I'll simply use more than that's called a. And what I'll do, I'll just call this select Done mode false and okay, so and I'll pass a liquid on this. Okay? So let me commend this and I need to put into the mode, mode, and that's called a. Okay, so let me run this again. So see here what I'm getting now. I'm getting the same this dataset right? Now. What I need to do, I need to do some calculations here. I need to write some logic here to find the unique value number of occurrence of a particular. We'll find out like this 45 number, most occurring number in this series. Okay? So for this, what I'll do, I'll use unique underscore a. And what I'll apply here, I love playing rich, dark max. And then I'll use tablet, tablet, tablet. And then what I'll use the match function. And what I'll do, I'll use pasta, a coma, unique and does good day. Okay. And now if I get the return, if I run this, I'll get the mode, okay? Yeah, 45, I'm getting done to digitize 45. So this is how we can. To create a user-defined function for finding mode. So here we are creating our own function, return underscore more, which is taking this vector as input vector or the input data we are giving the data here. So this will, this dataset will go inside this function. And here what I'm doing, I'm creating another valuable unique underscore j. And here I'm using the inbuilt function in R that is unique, unique. So it will give the unique well-worth. And then what I'm using for this finding, this unique underscore a, um, you didn't give it to dot max, which is the, which is occurring maximum number of times. So I'm matching with this e, this original dataset with this unique data set and finding which number is happening most of the time and I'm getting the result. So here I'm creating another mode underscore j, and I'm just calling this function and it will give us the most number of occurrence, most occurring number from this dataset. That is 45. If I suppose a good 76, most number of times. And if I tried to run this, what will happen? Let's see. Now we're getting 76 as the most struggling number. So this way we can use this, we can undo the same thing with the text as your MBA support. I want to create character underscored some dataset. And here what I'll do, I'll put some string values. Suppose I'll put the country's name like India, USA, South Africa, Australia like this. And what I'll do, I'll support that. They beat this Australia few times. Okay. Repeat this Australia twice, okay. And repeat this pretend twice. Okay. So now this is the, this is the dataset where we have written twice and Australia twice. Okay? And I want to find the mode off. That's called a late. So what I can do, I can just call this return underscore mode. And I can just Boss this dataset here to this function written mode, okay? And if I print this, I'll get the Australia as a result. So let me run this. Okay, so I forgot to put C here, silly mistake that I have done. So let me put this in here. Okay. So let's stop the thing is okay. So we were getting better because we have not put C here. That's where you are getting into it. And I was so ignoring that, I didn't see that. Okay. So sorry for that. Let me run this again and see you. Now we're getting Australia the digital, so hostilities reporting twice and Britain is twice. So we're getting Australia. Let me try something. Like I put Australia probably send everything twice as well. Both the things as two times. Let me see what we get. We get Britain. So when you get the first stock grants first, okay, so pretend will come first and then Australia. And suppose output India as well. Twice that we see what we get. In C, we're getting India, how k. And suppose output, pretend first, Curtis. And I put it here. And if I run this, let's see, we're getting Britain. So what do we ever the first document that will be given highest preference. So even though meridian, India and Australia all are occurring twice, it is giving us the result as Britain we call pretend it's flush TO hockey. And if I put Australia, let me put Australia again. And if I run this now, we get to Australia and somehow getting attached to them. So this way we can find the mode of our data more from our data. Most marketing value from our dataset. We eat us. Character dataset or numeric data set. Okay, so we can create our own function I didn't mode, where we lose unique function and then we will allude which dot max. And we'll tablet that and we'll match and find the maximum number of occurrence of a particular value. Okay, So this way we can use more. So we have seen meet in the previous lecture and median and mode in this lecture. So now we know how to do the basic of these things, mean, median and mode in our programming, even though Morty not having inbuilt function. To find more, we have created our own user function to find the mode of a dataset. I hope you've got to understand how to do that. See you inside the next lecture. 40. What is Linear Regression: Hello and welcome back. So in the previous lectures, we have seen how we can do statistical analysis in R for machine learning or data science. So these are the things that are very useful and we should be knowing all this statistics to proceed further in machine learning and artificial intelligence or deep learning, whatever you want to learn further. In this course, we're learning data science and machine learning through our programming. So we have lawn mean, median and mode, where we have seen that inverse function to find mean and median, whereas does not have inbuilt function to find the mode. So what we have done, we have created a user-defined function that we have created a custom fonts and to find the, to calculate them more. And then we have calculated the mode we have done in the previous lecture. So you can go and see that if you have not seen it. So what further? Now, we're going to learn a very important concept that is called linear regression. Linear regression is very important and it's widely used in machine learning and artificial intelligence. So if you want to proceed further, you have to know what is linear regression and how you can use that to predict. So linear regression is a machine-learning model. With that, we can predict the values based on, suppose we have data, suppose we have data here, height and weight, where we have the weight of proportion. And based on the weight of proportion, we are calculating the height or width and the height. We want to calculate the weight of proportion. So anything that you want to establish the relationship between these two variables, height and weight. We can do that and to predict, suppose this is the sample data that we have, that we have collected through our experiments. So now we have the data height and weight. And based on this data, we suppose we have these discrepancies thing up to 19 dataset here. And based on this dataset, we want to train our model, machine learning model on linear regression model. And suppose I want to a new height of a person, I'll give a new high-tech is not present in this column and I want to predict the expected weight of that person. So what I do, I train my model with this data. And whenever I give on new height of a person, it, the system will predict the weight of the person, the expected weight of the person based on the calculus and it will do. So this way, we can predict our weight of a person based on their height. So we'll train with this data and any data, any new data we can give height. And it will predict expected weight of that person. Okay, So that is what we are going to do through linear regression. So what is linear regression? Linear regression is as tactical statistical method used in finance, investing or any other discipline that attempts to determine the strength and character of the relationship between one dependent variable, usually denoted by Y, and a series of other variable known as independent variable. So what did you understand what this model, if some method to find the relationship between two variables, one is independent, Hannah that is dependent. So what is depending on what is independent? Suppose, based on the height, I want to find the weight of the portion here, height, which we are giving the input will be as, will be called as independent variables. So here height will be independent variable and the weight will be the dependent variable because based on the height, we are predicting the weight. So based on height, the input which we have like height, that this will build on the value which we want to predict. Based on the height we want to predict. Height will be the independent variable, and the weight will be the dependent variable. Because depending on the height, we are predicting, the weight, height will be though independent variable, and the weight will be the dependent variable. Okay? So you've got to know what is dependent and independent variable, right? So dependent variable is denoted by Y. And independent variables are the series of other variables. So there will be one leaf. Suppose you have a company and you want to predict the revenue of the company. So the revenue of the company can be only one variable. So that we can predict based on the several other things like how your company is performing, what is the appropriate, what is the loss? Hey, how many clients you got, how many clients you lost, and how, how you are giving Salish to our implies. So all these salary component, profit, loss, market condition, all these are the independent variables that are going to define your revenue. So revenue here is a dependent variable and all other things that are going to affect your revenue is called dependent variables. So there, there can be many dependent variables and there will be only one independent variable that we are going to predict. Okay? So regression analysis is set-top statistical processes for estimating the relationship between a dependent variable, often called outcomes. So what outcome we are going to get that is called dependent variable. And one or more independent variables. So one or more independent variables implies salary, profit, loss, market condition, all those things often called predictors or covariates or features. So these are also called features based on the features we're going to predict some value. I'll pop particular thing, okay? And these are also called predictors because they are going to help us in revenue or the weight of a person. The most common form of regression analysis, analysis is linear regression. Okay? So here there are multiple regression analysis like multiple or were they going to send an analysis of simple linear regression, linear regression, all those things. Okay? So in the linear regression what we do, we find researcher finds the line are more complex linear combination that most closely fits the data according to the specific mathematical criterion. Okay, so what does it mean sea? These are the data Nick weight on the x-axis and Height in the y-axis. Okay? So based on the height, we're going to predict the weight of proportion. So here, if you see these are the points. These points it like 64, 64, R1, 77 KG, 177, height. The weight is 64 point something, right? So this point, these are the data points that we have. So what we do with linear regression, we try to find a line here. Try to find a line here which will represent the data, which will prepare them. Outcome data like support, you find any point here like 65. And when you draw a line here, it will cross there, you'll get a point and build on this weight. You will get that height are better than this height y value, you'll get the X value, right? So this point will give you the x and y combination, or height and width data, right? So when you draw a line here, you'll get the one. Suppose we are given 65.8 or we're giving 162 has the highest. So when we come here, we'll get the x value here. That is, that will be the weight of the person. So we will try to find a line which will correctly represent the line, okay? And data, right? Data according to the specific mathematical criteria. Okay? So linear regression, these two variables or through an equation. In linear regression, these two variables, dependent and independent variables, will wear exponent power of both digital level is one. Because what we're going to use these one simple line equation write y is equal to m x plus c. Y is equal to m x plus c, where m is the slope and the constant value, alright? So is a constant, right? So here m x m is slope. So based on this, we can draw any line, right? So this is the simple line equation, right? Straight line equation y is equal to mx plus c. So based on this X and Y, we try to find this line which will truly represent those data, height and weight data. And since they're called their power is one, y squared m x plus c. The power of x and y is one. It is called simple line equation. And if their power is not equal to one, it will form a car. It will not be a straight line. It will be some curved line like sigmoid function or something, right? So based on this equation will try to find, okay, so y is the response variable, right? Because y value we are going to predict based on the x value. And x is the predictor value, value or independent variable. A and b are called coefficients. So when we perform linear regression, we'll try to find that. A and b value, these are called coefficients, are coefficients of regression. And build on these two values, a and b, we find the y value, okay? And with that, we will get these points, these points on the line. And when we draw a line, joining those points will get a straight line or a decrease on line. And this line will give you the true prediction of the data. Okay? Where is it used? Well, we use the linear regression. So regression analysis primarily used for two conceptual distinct purpose forces. Analysis is widely used for prediction or forecasting. Suppose we have height and weight data. We want to build on the height. We want to predict the height of a person there. We can use our forecasting. Suppose tomorrow it will rain or not, true or false cases. Okay? So that kind of forecasting we can do, like on the weather will be like a sunny day. I'll go out on a sunny day. It will not rain. Forecasting, we can do so in the forecasting and prediction situation, we use the linear regression in machine learning. And then second is situation rigorous analysis can be used in for casual relationship between the independent and dependent variables support you when you have two variables and dependent and independent, you want to find how they are related. They are, we can use that linear regression to find the relationship between the dependent and independent variable. So now how to establish the linear regression or regression. Simple example that we are going to do in our hand, sunsets on, that will be predicting the weight of a person based on the height. So if we know the height of a person, we can predict the weight with this linear regression analysis. So to do this, we need to have relationship between height and weight of a person and how we do that, because we have that data height and width here that we're going to use in our example. So what I do first, we need to collect the data for which we want to establish that relationship. So we will have this data and then we'll, what we'll, we'll do once we have the data will. The actual thing is, once we have the data will perform some exploratory data analysis to clean the data if some missing values there, how we can remove, what, how can feel that our data. So those kind of things we do in a real life. But here, what we'll do, we will have sorted a very character data here. So we know not to do all those sort of things that we'll see in the latter part of the course. But for now here we have the very clean data. So what we'll do, we will perform that. We'll use the lm function and we'll try to establish the relationship between height and weight. Height and width, y and x, right? Height and weight will try to perform. The lessor will try to create a linear regression model using the lm function. Lm function is the inverse function in R that will do all the mathematical calculation in the background. And it will give us the relationship between x and y. Write y and x are heightened wet. It will create the real essence IF function. And when we get, when we use summary of that real essence, if we can find the coefficients a and b and how they are related. So based on that, we can draw the regular insulin line. And we can also predict the weight of a person. You didn't predict function. There is another function called predict, which we'll use this relationship that is lm function. Okay, so it will use the linear regression, our relationship that we have established in this step. And it will use this x and y values and this relationship and try to draw the regression line for us. We'll plot the regression line here, but here with the predict function. And predict function will internally use that function or relationship that we get through linear regression model. And it will use this model to predict the height based on the weight. So we'll see how we can do in the next lecture. So I hope you got to understand what is linear regression. Linear regression is something like support you. We have the points here and we want to predict based on the new data. We want to predict that based on the height, I want, suppose based on the 230 height, what will be the weight I want to find? So here, if we only have the data points, we cannot do that. So what linear regression model will do? It will create a straight line which will truly represent these data points. So it will be done to 30 if you draw a line here and here, the intersection point here, it will give you something like what will be the height of that person. So it will give you a regression line. Based on that, you can predict the predictor value of R, predict the weight of a person based on the height. Okay? So see you inside the next lecture. 41. Prediction Using Linear Regression Model: Hello and welcome back. So in the previous lecture, we have learned what is regression and what we have learned, what is the linear regression and how we can use and where it is. You're right. So this is the practical system. And here we will try to write our code to implement simple linear regression, where we will be using one of two vectors, x and y, where x is containing the weight of the portions. X each containing the height of the patient's height of partial height of few people, okay? And then y will contain the weight. So x is height and y is containing the weight corresponding to this x. Okay? So based on no height built on the x, we want to find the y, right? So this is our dataset, right? So based on X, on the height of a person, we want to predict the height of a partial Saudi made on the height. We want to predict weight of the person, right? So based on x, we want to predict the y, which is here we are considering why other weight. So x is height and weight. And it is pretty simple to establish the relationship between x and y in R. And because R is having oh, lm function, lm function is linear regression function or linear model. Lm function will create that relationship between x and y. So here the first variable, the first data vector data we will define here that is the dependent variables. So this why we want to predict based on the x. So when we write like this, it means based on the x, we will predict y. So y is not predict value, and x is the predictor. X is the independent variable and y is the dependent variable. Y. Y is dependent because y value is depending on the X provided, right? So whatever x value will provide based on that, our model will predict y value, that is the weight, right? So we'll provide X that is height of proportion. And based on that, it will predict the dependent variable Y, which is weight and height. We will predict the weight of proportion. So this lm function simple, we are not going to do the background math because lm function will implement it, will find the coefficient and all it will establish that relationship. But in R, since we are using in vitro in built libraries of lm function, it will automatically create the relationship between x and y. And based on that, we can predict. So first thing, first step is trying to establish the linear relationship between x and y, okay? So predictor will be here or dependent variable will be here. And this is the independent variable. So based on the height will predict x. Why that is, wait up the portion. So here I'm simply giving a filename, W, height, weight one dot PNG, any file. Let me, you can give them here what I'm doing. I'm just plotting y, x, y values, x values on the x y plane. And I'm giving it a color blue. You can give it green. And we can write here height, weight the lessons. If you can print the lessons, the regression, whatever you want, you can put here, okay, and then I'm using AB line. Every line will create the regression line, and yet I'm providing the LM. Okay? And then in this we'll, we'll see how we can change and what is that? It's it's like haha, how you want to put the data points on the graph. And then on the x-axis, waiting cagey and y-axis, height in centimeters. Okay? So this is how we will plot it. And then what I'm doing here. I will provide the high-touch 170 centimeter. So here I'm providing X equal to 170 centimeter. And based on this x value, I want to predict the predicted weight of the person Y. So X I'm providing here. So I'm creating a DataFrame and I'm plotting x equal to 170. This will be, I'm giving this variable name as a new edge, new height. Okay? And then what I'm doing, I'm simply predicting a museum, the predict function, predict and I'm using this relationship that we have established. And I'm providing. The independent variable that is x. So this x value is here. New wet. I'm providing new yet that is one 70s, one providing what I'm plotting x here and I want to predict the Y. So when we run this, we'll get the y-value. So for 170 centimeter height, what will be the weight of the person that will predict based on the relationship that we have established in the linear model, linear regression model. So let me run this and see the output z. Output we are getting as 67.33. So for 170 centimeter, height of a person is having height of one 70s centimeter. Our system is predicting that it should have the high top 67.33. Ceo. If you see 170, it will be about that. Okay, I see 167. It is. Our system is predicting for one, 70s, 67. Okay? So it will do the mathematical calculation inside and established earlier lesson and we're done that listen, based on this model, that linear regression model that we have created, it will predict the weight of the person that is coming about 67.33. Let me show you the graph. Well, so this is the graph that we have printed here. Make sure you see here these dots, the data points are coming in green because we have given here in green. If I make it read, the data points will convert into right, right. Okay, so let me open the file again. The data points are in red. And one more thing I wanted to tell you, see, this CX value will define if I make it supports 2.5 and if I run it, the data points will be bigger. Okay? So see ya, now the data points are coming in a bigger form. Okay? So this way, this is the x value is to increase or decrease the data points sides. Okay? And this piece here, cysteine and let me change it to 26 and see what impact it is voting. Now it is coming like this. So this is basically how you want to put your data. So now it is coming now. Rectangular manner, straight. If I put it to the project in, it's coming in a diamond same. So this is basically how you want to put your needle, how you want to like make it a solid color. So basically PC's value will change the shape of the data points, right? Portrait cysteine, it will come in more or less circular in circles. Okay? So now and xlab is what we put on the x-axis and the name on the x-axis and then the y-axis height. So this is height and this edge weight. And this is the main that if we have used manage equal to height and weight and zip. So that will come here and see, yeah, this is the regression line that our model has drawn. So this will give you them. If you want to predict a value, you can just put here 67 and it will give you this value system will predict. So see your process T7, the height is, let me go a little deeper into the thing. So searched for 67, the height is one. Such T7. I didn't want preparatory it. And let's see what our system is predicting for 150, okay? That's my port 165. So you can see for such T7, this one too. But our system is predicting five. So it is similar like 67.25, 1321 to T7. And our system is predicting one that will put the checkpoint and C. So this is just like playing around with the code. If I pour this, let's see what we get. C, we get 65.44. In our dataset, what is that? Coming around 67. So this is how our system is predicting. It is quite accurate, okay? And this says that regression line that it has drawn. So here we're predicting, okay? Next thing what we can do is like my ideal state. Okay? What I wanted to tell you more, we can somebody off lesson and we'll see what somebody it gives us. So if I put weight on this alone, CEO, we ought to getting all the residuals like minimum, one quartile median, third quartile maximum values and then the copy coefficient m and c m and c m x plus c m and c values are coming out like intercept here. And that is coming this t value and P value is coming like this. So these are the things, these are the cooperation that is calculated by the lm function. And based on this, it is predicting the weather. And based on this, it is trying the particular cell line. And CEO multiple R-squared values are coming like this. Adjusted R-squared is coming like this. And F statistics coming this. Okay? And the p-values here, we'll see in the like, these are the beyond the scope of this course. When we learn, when you learn machine learning algorithms in deep, you will be coming to know what are these copies and how they are used for. That is beyond the scope of this course because we are just learning our programming for machine learning part. You should be knowing that now lm function will internally use these 3D worlds cost coefficients and the standard error that it will come because no model will predict Connect button, so there will be error. So the residual standard error is coming around this 17 degrees of freedom R-squared value. So what I want the predicted values and the expected value, the predicted values and the whatever it would be. So that will be the error. Whatever we are looking for and what system is giving that there will be an edit. And if we do is quite a bad day, we will get those are described at that. Okay? So these are the values will be used by machine learning linear regression model internally and to predict the dependent variable. Okay, so I hope you got to understand little bit of a linear regression model and how you can use lm function to predict and how you can draw that linear regression line. Okay, So Stephen site, the next lecture. 42. Reading CSV creating LR model and Predicting: Hey guys, welcome back. So in the previous lecture we have seen how we can use linear regression model and how we can create the relationship between the dependent variable and independent variable, and how we can predict, and how we can even plot the linear regression line. Regression line I meant. Okay, and we have seen what are the coefficient when we use the summary and how the linear model for pixel, and we have seen how we can predict the data. But in the previous example, what we have done, we have taken the vector r, we have taken that data vector for height and their weight. And then we have created the linear regression model for x and y. And then we have predicted, now, in the real world, we will not have the data in this format, in the vector format. Instead, we, most of the time we'll have the data in our databases are in the Excel or CSV files. Most of the cases it will be dot CSV file, comma separated files, right? You know. So in this example, what I'm going to do, I'll tell you, like we have already seen how we can read the CSV file in the previous lectures. And that, that knowledge we are going to use in this lecture to create linear model. And we'll read the data from the CSV file. So for that, I have created with the same data height and weight that we have used in the previous lecture, I have created a CSV file with weight and height. So this column is the weight and this column is height. Okay? So this is the CSV file, height, width dot CSV file. And what I'll do, the first step is we'll read the data from the CSV file, will treat the CSV file. So for that, I'm using a metadata and I'm using read.csv function to read the CSV file. So with this read.csv, it will read the CSV file and it will get the data into this vector, this object, data object. So we can see with our read.csv file is reading the data properly or not. So for that, we can use the view function and provide the object inside that. So when this view lungs and we'll get this data object, it will go here and it will read the CSV file. And then with BYU we can get the view of the data. So let me run these two lines for you. See here, we are getting the view of the data TO in our weight and height. So the same, sorry. See the same data we can see in the RStudio now with the view function, wavefunction will saw the data in this way. So now we are clear that our data file, data we have read from the CSV file. Now what we can do next step is create the linear regression model for prediction. So we want to predict the weight and the height. We want to predict the weight of the person. So for that, we'll create the linear regression model. So we will use lm function and w here, weight will be the weight will be the dependent variable and height will be the independent variable. So based on the height, we want to predict the weight. Here data is called data. Data means we are using this data. You can give any name to it. You can. Now, we can modify this with, sorry, height weight, sorry, actually W data, height weight data. And so we are eating their CSP file. We'll put this here. And then data is CO2 will put height when data, okay? So this way, data source is height weight data that we have gotten from the CSV file. And here we are creating the linear regression model using the LM function. And we are giving the dependent or independent variable here wait, and independent variable is height and data equal to our data source will be that TWO data. And let me just comment this so that it will not disturb us. And now we have, now this model. I have given name and a model. And we'll use summary of our model to get the details. So let me run these two lines are, let me run till here because we have modified. So let me run this. So see here, now we are getting the Summary up our linear model, linear regression model like cholesterol levels or this one, quartile median. Third quartile max coefficient, tight intercept, all those values we are getting here. And then we're getting the standard error. Standard error as 1.21, 0.789, 17 degrees of freedom multiple R-squared value adjusted R-squared value, p-value. And we are also getting at statistics, okay? So now our model, that means a linear regression model is ready. Now we are ready to predict based on the new data. So what I'm doing here, I'm providing a new height that is 152.21. Okay, let me modify this to 16485. Okay? And for this height, I want to predict the weight, what this 165.85 centimeter height, I want to predict the weight of the person. So this amusing DataFrame, because whatever we read from the CSV file, it will come as a DataFrame. So I'm creating a DataFrame here and with height equal to 174.85 and I'm giving it a name knew yesterday's newest object will contain the site azure 164.85. Now, the last days is predicting the weight using a model. And for that, we are using predict function. And inside the predict function, we are providing two arguments. One is our model, our model that we have created here. So this LM model I'm providing here, so that based on this relationship it will predict and what predicts. And we are providing the data that is height. So we are providing height as 164.85 year. So we are providing the data object here that the new edge. So it will take this as an input. It will go to this model. And based on this real essence of height width, it will put height as 174.85 and it will predict the new weight, weight of this person. Okay, So now this predict some value. I'm keeping it in weight predicted because we are the weights. So I have given wet predicted as this object name. Now, the next thing is we have the new weight in this repeated weight in this object. We need to print that. So you can use print here. But since I wanted to print a multiple line, so I'm using cat function. Okay? So because printf function will print only 11 line at a time, I wanted to concatenate or two sentences. For height. This predicted weight is this. Okay? So this can be possible only with the cat function. We can not do this with the print function because print function has limitation of only printing the objects, object values. Only it can print, it cannot print the statement. So for that I'm using cat. If you want to print something like this, you can use cat function. So now I'm using cat and I'm using for height equal to modify this one for wind. It five predicted weight is, I'm printing now where it predicted here. So let me run the whole code here, so I'll click on Source and CEO for height 164.85 are predicted weight is 661794. So now with this we can predict, so now our model, this linear regression model is predicting the weight as 66.79. Let me show you this model and the earlier model is giving same result or not. So what I'll do, instead of one PIP2 in the previous example, I'll put 164.85 and I'll try to run this senior year also, we are getting the same result, 66.7, 9465. So both way, because the data is same here, we have kept it in the Here vector form and here we're reading it from the CSV file, but the data is same. That's why we're getting the same results from the two models. So this way, you can read from the CSV file and create a linear regression model. And then based on that, you can predict the value, whatever you want. You can do that. You can try this problem with any other data points that you can have. Like based on the age, you can detect the weight or based on something that categorical data, your data you can take and you can put into the CSV file and you can predict one value but on the other by creating the linear regression model. So I hope with this example, we are pretty much confident about the linear regression, how we can create linear regression model to predict the values. And I hope you will practice more with this. And you can do much better than this, right? So when you practice, you'll get better with our programming and the machine-learning things. I hope you got to understand linear regression model. And that's it for this lecture. So let's see what we can learn further. 43. Multiple Regression: In the previous lecture, we have seen how we can use linear regression. What we have seen like linear regression using two vectors are two data objects like weight and height. And we have created that the licensee brand, then we have hockey. And then we have that CSV file land or created our LM model, the linear relationship between the weight and height and then met and the height. We predicted the corresponding less productive. Great up the partial. Okay? Now, this was the linear or the collision, right? Now, our simple linear regression, now we have another problem where we are the response variable on our dependent variable, or the value which we are going to predict is going to it. It can, like in linear regression what we have seen, the dependent variable and independent variables. Dependent variable, depending on only one variable, one independent variable rate. But in real life situations, most of the times the output or the value which we are going to predict. It is not necessarily going to depend on only one dependent variable. But there may be many, many independent variables that are going to define are going to affect the independent variable. So it is not like the situation like we will always get the linear, simple linear regression. We can have multiple independent variables that are going to affect the output. Like in case of support the way that prediction, weather. Prediction can be dependent on many situations like whether it's a rainy day, which region it is, what kind of clouds are there? And many more things, right? Suppose you have a business and you are going to predict your revenue based on a few factors. So there must be, there may be many factors, like like I said earlier, like, you must have the number of employees, how we are going to praise your employees, that is also going to affect on your revenue. How is your profit and loss? If you are on profit, your revenue is going to be more. If you are on lost, your revenues going to decrease. And if you are not going to, suppose you have some our taxes, more taxes, then you are going to be revenue going to affect it. So there are many factors that are going to decide or define your revenue. So here, this, this problem cannot be handled with the linear regression. We have to use multiple linear regression because the output value or the revenue is dependent on many factors. So here the factors are features will be independent. Features are the factors are going to find the revenue. So in that case, we will have like a, B X1, X2 plus so on and so forth, bn XN. This means this, b1, b2, bn, and E, all the coefficients later we have seen in us, y equal to mx plus c right here, only one axis there. But in multiple regression there will be b1, b2, b3 before us up to Vn, right? That there can be n number of factors that are going to affect this y. Okay? So this is the simple linear regression. And this equation is for multiple linear regression because you're B1, B2. There may be a number of factors that are going to affect the outcome. So multiple linear regression, also known simply as multiple regression, is a statistical technique that you just several explanatory variables to predict the outcome of our response variable. Okay, So what I'll do, let me show you one dataset. So what I'll use, I'm going to use in real data that is available with the R, that is empty cars. So empirical data, this dataset is available with the R package itself. So when you download R, it will be coming in built with the R package. Okay? So empty cars, so you can use view. View. I'm triggers will give you the view of the dataset. So see, this is the dataset of various gods like Martha Jackson, Hornet. That's unmatched is okay. So there are many cars and it is like miles per gallon, cylinder displacement, horsepower, weight. Okay. So there are, there are many key a number of years car. Okay. So all these factors are going to define the miles per gallon or few factors may not be affecting much. And field vectors are going to affect more. Like if you consider miles per gallon as the output you want to come to, you want to predict the miles per gallon based on the displacement, horsepower, weight of the car. Then also, maybe number of cylinders are not going to affect much of the match under have much effect on the miles per gallon or in some cases, it can have much Lexi yeah, four cylinder is having a -33, 30, 26 more. And if you see eight cylinders are having less minus two, this number of cylinders in the engine is going to affect the myelin. So this is one factor. Then. Displacement can also have effect on the average of the mileage of the car. And then read up the car is also going to affect. See here that there are various features, are the independent variables, the number of cylinder displacement, weight, and number of gears. So these are the things that are going to affect the myelin. So here we have to use multiple regression if you want to predict the mileage or per car based on the number of cylinder displacement, weight. All those factors, if you are to consider, then we have to use that multiple regression. And if you, if you see this equation of multiple regression, this b0, b1, b2, x2, and x1. So this X one can be the weight of the car, extra can be the displacement of the car. And X3 could be the other factor, like number of cylinders in the car. So and this b1, b2, b3 be, for all these coefficients, are going to decide based on the how, the, how X1 or the weight of the car is affecting the overall Majlis or average up the car. So based on these values, how it is affecting it will be decided if it is a, b1 is more than this weightage of the X1 will be more right? So this way we can use the multiple regression. So what I'll do in the next lecture, we'll do hands-on on this dataset that is empty cars. And we'll try to use multiple linear regression or multiple regression to predict miles per gallon or average up a car based on the features this dependent, so here, cylinder displacement, weight of the car. All these will be dependent or independent variables. And miles per gallon will be the dependent variables. Means these values will be cylinder weight of the car, number of cylinder weight of the car, displacement of the car number of years in the car. These will be the predictor and miles per gallon will be the predicted value. Or these will be the features or the independent features, independent variables. And this will be the response variable, or the output variable, or the prediction value. So that thing we are going to do in a hands-on lesson in the next lecture. See you inside the next lecture. 44. Predicting Car Mileage using Multiple Regression in R: Hello and welcome back. So now we will do something with the multiple regression. So this is hands-on system under multiple regression. So we're going to use empty cars dataset. And for that, we are not going to use all the columns are all the things that are out there. But we're going to use MPG, displacement, SP, and wait up to God, okay, horsepower and weight of the car. Okay. So based on these three, we are going to predict mileage of the car or the average up the car, miles per gallon up the car. Okay, so we're going to use these three features, these four features up our dataset. And three will be the, these three will be the independent variables, and this NPD will be the dependent variable or the predicted value. And these three will be the predicted predictor. Okay? So first thing is we need to create the relationship model, the linear regression model, our multiple regression model. So same, we're going to use the lm function that we have here two into simple linear regression. The theme lm function we are going to use, but with a slight different formula. So whatever we give, hear this called formula. And the second input is, second argument is the data. Okay? So here, let me put this cost data. This has a cost to it. Okay? So now this is the, this is our data. This will contain these four features, okay? And their corresponding values. Okay? So data, we're using cost data hockey that we're getting from the empty cars. Okay? We are going to create the model. Okay? Let me make it linear regression model, multiple linear regression model, MLR model. I'm giving it the name. So let me, okay. So now here, our layer, what do we do? We had only one predicted value and one predictor, right? So we used to give like this, okay, married under displacement, find the Majlis. But now we have three independent variables that are going to affect the myelin up the car. So we have to give, this will be the predicted value or the dependent variable. And here these three, we can give displacement plus that p plus weight. These three will be there independent variables that are going to affect the myelin. So this way we can use the lm function for multiple regression and multiple regression because we are using multiple features for predicting one value. One predicted value will be coming based on the more, one or more independent variables, okay? And data equal to cos theta. So let me run till here. We'll print the model. Okay, Let's see what are the coefficient and all we are getting sodium. So see here now our model is giving the intercept as 37 and displacement as wind urogenital 97 at b coefficient is coming like this. And we'd coefficient is coming this. So here is the intercept. This intercept value is 37. That will be, that is a. If you look at this formula, y is equal to a plus b1 x1 plus x2 plus beta three x three. Okay? So here we have three, so it will come up to be three. X d. Here is 37. This intercept value will be the a, and then B1 will be the displacement. B2 will be the HP, HP coefficient, and B3 will be the weight copies, and that is -3.8. Okay? So now we have the coefficient values here. We can simply print the coefficient value. If I put x. This, I'll get the coefficient of displacement. The same value I'm getting here, right? So what I can do now, I can put these coefficient values in this, in this equation to get our predicted value y. Y will be the MPG. In this case, our pilots in this case, we have this intercept and we have the B1 as this, B2. We have this and this will be the V3. Now, if we have a car with displacement 324 and HP and 110 and weight as 2.5. We can predict the average by putting these values with the coefficient and things are separating this formula. So what I'm going to do, y equal to a plus, so a will be this value 37, okay? And x will be the intercept of displacement, this one. And then I'll multiply that with the displacement value 324. Then x is p will be the coefficient of HP, this one. And I'll multiply that with the 110. That will be the hottest part of the car. And then x t will be the coefficient of weight that we're getting here, -3.8. And I'll multiply that with the weight of the car. So see ya simply by putting in this equation, these values, we can predict the output, whatever this equation, gift output, it will be the predicted mileage of the car. So let me just run this let me run these two lines. Now Predicted miles of the card is 23.87. So based on the question here, based on this equation, because here E value will be 37 -0.00, 0937. And all these values. And if we put in this, we'll get the y-value. Y is the mpg miles up the car. So the car with 324 displacement, HP hundred and 10.2, 0.5, our system is predicting the myelin as 23.8 7247. So 23.8 will be the average of this car. So similarly, you can give another value for the displacement at B and weight of the car. So based on these three independent variable, we are predicting one dependent variable that is y. So see here we have the coefficients and intercept in this equation to get the mileage of the car. This is pretty simple and pretty real, right? We're getting the real output, we are getting the output. And that is, that is through a multiple regression model that we have created through this relationship, through this formula. Here the formula is you only changing all other things are quite similar to the linear regression. But here, if we want to put more features yet, you can put that more independent variable here. And there will be one dependent variable or the output variable. Okay? So based on this, we are getting the output as 23.87. So this way we can use the multiple linear regression and multiple regression to predict the value. So predict one value based on the many features are many independent variables. We have to use multiple linear regression. 45. Logistic Regression: Hello and welcome back. In this lecture we're going to learn logistic regression and we'll see a simple example of logistic regression and how we can use that. Okay, so first thing first, let us understand what is logistic regression. Logistic regression model. Logistic model, also known as logic model, is used to model the probability of a certain class. Like suppose if you want, you, if you have some Watson's data where I'm like diabetic label, label all those things. And based on that, you want to predict whether them sun is fit or not, right? So here the person is fit or not fit. So yeah, the data is the output which we want to predict is kind of binary in nature, right there. There are two things that fit, not fit, fit person or not a fit person, right? So the logistic model is used to model the probability of a certain class or the event exist such boss pill, if you're writing anything jam, you will either pass or fail rate. If you are participating in accompanies and you will win or lose it. So there are binary output right, past pale, win-lose, live, dead, healthy, or sick. This can be extended to model several classes of events such as determining whether an image contains a cat or dog, lion, right? So you can put our e-mails and you can train a system that it will find the, whether the images of cat or dog or line, right? So cat or dog, lion kind of thing, right? Each object being detected in the image would be assigned a probability 0-1, right? So it can be a cat or dog, right? So being cat, one being a dog. And if you present any, any image, it will either assign the value zero or one, or a probability between zero or one. It could be 00.5 or 1.65, like that, right? So probability will be 0-1 and all the sum of all the probability will be equal to one because the occurrence of events, probability of occurrence of events would be always one, right? So that way we use our logistic regression and logistic regression, right? Equation for logistic regression is y is equal to 1/1 plus e power a plus b 1X1 plus b2 x2 plus B3 X3, and so on. So this is the mathematical equation of logistic regression. In Python, we use GLM function to GLM function for logistic regression. So GLM function for it is also known as generalized logistic model, right? Glm. And it will contain the formula like we did in linear regression. So we'll write When formula with the dependent variable and independent variable, right? And then we have to give the data on which we want to perform the GLM function. And then we have to give the font family. Like family. Here we'll use the binomial in our examples. So what I'm going to do, I'm going to use the theme cars data, empty cars data that we have used earlier, right? So cost data that we have used in the previous example as well. And I'm going to use AIM. Aim is like automatic or manual transmission system. So HP, horsepower, weight, and number of cylinders in the engine, right? And this EM is automatic or manual. So the in the empty cars data, it contains zero or one. So let me show you, let me run these two lines and view the data. See here, for all the cars, HP, weight and cylinder, m is either zero or one. It's a binary value, zero or one. Either it's the automatic or manual, right? Okay. For all the cars, we have data for m, which is zero or one, then it is one or zero for all the HP weight and cylinder combination. So what I'm going to do here in this problem, in this lecture, what we're going to do with the GLM function or logistic regression function. That based on what we will do, that there are four things. Four columns, am, HP, weight and cylinder. So we will try to find this. We'll make this HP, weight and cylinder as a independent variables. And m will be the dependent variable. Means. We're going to predict. No, I am well, alright, so what we'll do, we'll see supportive you are making one model. We should be knowing which are the factors or which are the features are going to impact the predicted values. More support. If we create a logistic regression model with all these three response variables or independent variables or model may not be correct because in these three HP weight and cylinder, all three will not impact the cars being manual or automatic in the same way, right? Maybe the cylinder number of cylinder endocast will decide have more impact on carving considered as a maid, as an automatic or manual or weight up the car have more weightage or HP. Horsepower is deciding the factor that costs would be automatic or manual. So what we'll do with the simple GLM function will try to find which of these three dependent variables, sorry, we reach up these three independent variables are having impact on EM. So what it will do, it will help us in analyzing that, which are the features we should consider for further analysis. And leave the other IP support. We come to know that weight is having less impact on Am than other two. We can leave out the weight for the further analysis and we can go with the HB and cylinder, right? If cylinder is having not, not having any impact on this, we can go with the SPN, which so this analysis will help us in finding the irrelevant feature, irrelevant column in deciding the EM. Okay, so that's what we're going to do in this lecture. Okay? So how will we do that? We'll use the GLM function and we rewrite the formula like we'll make the m as a predicted values or the dependent variable. And these three independent variables and we'll try to create, will try to generate the GLM function. And then with the summary of GLM, will try to see what are the coefficient and p value we are getting. The p-value will decide that impact our variable is having or not predicted value. P-value. P-value means probability of occurrence of that thing, okay, probability of occurrence of weight on this HP weight and cylinder. So if p-value is greater than 0.5, it means that if supported as part HP p-value is coming greater than 05, it means that P is not having any impact, any significant impact on the aim. And we can leave the HP and go ahead with the p-value. Go ahead with the independent variables which are having p-value less than 0.05. So let me, so we have done with this. So let me, the next step, next step is we need to get it done logistic regression. So here I'm giving a variable name is car's data. Okay? So our dataset is cos Theta. And here I'm creating a logistic regression model. I'm giving you the name costs that I am and I'm using GLM function here in the formula, what I'm given, I'm given am as a independent, dependent variable, e omega, automatic or manual. It is having binary thing, right? Zero or one. And am I'm giving as a dependent variable that value we are going to predict based on the these three cylinder, SP and radar, the independent variables or features that are going to decide am right. And data I'm using cars, data that we're getting from the empty cars and families binomial. Okay, So now let me run this and print the summary of the data, a car's data logistic regression model. Now, we have used the GLM function and we have printed the summary of this. You can leave this print and you can simply run the summary. This will also give you the same thing, okay? So see here, it is giving us all the things that we got enough. Even the linear regression also regard the coefficient and all here also, we are calling the GLM function with the formula this. Okay? And we are using the dataset class data and family binomial and see your dividend. Mean, first quartile median, third quartile max, all those values we are getting. And then we're getting the coefficients for each independent variable. So here, see here intercept we are getting this value and this is the PR value, that is the p-value I was talking about. If p-value is greater than 050.05, that, well, that valuable, that response variable we can leave out, right? That is not having much impact on the aim. For cylinder. See you? The p-value is 0.6 491. It is way too high compared to 0.05, right? So cylinder, it means that p-value of cylinder is greater than 0.05. It means that this cylinder number of cylinders is not having significant impact on deciding whether the costs would be automatic or manual, right? So we can leave the cylinder next, HP, HP, the p-value is a probability value scheme 0.084, which is again greater than 0.05, right? We can leave the HP as well because that is also having a p-value greater than 0.05. Next is read the third response variable. And it is having the p-value is 0.0 276, which is less than 0.05. It means the weight is having an impact on the weight of the car will decide that the cars to be automatic or manual, right? So weight of the card is having significant effect on the m value because its probability value is coming 2.0, 276, right? Correct. So with this, what conclusions we can come to? We can come to the conclusion that on the further analysis, we are further. If you are going to build our logistic regression model further, we can leave the cylinder and HP and we can go ahead with the weight response variable alone because that is having, that is only having the significant impact on the m value compared to the cylinder and HP. Well, okay, we got to know because of this p-value, the p-value greater than 0.05 will decide the response variable is having any significant impact on the dependent variable or not. Okay, So this is the simple use of GLM function that we have learned in. Now. So you can practice with this and you can take any dataset and you can do this to find out which responds really well is having impact on the predicted values. And you can leave out the ones which are not having much impact based on the p-value. 46. Normal Distribution: Hello and welcome back. In this lecture, we are going to learn about normal distribution. So what the normal distribution. So before we understand what a normal distribution, let me tell you. In real life, most of the data, most of the data, they are normally distributed. And why I'm telling this? Because if you look at the data of height in feet, so suppose you have data, data for height and weight, and height and weight data, will we highly, highly normally distributed? Why I'm saying highly normally distributed? Because it is very rare to get the person who is having very less height. And it is also very rare to get the person with a very high idle, long height like parcel with less than 3 ft is very low probability of happening. A person with the height Clifford is very rare, right? Similarly, boston with having height of 6.5 free to 7 ft. Why triad, right? So in that sense, so most of the data is normally distributed between 4.5 to 6 ft, right? So why this kind of data distribution is known as normal distribution, okay? So this kind of beta distribution is known as normal distribution where most of the data is distributed to watch the mean of the data. So suppose if we consider this graph, this graph as height weight distribution of Parsons, then you can see that this will represent this data. This, this, this celebrate. I will represent the saddle height, salt taste Thailand. This is the longest type C or the Parsons with the very sort height. It is very rare to get right. So the person with us sort height less than three bit is very right. Let me make less than three. Less than three. Similarly, Parson width, very long height, right? Hi height. That is also very rare, right? Greater than 6.5, right? So these two data sort less than 3 ft and height 6 ft is very rare. And that's why you can see it's very low in the normally distributed curve. A normal distribution of the curve comes as when we distribute, when we plot the normally distributed data, it will come as a bell curve. Why it will come as a bell curve? Because see here, this is the very rare case of happening. It is distributed towards here and very high height is here, 6.5 ft. So these two are represented here. And most of the data, when you go towards here, the most of the data is distributed. Most of the data is distributed here, right? On this part. Correct. So this is the mean of the data, right? So this is the mean. This is the average height off Parcels. So if you can see correctly, most of the data is distributed either side of the mean, right? If you divide the data half, you'll get the mean and most of the retired divided towards the mean. So the parson height is increasing and we're going to watch the mean. And then after the average value, it is increasing and it is coming to a very high height of 6.5. So most of the data is normally distributed in this region. And this region is the region, right? So most of the people, we can easily say that they fall under the height of 526.5 ft. And after that it will be the finding of a person with the height 6.57 is very, they asked for it is coming like this. So this way we get a bell curve, a normal distribution curves are always BellKor And this is the mean. And this is called standard deviation. So standard deviation from the mean value. So like that, the more you deviate from the mean value, you'll be going either this side or this side. So this is called a normal distribution. Normal distribution curve can be very high carb like this, or it can be like this purple one, or it can be like this one. This also depends on their data, but this will be the mean of the data. Okay? This will be the mean of this data and this will be the mean of this data, right? So this is called normal distribution. The most of the things in our life are normally distributed, right? So in the next lecture, we will see what is the property of normal distribution and how we can plot normally distributed data. 47. Normal Distribution using dnorm and pnorm function: Hello and welcome back. So in the previous lecture we have understood what is normal distribution and how it works. In this lecture, we are going to do our hands-on with the normal distribution and we'll see what are the inbuilt functions that we have in our programming for normal distribution. So there are basically four types of normal distributions. Functions are level in our programming and they are denom. This function d norm, which takes three arguments or three valuables as input, that is x mean and sd, that is standard deviation. So what is X here? X is the vector number. Like what? Number's vector of numbers. So suppose you have a dataset, we have a number of support that you want to get the probability and all. So for that, the exit number vector work done number vector of numbers and mean is the mean that you want to, you have to decide what pin on what mean. You want to plot the normal distribution. And then that is the mean of the dataset, okay? And then standard deviation, you will have to give the standard deviation. The maximum default value is one for the standard deviation. And we can give any value of what the plot, but the default value is one. If you don't give the color to something, by default it will take as a standard deviation as a one. So d norm is one function. And then we have the p-norm be enormous. Also texts the top numbers, then mean and standard deviation. Then we have the Q norm, which takes the p. P is a vector of probabilities. Here X is a vector of numbers. In the qnorm, the input p is a vector of probabilities, then mean and standard deviation as it is, then we have another function called are non-banks and unknown function will take as input and mean and the standard deviation as well. Okay, So what is n Here? N is the number of observations. It means the sample size up the total number of sample that we'll go inside and take this, okay? And mean is the mean value of the sample data. Its default value is zero. Okay? Here what we are going and we're giving standard deviation, the standard default value is one. Okay? So let's start with darn normal distribution. So what I'll do first, we'll see what is normal distribution for X and d. Now, what is the norm function? So the null gift, the height of the probability distribution at each point for a given mean and standard deviation. So it will give you the height of the probability distribution at each point of the dataset. And for the given mean, because mean is the input that we give along with the standard deviation and the vector of numbers. Okay? So suppose we understand what the example, we are taking one x as a number of actors who are corrupt numbers. Okay? Whatever we're doing here, we're generating sequence of numbers from minus five to 20. And then what we are given, we are given the standard deviation as we are incrementing minus five to 20 by incrementing 0.15. Okay? So like -0.15, -5.15, -5.3, like that. Okay. So it will be incremented. Number generated number will have been increased white 0.5, okay? So X has a free toll lanes. See, this is how the numbers will get generated. Okay? And then what we're doing here, we're taking our vector y and region D norm function and we're plotting x as important that X we're getting from this sequence function. Then mean I am giving 7.5 suppose and standard deviation I'm giving pointed. And then I'm plotting the graph with the x and y. So x will be this value and y will get from this dnorm function. Okay? So let me comment this x and run this piece of code. So click here, c here. Now we are getting this normal distribution plot. So here the mean is 7.5. And 7.5 data is normally distributed with a standard deviation of 0.1. Standard deviation of point. It, if I give likes 0.5, see, our data will be coming like this point, it will come like this. If I give one, which is the default standard deviation, our graph will come something like this. If I gave 1.5, our graph will come like this. If I gave it as a tool, our plot will come like this. Okay, So you can give standard deviation as for the other quadrant and plot x and y, we will, we are plotting the x and y values. Okay? So this is 13 all toxin, which gives the height of the probability distribution at each point. So next one is p naught p norm. What it does it give the probability of normally distributed random number to be less than, less than the value of given number, okay? And it is also called cumulative distribution function. So here we are taking the same example and here we are incrementing by 0.4. And then we're generating y using the p-norm x. We're passing x here into inside the p-norm function. And then I'm giving someone by five again and standard deviation Let's pointed. And then I'm plotting x and y. And when we plot, we get this kind of data. If I put one-pointed support standard deviation, see, we're looking at like this, okay, if I increase this by wind, Support 0.5. Okay? So this way we can get the p-norm function we can use and we get this kind of plot. So what it will give it, give us the probability of normally distributed random number to be less than the given number. Okay? And the rest of the two 2-norm and our number we'll see in the next lecture. 48. Normal Distribution using qnorm and rnorm function: In the previous lecture, we have seen how we can implement dnorm function for normal distribution. And we have also seen p Now for normal distribution. Now in this lecture we'll see the qnorm and our Don Funk sense for normal distribution. So what is qnorm function? This function takes the probability of probability value and keep the number which cumulative value matches the probability value. This is pretty simple here we'll use the other important here. What we'll do, we'll use 0.1 and will generate the sequence of numbers 0-1 and three number will be incremented by 0.03. So we'll generate a sequence of numbers 0-1, and each number will be incremented by OpenGL or three. And then we will use the qnorm function and yet will provide this exogenous input. And what I'm giving, I'm giving mean S2 and standard deviation as two. You can give anything like you can if you want, give, mean as 1.5, you can give and standard deviation, you can give one or 1.5 and see how your graph is coming for two and a standard mean S2 and standard deviation two, we're getting this plot. And if I run this for 1.5, we're getting something like this. Okay? So this is how we use the qnorm function for normal distribution of the data. Okay? Now next thing is, are not anonymous used to generate the random numbers for distribution is normal. So what does it take our num and like I said in the previous lecture, it Dexter n as the input where n is the number of observations or the sample sites, right? So whatever other number we have, the sample size will give that as input to the function. So what it will do, it will take the sample size as input, like here, I'm giving cartilage and it will generate random generated numbers for the sample size. Suppose we are given 30 here, sample size. So it will generate the random generator numbers. And then we'll use the histogram to draw this normal distribution using unknown console. So this y vector will come up randomly generated numbers from this sample size 30. Okay? So once we run this, we get this histogram for those generated, randomly generated numbers. Okay, So this is how we use the unknown function for normal distribution. So let me just print y so we can see what numbers we are getting here, c here. So if you click on Console, so these are the numbers being generated using the by 30. So it will generate the random numbers like this. And the histogram we'll plot these are random generated numbers in a histogram. So this is the, this is the y between minus one to three, it is being sold. And then on the support, this is the x-axis, this is y-axis, this is the frequency. So g rho two, like minus one to less than minus one numbers that are occurring four times minus one to zero. This is the frequency of numbers and zero to 1.122 like this. Okay? So let me just change the data for support that'll make it 100. And if I run this, see, we're getting this histogram. And if you see the console, you'll see this is how we use the pnorm function to draw the diagram using our unknown function for normal distribution. Okay? So that is how we use the normal distribution. And we have seen the North Vietnam qnorm and unknown functions normal distribution of the data. How kay 49. Recursion in R: Hello and welcome back. In this lecture we are going to learn about recursion in our program. So what is Dickerson or what is a recursive function? So recursive function is a function which calls itself multiple times, right? So support you if you want to perform the same operation again and again. We can use the Dickerson to don't act like you must have learnt like a somewhat unnatural number. Some of you went number all those things like these problems like finding some natural numbers where n can vary from one to in any number, right? Up to one to 100,000, something like that. So here to solve this kind of problem, we need to use the Dickerson and your cursor is very important thing when we do the programming, right? So in our Also we can use the Dickerson function. Recursive function can call itself again and again and again to do the operations. And finally it will give us the result. So to understand how recursion works in our programming, we'll say, we'll do a simple program to find the sum of natural numbers up to n using recursion. Okay? So we'll try to find, will give a number, any number. And we will try to find the sum of natural numbers from one to that number. Like suppose here I am giving 85. So here I want to find the sum of natural numbers up to 85 min, one plus two plus three plus five plus six plus seven plus eight, up to plus 85, right? So to solve this kind of problem, we need to use the recursion. So what I'm doing here, I'm simply writing a simple function here and I'm giving it a name underscore n. That means some of the natural numbers, right? Some underscore n. And inside that, I'm taking input as an argument as n u n will be the number of natural numbers, right? Natural number up to which we want to calculate the sum. Right? Here. I'm doing the simple check. What I'm doing here. I'm simply checking if n is less than or equal to one, I simply return that. And so why I'm checking this condition here? Because if it is n is one, we need to return one because one is a one-month. The natural numbers start with one. And sometimes some people will consider natural numbers with zero all sorts. In case of 0.1, we need to return that particular number itself because you produce zero, we need to return zero and if it is one, we need to determine one only, right? No need to call the recursive function. Okay? So this is the case to take care of the 0.1 if you at all consider the natural numbers including zero, and if it is starting from one, if you consider then one, okay? So basically natural numbers start with one, but sometimes people consider in zero Agile starting from zero as well. Okay? And then I'm using this, I'm giving an if condition and then I'm using another else condition. And in else what I'm doing, I'm simply returning Gan plus some Yan plus means suppose a number is to, suppose I'll give you two. So suppose this number is two to two plus sum up to minus one, sum of one. So what it will do, it will do some two plus sum of two summer to minus one. We'll sum of one. It will come here and sum of one will return one. This function will return two plus one to plus one will be three. So if we run this function, we'll get the output as three here. Because some of natural numbers up to two is three, right? Suppose I'll give three here. What will happen? It will take, it will come into the else part because three is not satisfying this condition, right? Greater than one, right? So it will come into else part and it will just return n plus indictment three plus. And then it will call them again itself. The function will call itself the sum n function within Excel. And it will cause some of N three minus two, some have two, some have two will come here. Then n is equal to two, it will come here. Then what it will do, two plus three plus two plus sum of two minus one, sum of one. It will go in here and it will return one. So three plus two plus one. How much? Six. So this written sex. See here the output is six years. So we understood how it will do. Let me just put the command tense. Are you the first time when it will come? It will be like, it will be like hello, done. Three plus three plus sum underscore N. Three minus 13 minus one means two. So it will cause some up to, okay, then it will call itself. This is called recursion. Then again, in the next step, it will, the next step will be written three, sum up to how it will come. It will come here to function, will take input S2, and then it will come in the else part, bu plus. So here it will be coming like three plus two plus two minus 12, minus 12 minus one is sum of one, right? So next step is three plus two plus sum of one summer fun water it will go, it will go, it will call the function itself. And it will go to the sum of one. Sum of one will be returning n, that means one. So this will give us the result one. So it will come three plus two plus one. So this is called the ticarcillin, the final output we are getting as just six. And here also three plus 25 plus 16. So this is called Records sun, right? Support. I'll give you a big number, 785. So it will be going inside this function and it will be calling 75 minus one, then 782-52-7875 minus 1784. Then it will come here, 74 minus 17823. So it will keep on calling. This is called recursion. So if we run this, we'll get the output like this. Okay, relax something, okay? So this way we can use the recursion in our programming to find the sum of natural numbers. 50. Finding Factorial of a number using recursion in R: So in this lecture we'll do another Hansen. And in this lecture we will try to find the factorial using recursion. So, you know what is factorial of a number? You must be nice, right? You must have studied this in your math classes died. So high-school maths, you must be knowing what is factorial. Let me tell you what is factorial. Factorial, our phone number is the product of all the integers from one to that Number supports, if I say factorial of two, factor two will be the product of one to two. Product of numbers 1-2. So product of factorial of two will be one into two, right? If I say factorial of three is one into two into three. Similarly, if I say factorial of seven and we denote the factor like this. So seven factorial will be one from a product of numbers 1-7. So it will be one into two into three into 45 and 6.7, which will come around 5,040 as a product of the number. So factorial for number is starting from one till that number will multiply each number numbers and get the product of all the numbers. Okay? So 1287, if I put y factorial eight here, it will be product of numbers 1-8. And this will be something else, right? So let's find what is the factorial of it. So let me run this program and let me put this in print. Let me run this. C factorial of eight is 43 20, so factorial of it will be 43 20. So this is how we can find the factorial of a number using the recursion. So let me explain this function, which we have written to find the factorial of a number. So here I'm writing a function recursive factorial. And this will take the number as an input. And here I'm checking if n is less than or equal to one, I'll return simply that number one, okay? Then factor of zero is one also, okay? So if it is one or zero will return the output as one. And if number is greater than one will go into the else part. And here, what I'm doing, I'm using, I'm using N into, N into, and then I'm calling the function again the same function. This is called the concentrate to be. The function will call itself inside the function. The function will call itself inside the function, and that is called recursion. So inside this record factorial function, we'll call that the factorial function. And it will take you n minus one. It means for support factorial of it. So it into, then it will go to the fonts and recap factorial. And it will take the input as eight minus 17. So it will take the input as seven, and we'll call that record factorial. Then again, the function will come here. Then again it will come here and it will be eight into seven, into record factorial of six. And like that it will be a recurring, right? This is called recurse on, it will keep on calling itself inside the function pill. The number becomes zero. Alright? So it will start 8-7 to six. And then it will come like what will come one minus 10. And then factorial of n minus one. So factorial of zero will become one and we'll get them product of eight into seven into six into five into four into three into one. And this is how the factorial works. Let me change this to five and it will come around 120. See, the factorial of five is 120, okay? So this is how factorial function walks in our programming using the recursion. So the constant is supported and I'm writing this function regard factorial. And inside this function itself I'm calling the function itself. So inside the recursive factorial function, I'm calling that record factorial. And this is called recursion when the function calls itself inside is called the Carson. When the function calls itself, it calls the car son. Okay? So this is how recursion works in our programming. We have seen two examples. One is finding the factorial of a function using recursion. And in the previous example, we have seen how to find the sum of natural numbers using recursion, where we build that ecosystem to find the sum of natural numbers. And here we have found the factorial of a number using recursion. 51. Sample Data from a Population: Hello and welcome back. In this lecture, we are going to learn about sampling of a population from a dataset in R programming. So that is very important because when we have a machine learning problem or data science problem or the flooding problem, we have a huge set of data, right? And we want to get some insight from the data, or we want a sample of data. Suppose we have a most city population and we want to analyze how many Parsons having diabetes. So we one to get a sample of data, analyze that and based on that, we can create a model and then we can apply that model to a larger population, right? So taking a small population or a sample population from a large dataset is called sampling of a population. In, in a general term, sampling a Beta is very important. And to know this, we should be knowing what is, our functions are simple, how to create vectors. And so we know all those things. So we, what we do, we will see a simple example, how we can do sampling. Okay? So to handle sampling and to provide sampling support to the sampling, or has an inbuilt function called sample. So our sample function, this simple SAM PLE sample, this sample function, what it will do, it will simply sample population if we give sample and we give any integer number as the input. So what it will do, it will sample one to 20 numbers, means it will create a population, sample population of one to 20. So if I run this, see here, let me clear this so that we can see the output correctly. So if I run this sample 20, see what it will do. It will create some numbers in our population of numbers 1-20. And it is not against, it did not earn a certain random, okay? So it's just not arranged in increasing or decreasing order. Just the numbers 1-20, it will generate. So 123, all numbers 1-20. Okay? This is the way we can create a sample up numbers 1-20. That's a simple example. Next thing is what I'm going to do. I'll create a vector 1-15 numbers. Suppose this is the, this is our dataset which is containing numbers 1-21 to 15 here, I'm e.g. so one to 20, this is our x vector where we are keeping the numbers 1-20. So this is the population. This x will be known as a population because this is the entire dataset that we have. So X is the population. And from this population of x, I want to sample outwardly five elements, or five. I want to sample file. I want to get the five. These are the marks. Are these, these are the number of preprint people. I want the numbers of only five people. I want to sample five elements. How we can do that? We can use the sample function. We can provide the population here, x and then comma five, how many elements you want to sample from the lot dataset from X. So sample x comma five will give us the five random numbers from this large population data and it will create a sample for us. So let me run this rushed and then sample exon five. See here. Now we are getting 481,410.12 from the population data-set, right? Populism dataset is this one, okay? One to 15. So from that, we are getting the random for 81,410.12. This is the our sample dataset. If I run it again, it will give us different five numbers. See here, now the sample is changed and so it is randomly pick five numbers from this x vector or x population, and it will create a sample of data. So this is how our sample function works. Next thing, we can give replace equal to true. So for this one, replace equal to two. It will generate the same numbers. If I run it again, it is going to give us the different five numbers here replaced is not giving us the war what it does. But If we use our sample and we'll provide acts as a population, and we'll simply use replace equal to true. What will happen? Let's see. It will replace the element c. Now 51302, 74, 414-151-2312. See, all the numbers are not the right one to 15. Some numbers have been replaced like 131 time, but five is also one time, 14 is repeated thrice, 12 is repeated twice. Hey, there. In the similar way, some of the numbers will be replaced with some other numbers like some supporting one, support. There is no one in this, but our original dataset is having one to 15. But here not all numbers, 1-15 here, right? Most of the numbers are replaced. So what replaced equal to true will do, it will keep on replacing the numbers from the dataset. So from the one-two-three percent, it will take some of the data, it will repeat some of the data, and it will replace some of the numbers. So some of the numbers has been replaced with the sum of the other numbers like one. Suppose one is repeated, one has been replaced by four, right? In the similar way, 14 has replaced some other number like six or something. Because sex is missing here. It is missing here. So those 6.8 which are missing, which has been replaced by the other numbers in the population dataset. That is 14412, right? So what it will do, it will replace the number internally and cool gender into our sample population for us. So it will simply keep on replacing the data with one another. Okay? Then suppose we have event of head and tail tossing a coin. So we have two events, right? Either we get head, Are we good? Suppose we have this sample dataset and reciprocal event of H and T head and tails. And we want it to be sampled. It will give the size of the sample for print and replace equal to two. What it will do, it will simply run this, see what it will do. It will simply create a sample with t and edge, head and tails. And it will repeat head and tells multiple times. Because we have given the sample sizes purple. And so it will create the edge DHHS, an entity. This is quite random. They are sequencing that if you run it again, you may get the another sequence of H and T, and it will be upside scripting. So this way we can create a sample of head and tail events. And we can give the numbers here. If I give five, it will be done. It will give us the TTT. See here in this temporal trends we have given only five tails. Tails, tails are coming. No, there is no head. If I make it, suppose six. Now we're going to head, tail, tail, tail, tail and head. So it is quite random. So this way we can use the sample function in R programming to get the sample from a large population, like we did here. We have taken this population x, which is containing numbers one to nine, and we have taken only sample five elements from that dataset. And we have created our sample of five elements. So this is how we can use sample function to get the sample from a population data set, okay? 52. Program to check Prime Numbers: Hello and welcome back. So in this lecture we're going to learn about how to check whether a number is prime or not. So these are the problems that we might be facing in our competitive coding interviews, where the interviewer may ask you to write a program to find sum of n numbers or whether to take a number is prime or not, or some of even numbers or find even numbers. How to check even number, how to take prime number. So these are the problems, are quite often asking the comprehensive exams and come to, to programming tests. So it's better to know how to implement this in our programming as well. If you are an aspiring data scientists and machine learning and AI engineers. So it is good to know the basics, right? So in the series of lectures, we are exploiting these things, how to, how we can write a program in our programming to find a prime number, particularly in this lecture. Okay? So this program is to check if the number is prime or not. And the number we will take as a jack inputs. So we'll ask user to enter the input, and once the user enters the input number, we'll take that input number and we will check whether that number is prime or not. And for that, we will use the for loop. So if you know how to you use if and else, and for loop, you will be pretty good to do this program. Okay? And let me tell you what is prime number? So prime number is a positive integer greater than one, which has no other factors except one and the number itself. So what does it mean? Suppose we have a number for number four, we can write in the form of two into two. So it has two factors. Two into, two into two is four, right? So this is not a prime number. Suppose we have a number. Suppose we have number six. Number six, we can write in a form two into T3. So it has two factors, 2.3. Similarly, we have number. Suppose we have number five. Number five, we cannot write in a factor, right? We cannot try number as two into something like two into, we can't wait. We can do two into 2.5, but that is not correct. We should have the integer factors straight. This way it is not possible, right? So the numbers like five, which has one, factor, one and itself. So we can write five in a form of one into five. Similarly, we like we have 77 also we can write in the form of wanting to seven Wanli. We can write it in a somewhat with the help of some other numbers like we have written six to three, we cannot, we cannot find a number which can divide. 77 can be divided by one, only 1.7 itself. So that's why the prime numbers are the numbers, positive integer numbers greater than one, which can be divided by one. Number itself, which has no other factors except one and the number itself. So the prime numbers or two, then three, then five, then seven, then LeBron, and then 13, then 17, and so on. So these are the prime numbers, C7. We cannot divide it by any other number except one and itself, 13 also, 11 also. So these are the prime number. So now we have the understanding of what are the prime numbers. Let's proceed to solve this problem. So first thing, how to take input from the user in R. So we can use the read line function to get input from the user. So you should be, you should know the readline function, which has a prompt Azure argument. So readline function will take prompt, so it will prompt the user to. And whatever you write here, it will be displayed on the console. And it will ask you to do whatever you write here to ask you to do suppose here we are entering, we're writing, please enter a number. So the prompt will take the number entered by the user is we'll pass it to read line function. And here we can convert that user input two integers. So what we are, we're storing the user entered number in. 0 and n variables. So as dot integer, it will convert to integer from whatever we get from this tree line function through this prompt. Okay? So if I run this here, if I run this, it will ask us to please enter a number. Okay? Then let me clear this out. Okay? And then we are setting a flag. It's called to zero. Initially. We'll, we'll see why we are using this flag equal to zero, and we'll also see why it is coming. This is sum, because we have entered the numbers. It is showing us the same number. This is sum, ah, edit that we'll see. Okay, so we're setting the flag equal to zero. And then we add giving. Like, as I said, the prime numbers are always greater than one, right? Start with two to 357-11-1317 like that. So it is always greater than one. So first thing what we'll check, we'll check if the number is greater than one. Then we'll go inside the loop. And if number is not greater than one, then it's definitely not a prime number, right? So for that flag will be zero. And for flag zero, what we are giving, for flag zero, we are given not a prime number. So if you enter number in minus, minus two, minus three, minus five, so that those will come here. Pulse and it will give, it will. So the message that you're entered number is not a prime number, okay? And what if number is greater than one? Support 235678, all these numbers. So what we'll do, we'll check for the factors. And before checking for the factors, what we'll do, we'll set the flag to one. Okay? One means number is prime number, okay? So initially we have set it to zero. Now as to nudge will enter inside this if, if loop, IF function, if statement, what we'll do if number is greater than one, will set the flag equal to one initially. And then what blue, we'll create a for loop. And what will be the for loop? For loop is for I in two, because the prime numbers start with 22 to n minus one. And minus one means suppose we are entering five, so two to four. So I in two to 4 min, it will take four to T for these three numbers. And what it will take for the factor, whether the number is a number is being divided by the two or three or four, it can be divided by 23.4 or not. And then if n person, this person did, I miss, it will check for the factor if the entered number is divided by, suppose we are entering five. So it will check whether the segment divided by two or not. And then it will check divided by three or not. Then it will take to either four or not equal to, equal to zero. And then we'll set the flag equal to zero. And if it is divided by the 234, it will set the flag equal to zero and it will come out of the for statement. It will break the for loop and come out of that. And okay, and if n equal to, equal to two, it will set the flag equal to one. So what it will do here, it will check whether the number, suppose we are entering six. So six is divided by two, right? So it will come as a equal to zero. So it will set the flag zero and it will come out of the loop. And it will flag. Flag zero will come in the else part, and it will, so it's not a prime number. Suppose we are entering five. So five is divided by two. No, it will come out and flag will be one year. So flag one means five is a prime number, right? Similarly supported, we are entering eight, so eight is divided by, it will take two to seven to first divide by two. Divide over to just flag is zero. It will come out of the loop and flag zero for it is not a prime number like that. It will check and it will give us the result. Support you, we are entering 11, so it will take N divided by 211/2. No, it will come out of the loop supported. We are entering cysteine. Cysteine divided by 2/2. Yes, Zero flag 016 divided by like that. Okay? So it will be coming out of the loop by saving, setting the flag equal to zero. And all numbers will not be a prime number. And suppose we are entering 1717/2. No. So what are two will happen, it will come out of the loop and it will set the Flag equal to one. Flag equal to one means it's a prime number. And if the number you were entering to, then directly prime numbers, right? And for one, we are coming in the else part, greater than one, we are coming in the else part. Okay? So let me run this whole source. What is the problem here? Okay, let me run it again. Now, the console is asking us to enter a number. Suppose I'll enter a number one. Sorry. Suppose I enter a number one here and hit Enter what will happen? One is not a prime number. Why one is not a prime number? It will come here and it will check whether n is greater than 100 or so. Yan is greater than one or not. So this is seriously not, it will not come inside this loop and flag will be zero for this one because it is not coming inside this if loop because F is one and it will come here, flag equal to zero. So now it will come here. And it will check flag equal to one node. So it will come in the else part and it will study Saudi. It will come into this else part. And also one is not a prime number like this. Okay? Let me run this again. And if I enter two, sorry, if I entered two here, what will happen? It into c2 is a prime number. Why two is prime number? It will come here and is greater than one, right? Two is greater than one. Then flag, it will set one, then it will come here to divide it by, to divide it by two to one, right? 2/2. So C is 2/2 if celebrate. So it will come out of this loop, right? And flag will be one. So two is a prime number. If I put three, sorry. If I run this again and put number three, then it received three is a prime number. Why? Because it will come inside this if statement flag. It will set the flag to 1.4. Flag one it will be prime number and then it will come inside this for-loop I in 222, right? So 3/2, no. So it will break this statement and flag will be one, right? So and will be the prime number. So like that, if you enter support the length or 17, Saudi support cell enters 17 here. 17 is a prime number, why? It will come here and 17 is greater than one. It will come here, set the flag to one, then it will go inside this for loop for i into 21617/2. Yes. 17 is not diverted by two, so it will come out of this loop and flag will remain 1.4. Flag one, the number is prime number and that's why those 17 is a prime number. So this is the way we can write a simple program to find prime numbers in our programming. 53. Program to check EVEN or ODD: In this lecture, we are going to write a program in our programming to find if the entered number is odd or even. So, we'll take the input from the user and we ask the user to enter a number. And based on the user's input, will see the user input number is an odd number or an even number. Right? So what is an odd number or even number? So number which are divided by 0/0 without the remainder. It's called even number. Suppose we have a number x and if we divide the number by two and we get zero as reminder, then it is an even number. And if the number is divided by two and it is giving us some remainder, then even number. So simply if a number is divided by two and the remainder is zero, so even number and divide it by two. And it is giving some demander, it is odd number, right? So let's take the program. So here I have written a program where I'm taking the input as integer. So I'm taking the input from the user ID as integer, and I'm asking user to enter the number, please enter a number. So readline function will use to take the user input and prompt we're giving. Please enter a number. Then we're converting to a yeoja, Ads dot integer. Okay? So whatever that is, a integer number, right? And then we are simply checking whether this number n is divided by two. And if the remainder is zero, so n divided by two. If Amanda is equal to zero, we'll say n is an even number. And if we get remainder other than zero or number, right? It's an odd number. So simple thing. If n is divided by two and the remainder is zero, then if they even number, and if remainder is other than zero, it's odd number, right? So I hope you got the idea. So let me tell you what are the even numbers and odd numbers. So 24681012. All these numbers are even number. And our numbers are like 35791113. All these numbers. So not only this, it will continue till Okay. So the number which f lady waited way too with Gino demanded. Demanded is called even number. An odd number if it gives that claim under other than zero. So let's run this program and see the output. So let me clear this first. Enter a number. So suppose I'll enter number 45. So the output voltage coming at 45 with them, or number by 45 is odd number because if we divide 45, we will get one, either demand or two into 20 to 44.1 we get as a reminder, so reminder is other than zero. So it will come into this else part. And it really was the method that 45 is an odd number. And suppose I run it again, and if I enter a number 12, sorry, I'll enter the number 12, then 12 is an even number. Suppose I run it again. And if I enter a number, here, has to give us even number straight. And if I run it again, and if I get five, and get five is an odd number. So this way we can identify whether a number is odd or even simple logic and divided by two is equal to zero. If it gets from zero. As a reminder, we say what number, otherwise it's sort numbers. So I hope this simple programs will help you understand how programming works and will your logic. And these are pretty popular questions in interviews also, especially for a freezer. And if we had a new graduate and looking for the job, these questions are pretty common in College's campus placement, okay. 54. Program to check Positive Negative or ZERO: In this lecture, we are going to write a program in where we will check whether a number is negative, positive, or zero. So it is going to be a simple test where we'll find whether a number is negative, positive, or zero. So for that, the same thing we will do what we have done Linda previous lecture, but slightly different here. The number can be given a double lumbar. Okay, So take the input as double and we'll use the read line function and the same prompt, please enter a number. Then we'll check whether the entered number is greater than zero. If it is greater than zero, will give the message. N is a positive number. And if it is equal to zero, we will give like it's a zero. And in this only if it is other than zero. Then, first it will check this. If n is greater than zero, then postrenal, alright? Then it will come into else part in L3 are using a nested if-else. If number is not greater than zero, it will come to this else part. And then in this else part, we are checking if number is equal to zero will give number is zero. Else, if number is not zero and greater than zero, less than zero, then what we'll do, we'll want to type this else part. That means it will be a negative number, right? If number is less than zero. If number is not greater than zero, if number is not greater than zero, right? Mantissa, it can be zero or less than zero, right? So if it is zero, we'll get zero. If it is less than zero, we'll come to the else part and put the message like number is negative number it simple check. Let's run this. Let me clear this out. Okay? Okay, here. Okay, so let's run this. So let me enter a number support the Atlanta. Oh, minus five. Minus five is a number. I suppose I'll run it again and enter 45. Sorry. I'll enter 45-year. Four to five is a portion or suppose I'll run it again, handy, I will put zero. J, the Saudi Tito number is zero. Okay? And then suppose I run it again and give double number like -78.5. So -7,578.5 is a negative number, right? In the similar way, if I put minus eight has a negative. So this is the simple program to find where the number is positive, negative, or zero. 55. Program to Check Leap Year or NOT: Hello and welcome back. In this lecture, we are going to write a program to find where that ear is, Bolivia or not. So what is leaf here? Leave here. You will get an extra day, like three to six days in a year. 376 days here suddenly appeared, right? So how we decide whether the yard wouldn't be leap year or not. So there is a simple formula. If the ear is divided by four and the remainder is zero, it could be a Libya, but that is not the one liter case, right? If the ear is divided by four, this is the modulus function. If you divide it by four and giving any vendor as zero, then that could be Libya. But we have to check another condition, whether the number is also equally divided by hundred. Also, if you divided the United Way study report and giving them into at zero, it could be a leap year, but we have to check whether that is divided by hundred and giving you a reminder as 0/100. Also giving the remainder is zero. Again, it could be a live here, but again, we have to check a condition where the data yet is divided by 400 and giving a reminder as zero, if that year is equally divided by 400 also, then that will be thus your shot leap year. And if it is not, then it will be not as Libya. So the first condition we need to check is whether the edit divided by four, then we have to check whether they are divided right forward and remainder as zero fully divided, right, for your model, that means modulus 40. Then we have project yet more or less 100, it's around, so come zero and year modulus 400. So they also come zero. If that he has his, uh, Livia. Livia. Okay. And if and here also, this is not a divided by 100, then it is not live yet. Also will come out of this panel, go to the else part. And if it is not divided by four, then certainly it is not a leap year. Okay? So here also, if the year is not directed right for this, it is not Olympiad. Okay? So let's run this program and support and enter 2020. 2020 is the preferred way because it is divided by four. It is also divided by hundred. And it will be divided right? 400 as well, right? Well, it really right around 400 2020s. Knew I did write for yes. And if it is not divided y, if it is being divided by hundred and just giving you a reminder as zero, then we have Project 400. But here it is, giving us the demanders something, right? So that's why the remainder is not zero here. So it will come to the else part and it will put you at 20:20 is Libya. So if it is more or less is zero, then we have to check divided by 400 or not. If modulus is not equal to zero, then it will come to this part and it will give us that 2020s Libya supports a frame around it again. And if I gave 1520500, yeah, rather suddenly for yet or not. So C yet 2051, that is divided by four. So it will come here. It will take and divide it by 100 or not. So it is divided by 100 fully. Then it will come to the divided this loop and it will check whether this is divided by 400 or not. So see 2500/400, what it will give us the remainder. Somebody render it will not required to zero, so it will come here and it will come to the else part because the remainder is not equal to zero. And this part, it will print 2,500 is not a Libya. So let's enter and see the result. C, 2,500 is not a leaf here, right? Similarly, if I put 19 1,900, not only fear like 2,300.2300, not alive yet, right? Like wait, I need again Tao Qian. 12. If suddenly we are why it is live here, because it is divided by four, then it will come to the this and see whether this year total annual rate divided by 100 and giving us a remainder of zero. No, it is not giving us the remainder zero. So we need to check again. And it will clearly come to the health part and it will print Libyan. Okay. So this way we can write a program to find whether a particular deities sleep yet or not. 56. Program for Multiplication Table: Hello and welcome back. So in this lecture we are going to write a simple program to print multiplication table, right? To like a multiplication table is a table for like. Suppose multiplication table up to will be two, then two into 24, then 2.236, doing 2482 into pipe like that. Okay? So simple multiplication table we're going to print. And for that we are going to write a program. So I've already written the program to save our time. Well, so first we'll take the user input as integer, and we'll prompt, please enter a number. As soon as you will enter a number. We'll take that number. And what we'll do, we'll run a for loop. What i in one to ten, because the multiplication table, we want to end it. 14, up to ten numbers Valley, right? So, and then we'll print multiplication, table up the number n. And what we'll do, we'll simply multiply N into i. So we'll first take them, or somebody will take number three to three to one. The next time the board will come, I will be one. So 3.22 and 3.23 up to ten. Like that. It will print the multiplication table and to just, this is for the output. So it will be three into one equal to 33 into two. Sex like that. Okay, so let's run this and suppose I'll enter three here. So what will be the output to see three into 13 and I is one year, okay, so three into one equal to 33 into one equal to three, then three into 26 up to ten. So if you want to increase this number, you can put support elbow here. And if I run it again, and if I bet for three years, so see here, this will go up to pre-prints. So this way you can generate the multiplication table or for any number n, for any number of primes up to obtain proper pain, a proton it whatever you want. So suppose I'll run it again and I'll keep the number 20, study. Number 25-year Atlanta 25. Let me clear it. I'll enter a number 25-year. I want to generate the multiplication table for 25. 25 one-to-one. If I wanted to pick t, 25 to three is equal to 75.25 into ten, please. So this way we can generate the multiplication table in our program. 57. What are Missing Values and Types of Missing Values: Hello and welcome back. In this lecture we are going to learn very important concept that is called imputing missing data in. So whenever we're doing any data analysis project, machine learning project, or even data science, artificial intelligence project or deep learning project. Whatever project you're doing related to the data, we need to work on the data. And one of the challenging problem, or the biggest problem is now missing values in the data. So what does I mean by missing values or missing data? So when it will support how we gather data, we gather data using surveys or we asked people to filling in the forms. And while people are surfing Internet, we collect data, while people are browsing, we collect data will be polar, are going for shopping. We collect data through poems by asking for feedback, all those things. So there are various ways of collecting data, right? And in those various ways of collecting data, sometimes the person who is giving us data or providing us data is reluctant in providing all the data that we ask Bob, right? And when he hesitates and Gibbs incorrect data or he does not keep data for some of the forms values, then the data will be missing, right? And that goes to the like, not completing the data. And when we try to analyze that data, we may face problems because many of the data columns or rows will be missing. Suppose the simple example is when you are going to shopping mall and they are, People are asking for you to fill out a feedback form where they will be asking like your family details, your mobile number, your email ID, and your mother named Father name, your street name, where do you leave? What are your preferences? How much you solve? In a month? You're like your salary details all these data. So most of us will not be ready to give all these data like our salary data or even home address or the mobile number. So we tend to hide and not are not provide these sensitive data or our family details like whether we are married or unmarried or we have a cage or not. These data we don't provide. So when the company collect the data from thousands of people in the mall, surveying them, they, when they collect the data, when they go in, they're going to analyze the data. They will find that most of the people have not answered like how much they own and what are the their marital status, how many kids they have when they not provide this kind of break up, they will not able to categorize them. Well, like they might support some people. They married and they have not filled the column, married or unmarried. So they will be either categorize them wrongly or they will not able to categorize them. And this is the big problem in data analysis and machine learning or data science projects. So to tackle this problem, there are ways to fill the missing values. So support you. If you have a numeric data, support, you have the age and weight data you have collected from thousands of people and some people where it is missing or aged missing. So what we can do in the case of numerical data, we can simply take mean of the available dataset. And we can fill the mean value or average value in the missing values. Whatever the values are missing will fill with the average value or mean value. And that will complete the data and we can get a good analysis on that. So that is a simple way to impute missing data in R or any data science project. Okay, So that in the case of missing values, numeric, what if the missing values are not numeric? What we will do. So now what we have understood, we have understood what are the missing values and what other Regions for them. So let's understand types of missing values to the types of missing values. We have classified them into the three types. One is M, CAR, Then MAR, or the last one is an MAR. So what is this MC, AR, M cat. So MCAR, Stanford missing completely at random, missing C4, completely A4 at arpa random missing completely at random. And is the reddish type of missing values when there is no cost to missingness. In other words, the missing values are unrelated to any features just as the name suggest. This lake. They just a very rare case where like while putting the data into the Excel CDART database, you missed the data. Or when we are sampling a data or a surveying some people, some people, randomly, you miss some data. So this is quite a rare case where we get the MCAR. Then the next column, the MAR data, MAR means missing at random. This is quite common and it implies that the values which are missing can be completely explained by the data we already have. E.g. there may be a case that males are likely to feel a survey related to depression regardless of how depressed they are. Like. Suppose some people are depressed and we're serving those people. So most of the people, they hide their mental status, right? They are very reluctant about sharing their mental, mental health, right? So they will hide that they have some mental issues. And this will tend to the getting the wrong information from the people. So these missing values or MAR, right, missing at random, actually comes from the making an assumption about the data. And there is no way to prove whether the missing data values up MAR. Whenever missing values are categorized as MAR or MCA or two larger number, then they can be safely ignored. So MCAR, MAR, since they are like missing at random, we can ignore them. We can, if you have a data with the filled with this, this kind of missing values, you can simply ignore those values and you can completely go ahead without filling those missing values and you can remove them from the dataset. And that is completely fine. You won't be missing much of that because all numbers are missing by random. That doesn't relate it to any feature or any, any variable that is going to affect much to your overall data outcome. Okay? The last one is an AR, and M stands for not missing at random. And this is important. Why? And CMR. And when the data is not MCAR, MAR, then it will be categorized as not missing at random. And what is not missing at random means that like I said earlier, when we are surveying people at mall, they may hide their spouse name, their kid's name, their address details, their credit card details, or their salary details. So this is not at random, right? People are willingly hiding data they're not providing. And when you, when we analyze the data, will find that people have not provided there marital status or their spouse name, or how many kids they have and how much they aren't. So this data is not missing at random. So this is important. And this is not out of randomness and we may not may or may not know which case the parcel lines in like if a person had not said anything about marital status, whether he is married or not, we will not able to categorize them as a married man or unmarried man. And that will lead to the problem that we are not able to categorize that person in particular category and hence will not able to analyze our data correctly. And if we ignore this NM AR not meeting missing at random data, then suppose analysts or data scientists to ignoring these data are not missing at random. He may end up at the wrong calculations, are wrong predictions and that will to the huge loss in the business. If you are support you, if you are predicting the revenue of your organization and if you ignoring not missing at random data in your dataset, then you may end up with the wrong calculation rock predictions and which will, which may lead to the huge loss, are not targeting your customers correctly. And this will lead to lots of problems for you. So you have to take care of these three kinds of missing values. And you, while enlarging, you must be knowing which are, which type of missing values are missing in your dataset. And if it is not missing at random dataset. And how, how do you know that not missing at random data, missing values are not missing at random. And that you can decide based on the how, how would that missing value is affecting. Okay. So like person is married or not, it is affecting because you're not able to categorize correctly the person. So this way you can identify the implications are and what impact that missing value is going to have on the entire data and outcome that you are going to get from the analysis of the data. So apart from the mean, if we haven't numeric data, you can put other missing values by using mean, mean of that entire dataset. So other than that, we have some R packages which we can use to impute the missing values. And these are the fiber from the popular R packages for imputing missing values. And the first one and very popular these mice MIC. This is the package which can be used to impute the missing values. Then Emilia is another package, missed forest is another, and miss another one. An MI is the fifth one. So these are the five of Popplet R packages for missing values. Or with the help of these packages, we can very much calculate, impute the missing values in our dataset and we will end up with a great analysis of our data. So I hope you got to understand what is imputing missing data in R. What are the missing values, types of missing value and how you can identify the correct type of missing values and how we can deal with the missing values. Okay, so see you in the next lecture. 58. Imputing Missing Values NAs in data set: Welcome back. So in the previous lecture we have covered what are we missing values and what are the types of missing values we have, we have learned about MCAR, MAR and MAR not missing at random. So we have learned what are missing values and what other type of missing values we can be seeing in real life when we do our skill, our data science projects. And we have also seen what are the packages available in our programming with which we can use to impute the missing values. So we have five, a popular R packages, mice, Aemilia, Ms. Water, Act, missed an MI. Okay. So these are the things we have covered in the last lecture. Now, we're going to do our real hands-on and our project where what we do, we will use the inbuilt data set that is available with our package, that is the iris dataset. And in that Iris dataset, what we do, we will include some missing values. So we intentionally put some missing values in the Iris dataset. Then we will try to impute missing values with some data. So we will use the package Ms perished and mice will use the mice and Ms. Perez to do this, okay? So impute the missing values in the dataset. Okay? So for that, we will be using the midst forest and we are going to use the Iris dataset which is in welding are. So the first step is we need to load the data. So to load the inbuilt dataset that is readily available in R, we need to simply call that data. So data is equal to itis. And when we put somebody up data, and when we run this, we will get done somebody up the iris dataset. So see here, this is the data that we get from the summary from this Irish data. So there are many more data, but this is just a simple data summary from the Irish data. Next thing is what we need to do. We will, since we are using the mice package here. So mice idioms missing at random values, right? So let's see missing values in our dataset since this Iris dataset will not have any missing values. So what we'll do, we'll put missing values, will see the missing values in our dataset using broad NA function and this broadening function we can get from the Miss Forester function. Okay, so with this we can use Iris dot miss the missing values, and we can use the broad n a function. And we use the dataset will give the dataset name here, iris and no any 0.01. Okay? And when we do this, it will, what it will do, it will see the missing values in our dataset. So when we try to run this chunk, we'll get into why we are getting this error, because this broadening function is not available because this belongs to the luxury the package miss polished. So what we'll do, we will try to install the MS perished and mice. So first let me, since we notification is coming here like a package mice to be installed. So just click on Install and senior missing data in RMD, the package mice will be installing. So it will start installing. Can see now the installation process that has been started installing mice. So it will download it, it will take a couple of seconds. Okay? So packets successfully installed. Okay, next thing is we need to install another package called Miss Forester. So we'll just type you miss it. Okay? So just click on Install and the Hmisc package will be installed. Install that package. Also, you can use the command installed packages and you can give them this forest name and it will be installed. Okay, so now the miss water storage also successfully installed. Let's try to run this one. Chunk above here. I just taught myth and religion, brought in a function and annual 0.01. Okay, so let's run this. I don't know why we are getting at it again. So now these misfolded package has been installed correctly. So now we can simply and then I'll run this I just thought Miss brought in. And then I'll run the summary of Irish Dartmouth. And we'll see here. So when you see this summary of Irish data, you will not see any, any values. You'll see that is no any values. But when we use this broad NA and impure some missing values in iris data. And we'll create a new art, missing values. Nadar from IDs. Ids taught me. See, yeah, it, what it is doing, it is doing any adding in sepal length, sepal length and sepal width, petal length, and petal width and is 19 and setosa RNA's proteins. So we have introduced these many 1,819.14. Any values using that brought in a function from the Hmisc package. Next thing is we need to remove the categorical variables. Okay? So let's run this and see here. Okay, so we have removed this species. Species like what's the color virginica. All these things we have removed, the categorical data has been removed from this. And next thing is, we will proceed with implementing mice so that we do in the next lecture. 59. Imputing Missing Values using PMM method: So what we have done in the previous lecture, we have used up brought any function from the myths forest package to introduce 10%, 10% missing values in our dataset. So new dataset we have either start miss, and when we run this 10% missing values will be introduced in our data. And to check that, we will run this and we'll get the summary here. And here, you can see that NA values has been introduced in our data. Okay? Next thing what I'm doing, I'm doing, I'm removing the categorical variables and so that we can focus on the continuous values. Okay? What are the continuous well, like? See, if you see here, these are the continuous thread value and the species like settles, What's the colored Virginia? They are not categorical data, so we'll remove that. So what we have removed that, we have removed select, we have used Irish taught miss, the dataset and what we are connecting, we are creating the subset and we entering the eye, they start missing data-set, that dataset with missing values and we're looking selected, you call two minus c species. It means it will remove the species, which is a categorical data in our dataset. And then we'll get the summary. So when we run this, we get this dataset where the species will be removed from our dataset so that we can concentrate and focus on the continuous values. Okay, the next thing, what we are going to do, we will install the mice brackets. And why we need these mice package, because these mice package is having one function called Md dot pattern. So Md dot pattern function, what it does, it returns a tabular form of missing value presenting each variable in a dataset. Okay, So for that, what we do, we will be installing the install.packages will use to install the package. And then we'll use the library mice. And then we use the Md dot pattern to the missing values present in each part, each variable will see what are the missing value present. So first, we'll install or we have already installed. Then also if you want, you can run this again. If we have not installed. And then after that, after the package isn't installed, you can come to this piece of code and you can run this Md dot pattern and you can put your dataset. So this is the Iris data set which is having the missing values. Okay, so let's run this. And when we run this, see here, what we get. We get here the missing values for it. Each right here, but like sepal width, sepal dot length, petal length, and petal width. So let's understand this table. This what we're getting in this tabular form. Let's understand. So there are 98 or 96 objects and here 96 observations are there, which are having no missing values in the missing values. And then, okay, so this six. 96. This one means no missing values and zero means missing. Well, so 96 variables that are having no missing values and sepal dot width. And here you proceed. Six runnables are having missing value than one variable is also having missing ReLu. So with this we can understand how many missing values. Zero means those variables are having missing values, okay? And this is looking quite bad. And if you want, you can see this as well. Quite overlapping. So see here. Okay, so next thing what we'll do, we'll queue, will create the visual representation of what we're seeing in this tabular form. So for that, what we'll do, we'll install the package. Okay? And after that we'll use the BIM library. And what we'll do, we'll use that. We'll plot the whatever we're getting through the mice package, empty dot pattern. We'll try to plot that. So we'll use this function and EDR function, and we'll use that dataset, iris Dartmouth. And then we'll use color, navy blue, yellow, whatever you want. You can put, suppose I'll put red and yellow and numbers to sort value and then label name. I'll give whatever names are there in either start missed. And why lab really missing data pattern. Okay, so let's run this and see what visualize and we are getting. Now, we are getting this visualization. So see here with these visualization, what we understand is 67% of the values, 67% of the values in the desert with no missing value, 67% data is not having any missing values. There are ten per cent of 13% is having missing values in our petal length and petal width and out. Okay? And we can see this histogram also. Histogram. Okay, Let me just see a histogram here, missing data and Leah, the variables petal, petal width is having around pre-print per cent up missing data, where the length is ten per cent, sepal length is having around 9% or something. Sepal width is having 8% or something of missing data. And here also you can understand, okay? So this way, and we can see the graphical representation of the missing values. Now next thing is the Critical stop. What we are doing here. We are in, we are going to impute some values in the missing values. So for that we'll use the mice function and we'll use the ID start missing data set with missing values. And m phi m is equal to five. What it will do, it will report to the image equal to five if you gave, it will create the file type dataset with missing values. So it will create five imputed datasets and maximum address and will be 50. And method we're going to use PMM. What is this PMF? Pmm is predictive mean matching for numeric values. We are going to use the method PMM, which is predictive mean matching, okay? And then we'll see it for 500. And then we'll see the summary of input at data. So let's run this piece of junk piece of code. And here we are getting some edit. Okay? It is not finding the function mice why? We then need to gain. Okay, so see here now, we are inputting the data with. Missing values. So the missing values will be imputed with some data and the processes going on. You can see here. So because it is 50, I dress on sand, we're selling for 500. So it may take some time. Now. It is done. Right? So these are the number of multiple imputation five. Or imputation method is PMM that we have used, predicts and matrix. You can see sepal length, sepal width 11. This is the confusion matrix, okay? And what the petal width? So this is the confusion matrix we are getting. Now, what we do, we can check the imputed values by using imputed data, and we can use the sepal width for that. So let's duplicate that. Are imported data not found. Why we are getting it at that again. Okay, So I've given the variable name as the dataset name as the time period. And if I run this, see here, these are the values that we have imputed for the sepal width. Okay? So in the same way, we can come down here and Control Alt. I can really add yard and they'll do, I will put some length. And if I run this, we'll get the imputed values for the sepal dot length as well. Okay, so now we can see what are the values we have inserted in our dataset, inputted in our dataset. Now we can get the complete imputed data. Since we have the five setup inputted data, we can get the second object by using imputed comma two. So this will give us the, okay, he had sort of be underscored, imputed hockey and run this. Then we can use summary. Complete data. Dorsi. This is the complete data uploading petition, okay, So this way we can impute the missing values. So suppose you have a need to yell world project where some missing values are there. So you can use this program or this mechanism to impute missing values using like here we have used PMM. You can use the other method as well. And there are few other methods as well that you can use, like we have used PMM predictive mean matching for numeric values. If you have binary variables with two levels, you can use the logistic regression. And for that, the function is log reg. And if we have Bayesian Bali Tom must regression, you can use a fourth factor variables with two or more of a levels. You can use the polymeric method. And if you have like a odd models, you can use the proportional 4D models for order level two or more. So these are the methods dr. These are the method and packages that you can use for inputting the data. So this is all about this project. And in this project we have learnt how to impute data. So in the same way, we can also implement your own project. And you can, you can take this dataset or you can go with any other data. Say try to impute the missing values and try to put the missing values inside the data. First, what we have done here, we have inserted some our data with variables with no values, right? And then we, what we have done, we have used the PMM to impute those values with some data, some relevant data so that we can get the correct output. So I hope you learn how to impute data in programming are for your machine learning and data science projects. So I hope you learned something. 60. Analyzing Data sets using R functions: In this lecture, we're going to analyze dataset. So first thing first, if you are planning to work as a data scientist or machine-learning engineer, or even data visualization data analyst. You have to have to have no. What is data analysis and how you can analyze the data. So the most important part of any data science project data, how you analyze the data. So data analysis is the most important part of any data science machine learning project or even data analysis project. So what I'm going to do, I'm going to use in-built dataset that is readily available with the packet or download. It comes with the the R package, okay? So you need not to download it separately. It will be it is coming with the heart, okay? So it is inbuilt dataset with the r. So we'll use that hand. I'll tell you how we can use the inbuilt functions to analyze the data, to get information about the data. Okay? So what is dataset? Dataset is basically collection of data. And we have most commonly seen datasets are like payable. We use in our databases. Databases what they gave. The most common dataset that we have seen is the. So in our databases like my sequel, our MongoDB or any database, if you see they are basically the calyx enough data, right? If you see MongoDB is a collection of data in terms of key and value pair. If you see my sequel database or RDBMS, relational database management system, they will be keeping the data in the form of rows and columns. And rows and columns will be keeping the data, right? So most common collection of data is table, okay? And we keep data in the XML format as well, in the JSON format as well. But the most common thing is table, okay? So you can, whatever you have seen in the Walmart table that is basically keeping the data with it, right? So we're going to use empty cars. Empty cars is an inbuilt dataset in R. And we're going to analyze this one. So empty cars is the motor trend cars dataset that is inbuilt in R and it is retrieved from the 1970s for Motor Trend US makin, okay? So this data is retrieved from this 1970 Ford Motor Trend US matching, okay? So first thing is when you Suppose we have this inbuilt data set that is empty cars and we want to load this data. So what we can do, we can simply write the dataset name. And when we run this, we will get the dataset. So this is the dataset that we have. Okay? And when you keep you out, there are more columns, right? So these are the rows. And these are the rows are different. God names, okay? And for each card there are several variables like MPG, cylinder displacement that we wait. Okay? So all these data that we have with the empty car, right? So it is having 11 columns and 32 roads. That means it contains the 32 cars details with 11 columns. 11 columns are 11 different variables for each card. Okay? So this is how simple type dataset name and you'll get the older information, although dataset all the rows and column of the dataset. Okay? Next thing is, suppose we have this in. We want to get the information, more information about the dataset. So how this dataset is at an angle from where we get that dataset. So we can simply put the question mark in front of the dataset name. And when we run this, we get the information about the dataset. And this dataset information is coming here. So when we are done. This one. Question mark, question mark empty cars will get this information empty cars dataset. And this is saying Motor Trend Car road test dataset. And this is coming from the R documentation. Okay. And so it is giving you the total description. And total uses a Brita, how this data has been formatted. Okay, so this is motor trend cars protest and data was directed from the 1970s for Motor Trend use. Makin comprises for fuel concepts and antenna aspects of automobile design and performance for 32 automobiles, 32 cars in 1973-1974 models. Okay. And then it is giving the format like mpg means miles per gallon. Cylinder means number of cylinders displacement, HP, horsepower, draft, axial ratio weight than 141 by four mile part-time time. Reassess the engine like VSEPR in general, straight engine. So we as before, we save and if it is having zero, it is received and one-fourth straight, then am for transmission automatic if an automatic emperor manual or automatic and manual. And good number of years and carved number of Cadbury test. Okay, so the other information we are getting for this data source, okay. Now, come back to the analysis part. So you just put question mark in front of the dataset name and you will get all the information about the input dataset. Now, we want to get the diamond cells and the variable name. Suppose we have the diamonds and slake, rows and columns, and these are the variable's name. So suppose how I want the one loop variable names. So how can I get that? So you can use. So first thing we need to do is we need to assign this dataset to some variable. And for that, what I'm doing, I'm creating a variable dataset underscore cars, and I'm assigning empty cars. So empty cars, dataset cars will represent the empty cars. So all the values to our empty cars will be here in the dataset cars and this we can use in our program further. So if I use dim and I will pass the dataset variable, this one dataset underscore cars. I'll get the diamond sum up the dataset. So if I use names and part of the dataset name dataset, or I'll get the variable names in the dataset or let me run this chunk. See here. Now it is giving the diamonds and 32 roles and 11 columns that we can verify from here, like 32 rows and n columns. Okay? So that is the diamonds and up the dataset. And then when we use names, we get the variable names. So you see here, we're getting the variable names. Mpg cylinder is SP dragged with all of those things. So 32 by 11, 32 rows and 11 columns are the dimensions of the dataset and these are the variable's name. Next thing is I want to extract the wrong name of the first column. So I want to find the row names of the columns. I can use row names. And I can pass the dataset dataset variable. So rho name and I'll pass that dataset and escort cards. This will fetch me all the guards name first, column names say C or D, or Delta T. How many? These are the 32 datasets variables. Sorry, these are the 32, 32 cars that has been used in the dataset that are available in that dataset. So the flush column, okay? And then if we use the dark, if I want to get the only one valuable information, like I want to get the miles per gallon from the dataset. I can use this dollar sign dataset name then dollar sign MPG. I'll get the mpg variable values. Okay? So this way we can get the values of a particular variable. So in case here, value of mpg, I can, I'm getting the values of mpg if I use EM here and you get 0.1 because that is automatic and manual. So let's see here, 1000, like this. Okay, so zero for automatic, one for manual. So this way we can use variable names here if I use MPG and get the next thing is if I want to sort this, this MPG is coming like this, okay? If I want to sort these in order, I can use the sort function on this. And I can sort data set and the scorecard dollar mpg. And it will sort the values of this mpg variable c, an artist coming now, sorted form, increasing order. Okay? So this is how we can sort them. I lose a valuable. Next thing is now I want to analyze the data set. So I can use somebody and I can give the variable name for the dataset and I'll get the summary of the data. See here that beautiful summary of the data like miles per gallon. What is the mean? What is the first quartile values? What is the median? What is the mean? What is the third quartile? What is the maximum value for each variable? We are getting these six values, right? Median first, like min, first quartile, median mean, and then minimum mean, median, maximum values. And the first quartile and third quartile for each of these variables, we are getting this information. So this will give you that summary of the data. And these are the things we will learn in the coming lectures. How to get mean, what is median, what is first quartile and third quartile? So this is how we can get information and we can analyze the dataset in R. We can use our dataset name to get the dataset. We can use dollar sign to get the information about the dataset and we'll dataset. We can use the lm function to get the dimension of the dataset. We can use names, function to get the name of the variables. We can use row names to get the drop a row, first column, each row in the first column, okay? Each row's values in the first column, then we can use this dollar sign to get the variable values for the data set. And then we can use the sort function to sort the variable values, and then we can use the summary to get the summary of the data. Okay, So this is how we can analyze that data set in R. 61. Data Manipulation Using dplyr package: Hello and welcome back. In this lecture, we are going to learn about data manipulation in R. So we are going to learn about data manipulation. And for this we're going to use the ggplot package. So duplicate packet is very important when you want to do data manipulation in. And I'm using this RMD file. I've already written the code so that we can just go ahead and we can learn about the basic things of the player and we can do some data manipulation. The first thing, if you want to use the player, you need to install the package right? In our whenever you want to use the inbuilt, we want to use some functionality. We need to get the package installed on our R and then we can proceed further. If you want to work with the plan, you have, you need to install that. You can either install the deep layer itself, one lady player, or if you want to install the whole thing that comes with the tidy verse, you can go with the tidyverse. So if you install tidy verse, by default, deep layer will be coming in that the plan will be inside this tidyverse. And if you don't want to install the whole thing inside these Tidyverse, you can go alone with the d plus. So you can run either this or this. Any of these. You can run. So you can go with that. You can go with the tidy verse or you can go with the deep layer, okay? So you build on your requirement. You can go with that. I will suggest you installed the tidyverse. So I have installed these two packages, so I will not run these p sub chunk of code, this piece of code, I will not running it because I have already installed them. So what I will do, I will show you the next thing. Okay, so let me clear this. So the first thing, what we can do with the data manipulation in art, using this, the player is we can use the filter we can use select. We can use group by all those things we can do using the plot in R. Okay, so what I'll do, I'll use, I'll also show you how we can use the filter in R using the blur libraries. So for this, what I'm going to use, I'm going to use the inbuilt dataset that is called Star Wars. Star Wars is an inbuilt dataset available in art. So what I'll use, I'll use Star Wars and then I'll use this pipe operator. And what I'll do, I'll use the filter function and I'll give filter species is called two triads. So what it will do, it will fetch all the species which are equal to draw it. So it will go to the Star Wars dataset. It will face the Star Wars dataset. And inside that it will go and apply the filter and it will fail. It will filter the data set based on this species. Okay? So this is how it will work. So species drag, it will apply the filter. So let me, I just run this code. So I'm just click on here and see. Now we are getting the recharge. Okay. So see your name, heightened, marsh hair color, skin color, eye color, fix all those things we are getting. Okay. So this we are getting for the species. Try it. Okay. See here, if you see the Species column, everything is dry. It only, right? So it is showing data only for dried because we have applied the filter dried here. So it will show us third data only for species that so this way, we can apply the filter, we can use the dataset name Via and then pipe operator and then filter function and its species. The column name is species specific equal to draw it. So it will, so all the species, the deleted data to us, okay. So this is how the filter works in the plant. Okay, next thing is we are going to use Select. So select how it will work. Again, the same thing. We'll use. This star was dataset and then pipe operator, and then we'll use Select, select, and then we need to keep the name ends with color, so it will fetch all the names ending with color. So let me run this. See here. Now what it is doing. It is fetching all the columns. All the columns ending with color, hair color, skin color, and eye color. So it is showing only the columns which are ending with color, right? So Star Wars select name, ends with color. So it will show you the Wanli George columns which are ending with color. So underscore, underscore class. So it is showing only those data. If you look at the earlier thing. Here. There are many columns like height, mass. And then we have ER, sex, gender. But it is not showing us all these columns, but it is showing only the columns which are ending with Carlos, which are ending with color. Okay? So this is the way to use the select. In. The next thing is we're going to learn about mutate. So what do mutate will do? It will create, if you want to add some new variables or variable value in the dataset, you can use the mutate function, okay? So mutate to add, to add something new, okay? Add something new, a new variable. If you want to add, you can use the use the mutate function. So how we can do that, we can simply give the Star Wars, then pipe operator and then mutate function, and then we need to give name. And then here you can define the new variable or the new value that you want to provide. So here, what we are given, we are given name and name. Here we are introducing new variable BMI and v here we're giving them How BMI should be calculated. So if you don't want to give you can hard code the value. But here what we are doing, we are using the mass divided by height by 100 power to, okay? So this is the formula we are using for calculating the BMI body mass index, okay? So height and the weight we are using to calculate the BMI. And then we are passing this BMI to select function. So we are using pipe operator twice here. So what it will do, it will first calculate this BMI here, and then this BMI will be, this BMI will be passed to this select function right here. See your BMI is coming here from there. So whatever collected here, it will come to this. So this BMI will come here, right? So that's what the pipe operators and operator does, right? It will pass the, pass the output of this to the next function, output off mute. It will go to the select mutate. Mutate, we will mutate, we will create the BMI, will create the BMI, and then we'll pass to the select function. Okay? So how, if I gave a pictorial representation, what, how it will work, IT support, we will use mutate. Mutate function will give some output. And this mutate function output will go to the Select. Okay? So suppose here we have the mass and height. Mass and height. Mass and height will go to the mutate, and then mutate output will go to the, go to the Select. So this is how the pipe operator works, right? It will pass the output of this to this function. Okay? So this is how the pipe operator rockstar. Here we are creating the getting the new variable BMI, new value BMI, and we're passing to the select function to select function what we are doing named mass and BMI. So it will select the mask and it will sorta BMI. So let me run this code. So click here and see here name, height and BMI. So based on mass and height, weight and height, it is showing us the BMI soon. This BMI, BMI column was not there in the earlier data, right? Let me show you. Here. We do not have the BMI, BMI rehab created with the BMI we have created with this mutate function, right? So with the mutate, we have created BMI and we have added to our dataset. Okay, So this way you can create a new variable or new value and you can add that to your dataset with the mutate function in the play. Next is describe mass source. We are what we can do. We can arrange the mass. So let's run this and see. See here now we are getting the height, mass and hair color, skin color, eye color, all the data, all the columns we're getting right? And it is giving the mass value right? Here. It is. It is like a decreasing order. It is giving us the mass, height, hair color, and all those things. Okay. If you want to change it to, you can change it to some other thing and see how it is coming. Next. One is, we can use the group by as well here. So what group by, group BY will do? It will group by, by species. So what we are doing very lean Starbucks dataset and this is the pipe operator. And then group by whatever the group by species postulate what it will do. It will group either species and then it will give us the will. So as the summary in a single dataset, okay? It will give what summarize function will do. It will give you the single summary of the dataset. So here we are using n is equal to n, mass equal to mean, mean up mass. And a na.rm means whatever value is not available that it will remove what is ended at our umbilical. See if you look here, some of the values are not available, right? So it will remove the values which are not level. Okay, so I ended up aren't. What I will do. It will remove the data rows which are not having any values. Okay? So what it will do, it will create the mask mean of the mass. Okay? So it will, since we are grouping by species. So for each SPC, it will show you the mean mass, average mass. And for that, we are using the mean function to get the, get the average of this or average of the total mass for a particular species. And we're grouping by the species. Okay? Next thing, What we are doing, we are using the filter. We're using filter here. And greater than one, mass, greater than 50, we are applying the filter here. What filter? And so be greater than one and must be greater than 50. So this is our filter, this filter we are applying. Okay? So what other steps we are doing? First? We're grouping by the species, and then we are getting the single summary. And we're getting the mean of the mass for that particular species. And then we are applying a filter where n is greater than one and mass equal to. So let's run this and see what output we are getting. So let's run this and see the output here. See the output C. Now we are not getting the name, we are getting the species because we have group. We have used, we have used the group by species, we have used group by species. And we have calculated the mean of the mass for the species. So that's why it is showing the n. Greater than one. For n one it is not showing, it is sewing and values which are more than one. Okay. And the other species, dry, gone gum, human. Can you own MATLAB, tweak, rookie jab rack. So all the species, it is swing and it is showing for the values 1250, because what we have given, what I've given you the answer be greater than one. So that's why it is showing us the n values greater than 123630 bytes. Now showing the values less than one, okay? And then mass, it is showing greater than properly. So what filter we have applied a filter we applied as masturbate greater than 50. So it is showing us the mask greater than puberty. These masses mean mass up these species. So for this to our display there maybe many things, right? But we have taking the average or mean of all those pieces and we have given the average up the dried. Okay, so let me show you in the data-set. See here like all these species belong to the drought and they have the mass, right? They have the math, right? And they have enable us to what we have done. We have years ended that arm to remove these rows. And for all these names belong to the species. Draw it, draw it, and they have mass. So what we have done, we have taken all the mass, sum up all the mass divided by the number of species, number up names in the dry. This precision, we have calculated the mean by using the mean function here. Let me show you here, right, I mean, so that's why it is so in a star mean mosfet each group of species. So for human main message that T2 for dryer mean masses. This, okay, So this way we can use the plot for data manipulation. Okay, so what are the things we have learned? We have learned that our tidy verse package we need to install to use the duplicate. And we have used how to filter and how we can apply filter on our dataset. And how we can use select in our dataset. And then how we can mutate and add, mutate or add a new variable in our dataset. And then we have seen how we can arrange our dataset. And then lastly, we have seen groupBy and summarize along with the filter in our last example. So I hope you've got to know what are the things we can do with the deep layer and how we do the data manipulation in art. 62. Introduction to Shiny Interactive Dashboards in R: Hello and welcome back. In this lecture we're going to learn about signing from RStudio. So what is this sign? From our total? So this tiny is a way to communicate with the data. It's basically a dashpot solution or a visualisation visualisation visualisation tool for our, okay. So if you want to visualize data in R, you can use this tiny RStudio. This is interact, analyze, and communicate with the sign-in. We can create interactive dashboards where not only will display the data in the graphical format or in a dashboard, but we can also interact with the data thin and they're on the dashboard. So that's the beauty of the sign-in from RStudio that not only will create a dashboard built on the datasets, we also interact on the dashboard itself. So it says take a phrase interactive approach to telling your data story with signing, let users interact with your data and your analysis and do it all with ours. So not only they will see the data in graphical format and in the graphs and dashboard, but they can also interact with the data and they can really see what impact our analysis making on the data. So let me just scroll down to see what are the other options saying. So are in arsine is an R package that makes it easy to build interactive web apps straight from our. So we're not going to use any other things, but we'll use the R programming itself. And we can create interactive web app based dashboards. You can host standalone apps on web-based or embed them in R Markdown documents are built as sports. So see, this must flexibility. It is given that we can, we can not only create a stand alone webpages and apps, but also we can embed them into the embed them into the RMarkdown file. Documents are in a dashboard, so that's the beauty of signing in R. You can also extend your signing apps with CSS teams, HTML, widgets, and JavaScript, Exxon's CEO. So that's the beauty you can use. Css, teams, HTML, widgets, and excellence, so that it will be more interactive. So once created, we can work in various ways with the dashboard created with signing. So what other things signing can do? Assignee combines the computational power of R with the interactivity of the modern wave. So we all know that R has packages that we can like work on the data. We can get the insights. We can get insights from the data right through our packages and add programming. And not only these insights, but also whatever we get. Compute from the data. We can put them into the modern wave and we can interact with the dashboards and data or graphs. So that will give us more insights from the data, right? So see you one simple example of assigning app. You can see you. Okay, so Let me see Google index, Trend Index. This is a simple, Shiny app they have created. And here you can see a very beautiful graph is the right that we can see. Now with the assignee, we can hear, you can select the index, Trend, Index travel. Or if you want to change, you can change it to advertising and marketing and SEO. The graph is changing. I can select something from here. And this graph we'll just build on the Trend Index unemployment. I want to see, See. Now this is showing the unemployment trend. I can select small business and it will solve the small business trend for this data, rent, this new trend. Okay, So, so this is how we can add selector. We can create an interactive dashboard using signing. Okay? And see, yeah, this is created with purely our programs, our code. Now, we will also create one such Shiny app in the next lecture. And see here, this is the app data that is the program that is written in R4, this kind of, this kind of desperate interactive dashboard. So here they're using signage package, deep layer reader, si110 teams and a deep layer. And this is the description. If you want, you can go through it. This is the official website of signing.rstudio.com. If you want to learn more about the sign, you can go and see the gallery and you'll get more information about the signing. Okay, arsine, you can go and see detective modelling process linear regression. And then our tiny package to learn immune response modeling. There are various projects that you can go and read by yourself. I will label on the signage to the website. Okay. So in the next lecture, we will also create one Shiny app. And we'll see how we can interact with the dashboard, okay. 63. ShinyApp Creating Interactive Dashboard with Shiny: Hello and welcome back. So in this lecture we are going to create our own Shiny app, where we will be creating a Shiny app. And so what we're going to create, and then we will see how to create. Okay? So see here this is our Shiny app. And here we're going to do that. We're going to create a diaspora based on the Mario Kart eight video game and based on the video game characters we are going to analyze. We're going to designer. We're going to create a dashboard where you can interact with the dashboard. Click on the Visualization and see here, by default, split will be there. And here on the other variables will be there, okay? The y-axis and x-axis, okay? So you get the x-axis will be the y variable, here will be the speed, and here on the y-axis will be sewing the character. So here we have the characters, the characters and the names of the characters are here, okay? And we're going to analyze what are their speed and other variables like speed in water. And we can also select acceleration and you can see what beta acceleration. So see here, this is the dashboard where we can, we can interact with the dashboard. We can select the variable here, handling. And that's what we'll solve for the, for the handling and say, Yeah, I'll put handling in water and it will generate. This way we can create that's more enough, which will be interactive. And you can select the variable here and see the effect on the dashboard, okay? Graph. And see here the different colors are throwing glass, heavy, light, and medium. Okay, So this is how we can use the signing app to us, the interactive dashboard, we're done the Mario Kart eight game. Okay, so now we have seen what is this? So see here the classes, the different colors are showing the class heavy, light, and radium. And here we can select the speed and it will change variable handling, then the graph will change. Okay, So this is the app, sign the app that we're going to Create, okay? And this, we can open in the browser that it's also because it is opening on our local host. Okay, so we can open this in the browser also to click on open in browser, and it will open in your default project in the same way. Okay, So see nowadays operating in the browser hockey, and we're losing the data source level on Kaggle website. So you can go to the Kaggle website and see the data and see your visualization. You can select change, okay? So based on what you are selecting, it will be okay, So this interactive, that's what we're going to create. So let me quickly take you to the code. It's pretty simple code. So what we need to do here, basically, this signing app will have three components. Three components x really three components, or three webpages, three pages we need to can you get the first one will be the first one will be the app dot, and then the second will be the server. And the third one will be the UI part. Okay? So set of words and then UA. So app.all server and ui.R. So these are the three files we're going to write, okay? And apart from this, we have created one folder. You will have to create one for leucine the app, whatever name you want to give, you can give. And then upgraded one data folder app kept the CSV file, which each containing the dataset. Okay. So let me open and soil. This is the dataset and it is containing all the characters, glass bead, the other details, and these are the names and these are the speed and all those things. Okay? So this is the dataset I have kept. Bring us Character dot CSV, download it from the Kaggle. Okay. Next thing is here, the image which we are seeing on the front face. That images here. W, W, W folder. And these are the three files app guitar solo data anyway, data that I've created. Okay, so let me go to the code. The first thing is app.all. What we need. We need we need we need the assignee library, okay, so you have to download the download and install the Shiny library. You can come here the installed packages sex and year. And you have to click, just click Install and thus tiny name there, and it will be done. So let me show you, just click on here. And here you need to put the signing S, I, N, Y, sign. And then we have to click on Install. And the Sami package will be installed. So I have already installed, so I'm not going to do it again. So you need to put the shiny and then click on Install and it will be installed. Other than this, you can also use install.packages. You can use some problem with the pen, okay? And you can install the name sign here. Literally be done, okay? So you can use install.packages and put the package name and assign me. And here we are giving those source adds ui.R sources, ui.R and other sources server that option. So these are the two things we're going to use in the app. Dot dot, dot, dot is our main main file inside that we are calling ui.R and cellular data. Then we are creating the Shiny app by giving named Shiny app. And here, UH, equal to UA and serological to server. Okay, so whatever name you are giving, the file name that you can give here, okay. So this is the simple app dot our file where we're just loading the library, giving the source like we're going to use E-Y-E dot r and server that are inside this afterthought. Okay, and then we're creating assigning app where we are specifying the UID filename and sulfide limb. Okay. So this is the app that outfile. Then the next one is third word dot are. So sad about that. I'll go to the cellular data. And here you can see we have loading the GG plot to library. Because we are going to, we are creating a dashboard or we are creating a graph to the graph. We are seeing that we're creating with the help of GG plot two. And then here we have a specifying that dataset. So from here we're going to get the dataset. Data's less character, not CSV file app. So on you and we are using read dot csv total, this CSV file. And how to read CSV file that we have already seen in the previous lectures. Okay, Then we are creating the server here. To create server, we're running server where given the value of my variable names are well, and then we're losing function, input and output and output. We are reading at plot. And we are using the renderPlot function to render through the plot. And we're using character, character, and then we are using factors for creating data, labels and order. We're using character dollar class, okay? And then we're using GG plot to draw that Graph. And here, data we are reading characters dot CSV and the x-axis, x value WM character, stat identity. And the y input. Dollar y value means whatever input that we will select for the y that will be selected. That this is the part where we have the interaction or the drop-down that we are selecting, it will come there, okay? And then we are reading cards flip. So coordinates will be flipped. Okay? So next thing is ui.R. So if you see why that odd, it is also pretty simplified. We are using the character characteristic dot CSV and we're reiterating the character dot csv file. And then we're using the page one introduction, since we have two pages and products and n, visualizations and production phase is in Draw tab panel to create the panel for introduction. And we're using the heading Introduction title for the final. And then we're using the image source. And we are giving the break here. And this is the paragraph writing. And then here we are giving the link here, the Kaggle link that we have seen, that we have protein that then page to page one is simple. The interaction and the panel, we're creating the best tool for the visualization. What we are doing, we are using select values, column name. Column name of the characters will be the values. And then what do we are using? Select values, select where losing characters. Okay? So we're part of the class, okay? Here we are using the select query, so good. Not in the select Value job. Unlike basically we are removing the unwanted columns here. And we're going with the character in that class. Here we are getting the sidebar using the side bar panel. And here we're selecting the input to y, where y underscore where and level will be the y-variable. You, if you want, you can change it to anything, choices, select values. And already the loaded value will be those for the speed selected is equal to speed. So speed will be there by default selected for our desperate. Okay. Next thing is main content. We are using main panel and plot output plot, okay? And this will come from the Southern blot. And the second panelists, the tab panel where we are so individualization title panel, that title we are giving, that paragraph that we're writing. And then the side bar layout, which sidebar underscore content, the main content will come here. Okay? So this is how we created this three files. And these two points, server and uy are being calling Jewish inside the, inside the app data. Okay, so now we will run the app. So click on Run app, and it will run and opening up RStudio window. And now we're seeing the page, our hand here, you can see the introduction page and visualize and paste two pages. And introduction pays. Nothing just headed. And the image that we have included, and this is the paragraph and this is the link that we have given you where we have specified this. If you come here in ui.R, see here the title we have given, and then the image we have even better Calabria, we win. And yet, even though we have given the Kaggle link, okay, so that's what we're seeing inside the introduction panel and the second panel, when you open the visualization. Here, you can see why you enable speed by default, because here we have given selected as speed and the level is why you are able to, okay? And we're giving it the plot as output and see the title. And this is a paragraph that we are giving. Okay. So let me show you the how it is changing based on the selection. If I select Speed ground, the graph will get chain. If I select handling, the graph will change if I select tracks on the graph related. So this is the interactive dashboard that we have created using Shiny app. Okay? See here the different colors are showing the heavy, light and medium glass. And here you can select and you can see the difference between these things. Okay, So this is how the sign, the app works. 64. Some Examples of Shiny Apps in R: In this lecture, we are going to look at some of the already written examples of signing apps that are already there on the website of signing. So these are the example of Shiny app that you can look into hand. You can see them and you can modify the code. And you can create your own Shiny app, signing into Esports. So let me tell you how you can run this too. You just need to called the Shiny library. So we need to write library and then signing, and we need to run this. And once that is done, you can start using the examples like the first one is what, a histogram. So one example is the function that you can use to call this already written programs Signing app. Okay, so 01 underscore Hello is this one. And let's run this and see what we are getting. So when you run this, you'll open the assignee app. And see here it is saying hello signing. And here you can see number of bins are coming and here a histogram of waiting times, okay? So here, as you increase the number of bins, see here, number of bins are getting increased. And now there are peptide means. If you make it one, there is only one winner, right? If you put six, there are 6.12 3456 bins. If you see here, it is interactive so you can increase or decrease the number of pins in this task board. So this is the speciality of the Shiny app that we can play with the dashboard that we have created, interactive dashboards that Shiny App Create. So now here you can see, if you come down here, it will give you the description. This is small, tiny application demonstrate, sign this automatic UI updates. Okay? So here the region renderPlot and input bins. And see here abduct, our file is already given here. So you can just copy the code from here. And you can run in, use this code, apple.tar. See your laboratory using library shiny and uy, and then title, then sidebar layout. And here sidebar input ID that we are giving beans, number of bins and minimum is one, maximum is property and value started so by default, it will be selected. And then here, the main panel plot output that we are getting, the output or the main panel. So this is the output idea of the plot. So if you come to the server, you can see your server function inputs or we'll take the function. F function will take the input as input and it will give the output. Okay? And here output this plot. And then we are calling the renderPlot. And here we are giving the x value. And x value is faithful waiting. And Vinci's here you can give debates as sequence of min to max and then dot out input pins plus one. Okay, here we are drawing the histogram. So histogram we're trying with the x value and we are giving the Brexit equal to paint and colored. You can give whatever you want hand xlab and main heading, you can give whatever they want. And then we'll create the app by using Shiny app and then UY equal to ui and server, each call to server. So somebody is deceased the server. And uy is we are defining here. So if you want, you can put this into the ui.R till here. You can put in the ui.R and the server part. You can put in the server that r, and this will be your app data and where we will call the UI and server. So this way, you can take this app dot r, which is a single file and convert these into three files if you want, and if you want to add some more things, if you want to add, if we want to upload some other Graph here, giardia, you can do that. So you can modify this is because the code is already given with duck sign the app website from there. So this is in building the sign-up sign-in package, right? So let me close this. Let's see the second example. Second example is running example zero to underscore texts. Let's see what is there. This is Table send DataFrame. So let's run this Shiny app and see you. When you click on that, it will be opening the sign-up texts. He see here. Here we can select the dataset rock, Fraser and car support. I'm selecting card. So it is showing us the speed and this tends to speed and distance. And here you can define the number of rows, number of observation to view. So now it is 1012. So see here nowadays. So in gravel. And if I reduce this to suppose one, it is showing only one, I can increase, keep on increasing. 123. See here, are interacting with this and here I can select the pressure, so it will show you the temperature and pressure. And here we can simply increase the temperature and pressure, this table as well. So this way, we can play around and learn from this, and this code is also here, app.all that you can take and you can modify it as per your requirement. So let's move to the third example. That is the reactive expression app. So 03 underscores reactivity. And let's run this and see what our si110 app ready getting here. So see here the data assembly gaps and needs. Data assembly and dataset is rock preserved and cars again. So we can select braces and here, the same thing as rows and columns. You can select n for this also. The outfile is given, the assignee, our file is given here. You can take it to look into it and you can learn odd if we want to modify, you can modify it. It's open source, so you can take that and modify and use by yourself. And next, let's see judo foreigners called MPG. And click on Run. So the apogee opening noticed swing number of cylinders, transmission and gears. So it is showing miles per gallon. The relationship between the MPT and gear. And when you select transmission equals Toyoda, lessons in between the automatic, manual car with the Majlis, okay? And if you select gears with the gear cylinder with the cylinder, and for this one, so you can get the tar file and you can modify n niche for yourself. Next thing is slider bars. So let's run this and see what. So these are the pretty good example to learn about signage package and you can modify and lawn. See here. This is the sliders. So there are so many sliders on this that sport. And for this quarter is already there. You can see here I can integer value. I can make 1,000 or 172 decimal, 0.8 or one like this. And then the range, I can modify it from here. And as soon as you modify it here you can see the changes there. Custom formatter we can define looping animation, we can define here. So this way we can add anti-A here. That is the option to play on these custom format. You can click on here and Syria 0-500500 automatically it is changing. It's playing like okay, so see here, now it is slowly changing here. And see that this number animation, this is also tending. So as this is changing, slider is changing. This is also getting changed and add. The other figures will also get changed. So this is the Magic of si110 package in R that you can create these interactive dashboards. See here, this is changing and this is also getting change. Okay, so let's move on to the next example that is type cells. So just click on run and see what we get in the depth it normally uniform, Long, normal and exponential. So we can just click on here and we can see the summary, we can see the table, see how it is changing. In one click, you can see the normal. If we want to see the uniform, you can see the uniform. And if we want to see the long form, you can see the long poem. You can see the older treating, plot summary and tables. And all three are changing together. One-click all the data here, number of observation you can increase. And instantly you can see that this is very good tool for analyzing data and data analysts or data scientists. Or if you want to enlarge your data, you can use the arsine and this kind of interactive dashboard you can create and said with your team or your manager, you're out with your client and they can instantly get the insight of the data. So in the similar way you can see the other thing. See here an example, arginine upload that will give us the vacillated to upload the files and all. Let's see. You can click on here and you can simply select any file and you can upload it. Suppose I want to upload, now it's uploaded and because it is not our CSV file, so we're getting into, so if you want to upload a CSV file, you can go ahead and you can upload a CSV file. So let me go to the trial court to our 2020, and I'll try to open a CSV file. Let me open nato dot CSV data we have cured in one of our example, let's open that. Let's upload data here. Now, we can see that Data dot CSV file has been uploaded and we can see the data inside the data dot CSV. We can remove the handout. You can put the header like that. We can do all the things. We can put semicolon. We can use a separator comma semicolon, or even we can use duct tape and quote Non double-quotes. All those things you can do here. I already showing all the data that is there in the Data dot CSV, right? So your karma and see if I put head, it will show you only fueled the dog is like somebody and getting the full retail. Notice throwing data. So this is quite important thing. And here you can create this. This happen in our si110 and you can share with your client where they can upload the CSV file and see the CSV file without having XL T-TIP. They're not having a fair down to ally. They can simply select the CSV pylon. I'll see look into the CSP Friday, okay? And for this also code is already given here. So that's pretty cool thing, vegetable, the things you can analyze as an assignment and you can learn from these things you can modify and you can use for yourself. See the timer. You can click on here and the timer will come from the current time is this thing. This way we can use the signing. 65. 2 File Shiny App in RStudio: Hello and welcome back. In this lecture we are going to create two-page signing app. So two files signing up, we're going to create. So we have seen how we can do three-page, like we have done apple.tar, ui.R and Teradata. One other example. Here. What we are going to do, we're going to create one lead two files. One is ui.R, and the second one will be server data. So first thing, what we need to do, we need to come to our working directory and we need to create a folder with the acronym. So here I'm giving the app name as true while signing app. Okay? And inside that, I'll create two files, ui.R and Solver dot art, so that I have already created and I have written the code for the way that that is inside the file signing app folder. Okay. So the app name will be the folder name. And inside the folder we'll save this ui.R and server dot r. So what this dot will have the same UI equal to fluid page and sidebar layout and sidebar pane, panel and sidebar input. So here we are giving OBS and number of observation, observation and number about visitors and minimum is ten, maximum is 500, and value will be hundred. Okay? Then we're calling the main panel and plot output. This plot and this plot will be coming from the server that our sources are what we're calling function input and output and output will be the dish plot. And this plot will be rendered in to the histogram, where the data will be the input of OBS will be the input for our norms and the color I'm giving us a red and green. Okay, so only these two files and no need to write the app are and how to run this file. We can simply come to the console here. So let me clear the console. And here you just need to write a run app. Run that. Now we need to call run app and inside the runApp unit to provide the folder name or the app name to file sign the app that we have created. So this four that we have created to sign the app. So we'll give that a name here and we need to run this. So hit Enter and your Simon app will be finding CEO number of observation. And here is a histogram. So I've given green, red and the border will be green. And if I change this number up of generation, it will keep changing. So this way we can create or to file signing app where ui.R answer for that data, will we kill it? If want to change the color, you can put the yellow and green. And here we need to just close this and we need to run it again. I ran app to files any app and hit Enter. Let's see here now it will be coming in yellow and clean. Ceo, the yellow and the border is green. So you can do like this. And you can see that a number of varied and abnormal apoptosis and you can see the histogram is changing. Okay? So this way we can create two files. Sign me up. 66. Generating Downloadable Reports in Shiny: In this lecture, we are going to learn about generating down loadable reports. How we can generate the reports which we can download our downloadable divorce. Okay. So when you come to the assignee.rstudio.com website, you can see there's already a substance and readies things you can learn about signing from our studio. So creating the interactive dashboard here that is opsin. You can see it generating downloadable reports. So here you can see the example like app.all and that they have given. And here you can try this out and report dot RMD. And he believed that report that are empty and it will create a plot. And the thing that report you can download. And there is another vertex example that I'll show you, where we can download the report in the form of PDF or HTML and bars. So see here, this is the maiden, the empty cars dataset that is readily available with the R. Here, you can see based on the number of cylinder, we can see the regression model. This is the regression model. Cylinder and number of cylinders, MPG, how the number of cylinders are affecting the mileage, miles per gallon, how the hotspot is affecting the MPG, at how weight of the car is affecting the miles per gallon, the evidence or mileage of the car, or the radius, how these factors are affecting the number of gears is affecting the myelin that these are the things that we are analyzing your debt we have done through our programming. And these are the things we can like the internally it has been generated when developed a regression model where they're trying to find the best fit line. And for this, now, this is the report and supportive for your displacement. Can put like this and there is option document format, PDF or HTML or work to support them clicking PDF and I click on Download. This whole report will be downloaded in PDF format if I open. And so you see here, it is showing that it worked like this. Here is the regression model. And from where we're getting these things, this thing is going from the report dot RMD file. We are there creating the linear regression model based on the data sources, empty cars. And then they're using the coefficient for it. And then they're putting that into the plot to find the best fit line, and then they're getting the best-fit line, our regression line. So this report is coming like this. When you see here it is showing us the data points, but when you download it, we are getting the complete report. And this is the File set of r dot r and c here. The averaging the formula, the tip and MPG input, then the output, deleveraging other plot and then download report here, mighty portion, the filename format will be the PDF and then content, they're getting the normalized. But let's import that are empty. This is important because all the calculations and all that we have seen in this report here, these are the things coming from the assembly files. Okay, so this is pretty important. And then here they are, simply are setting the working directory to temporary directory so that it will not ask for the admin access. And I'll look for that. They're using this and then find lot copy report that at MD, right? Then library the original markdown and this is to render them RMD file into the PDF. Okay? So, and then find out rename output file. Okay. So these things, let me also show you the RMD file that has been used internally. So you can click on the Get code and it will take you to that are signing GitHub repository. Here you can see a report that our employees are words that are anyway, dot, dot all the files. So when you click on the Report dot RMD, you can see here the averaging the regression model and this is the code here is my regression model. And then using our model collapse true. And he had opsins. These other things we're seeing great. And then the fruiting original MPG and this thing. Okay, so they're creating a scatter plot and then there, finding the best fit line using the abline. And they are putting the color right? So this is how we are getting this report from my reading that assembly file and preparing the reports. So this way you can get a report which can be downloadable. If you want to download this report in the Word format, you can click on that and it will be downloaded it in a doc X4, right? Okay. And if you want to download this in HTML, you can click HTML and download. And it will be downloaded in the HTML forward, Let's open this and CSIA data, my report dot HTML file. So this way, if you want to create a report or dashboard that can be downloaded, you can use the discord and you can write your own. Sign the app. 67. Analysis of Covariance: Hello and welcome back. So in this lecture we are going to learn about analysis of covariance. And it is also called and cola. So ENCO Reais. That means analysis of covariance and covert and sort we call it. So what is this analysis of covariance? So you know what we do in the machine-learning algorithms or in a linear regression model. What we do in regression, we try to find real value, a continuous value, right? Unlike in the categorical data, we always try to find the arc length yes or no, true or false, zero or one like that, right? So in regression we tried to find a continuous value. So we create a regression analysis to Lake. We use regression models. Regression analysis to build models which describe the effect of variance in predictor variable, right? On the response variables. So what is the effect of the variance in predict variables on the response model? What effect the predictor variables are going to put on the response variable. That's what we're trying to do with the regression analysis. And whatever models we are creating with that we're going to do this one way. But sometimes what happens? We have a categorical dataset. So if you see these empty cars data set that is inbuilt in the R and that we are going to use if we run this piece of code, you will see we have a column E m. What is this m. M is automatic or manual. See its value, its value is zero or 10 means automatic and means manuals, so something like that. Okay? So D represents the Automatic and Manual, okay? So this is a categorical data, is either zero or one. It is not having a continuous value like miles per gallon, 2,122.8, 21.14, 0.7, right? So it is difficult to find the effect of this categorical variable on this mpg. See here the horsepower, HP is also there, but it is a continuous value is not a categorical. Hello. So what happens? We have to, we have a categorical variable with values as a no or zero or one, male or female. In some cases. Anything, it could be profit loss, whatever, so whatever categorical values we can have. So the simple regression analysis give multiple results for each value of the categorical variable. So for this G row and one for zero, we'll get 21 point for this 018.7. So we're getting a different, different value for MPG, for same zero, right? Zero represent the Hornet and these represent the Hornets sport about, right? So these values are different. So how will we will find what effect it is putting on the miles per gallon. So that's what comes into picture, analysis of variance. So let me give you a little explanation. Analysis of covariance is used to test the main and interaction effect of categorical variable. That means this aim on a continuous dependent variable, that is mpg, controlling for the effect of selecting other continuous variables. So if we know what effect this is putting on this, we can have even decided what HP is having an effect on the MPG or what this m is having an effect on the HP or mpg, which co-vary with the dependent. The control variables are called covariates. Still controlled variables are called covariate. Sometimes, if we have a categorical variable with values like yes or no or male or female, or profit or loss. Simple regression and lances get multiple results for each value of the categorical variable that I discussed here. For zero, there are many values but MPG and for one also there are many values. The simple regression analysis gives multiple results for each value of the categorical variable. In such scenario, we can study the effect of categorical variable by using it along with dark predictor variable and comparing the regression lines for each level of a categorical variable. So what it is saying, it is saying that in such scenario we can study the effect of categorical variable by using it along with the predictor. So along with the predictor like HP. So we'll use a m along with the HP to predict the MPG. And we'll find the regression line for each level of the categorical variable for 0.41, for each, but each level will find to find the categorical variable. Okay? And, and this is called the analysis of covariance. So let's consider this empty cars dataset where m represents the transmission auto manual. It is a categorical variable with values 0.1, as I discussed earlier, miles per gallon or MPG of a car can be depend dependent on it. Besides, the value of horsepower, mpg can be dependent on whether the car is automatic or manual, and it can be dependent on the horsepower as well. So we studied the effect of EM on the regression between between MPG and HP. And it is done by using a or V function. Aov function is analysis of variance function followed by the NOR function. So we'll use the Bot functions. We'll first use the AOV function to find the regression. By using these two variables are predictable fellow HP and aim to find the MPG. And then we'll use the Nahuatl and decide which model is giving us what and how this categorical variable or a categorical predictor variable is affecting the dependent variable MPG. So what we do first, we get the input data that is empty cars. So we're storing that into the empty cars data. And then we are creating a regression model and reversal model. For regression model, we're using a AOV or with analysis of variance function, AOV function we are using and we are creating a regression model between MPG and HP along with am. And we're leveraging data sources, empty cars, data that we're getting from the empty cars. And then we'll, we will print two that. We will get the summary of the model one. So let me run this and let me run the model one. So we're creating a regression model here. So what insight we are getting from this regression model here, this original source that both the horsepower and transmission type that HP and M has significant effect on the miles per gallon as the p-value or both. P value in both the cases is less than 0.5, right? So it is having both p-values having less than 0.5. But when we take this HP and m together, the, the p-value is more than 0.5 and that means that SPN am together will not have any significant effect on the MPG. So interaction between the, this AM and SB is not significant for will not have any effect on the, on the MPG. So this model is giving us this insight that HP and am, if you take them alone, they are going to have effect on the MPG. But when we take them together, they're not going to have any significant effect on the MPG as the p-value is more than 0.5. Next thing what I will do, I will, I will create another regression model. Where will you, what we'll do? We will not include the categorical value. Value, okay? We model without the categorical value, that is am. So I mean that without the interaction between the HP and m, and we are going to create another regression model, AOV function. And we'll see what effect they are having on the MPG. So let me run this. Yet I'm using empty cart to as valuable and it will have the same data as empty cars. So see here. Now our model with this, there is no interaction between HP and m. Then what insights we are getting. This is your souls that both horsepower and transmits them has significant effect on miles per gallon as the p-value is less than 0.5. Okay, so now what I'll do, I'll compare these two models, model 11 with the categorical value. Interaction between the HP and m. And another model is without the interaction between the PNM. Okay? So now let me compare these two models, model two and model one. So what i'm, I'm using, I'm using NOR function for comparing these two regression models. So let me run this. See now what insights we are getting from this region. We are getting that the p-value is greater than 0.5. And so we can get a conclusion that the interaction between horsepower and transmission type is not significant. So the miles per gallon will depend on depend in similar fashion to both HP and am. Okay? So the HP and m both are going to affect, both are going to have effect on that MPG enough. Same way. It is not like both are going to affect differently. Both are going to affect in the same way. So the MPG will depend in similar manner on the horsepower of the car in both auto and manual. So whether the horsepower support horsepower of one car, which is automatic, and he's giving more mileage. And if we didn't manual with the same hotspot, it will give less myelin. That is not going to the case the mileage of the car, whether with the same support, one half. Now, suppose 1 hp, one car is there and it is manual. And another card is the same, 1 hp and it is automatic. Both will give the same mpg. The average is not going to change whether the car is automatic or manual. Okay. So it is depending on the horsepower and manual or automatic will not affect much on the car's mileage. So this is the way we can do the analysis of covariance. Covariance means the other predictor variables that are going to go Eddie, I'm going to effect on the dependent variable. In this case, it's b and m are the covariant and mpg is the dependent variable. So we have found that whether the car is automatic or manual with the same ball is going to have the same kind of mindless. 68. Handson with dplyr library: Hello and welcome back. In this lecture, we're going to revisit the data manipulation. And data manipulation activity will do with the deep layer package that is relatively level with the odd. So the player is a package which provide Egypt tools for most common data manipulation task. So we'll learn all about the plan today and we'll see what are the manipulation task we can do. And we'll do this long file that I have written. We will do all the things and we'll see what are the things we can do and how we can play with the data, how we can manipulate the data manipulation tasks. How we can use the data manipulation task using the plan. So let's get started. So deep layer provides the grammar of data manipulation, providing a consistent set of warps that helped you solve most common data manipulation challenges. So the very first function that we'll discuss is the mutate function. And what it does is it adds new variables that are Cells of existing variables. So what it will do, it will add new variables that are functions of existing variable, will come to know what does it mean when we do the hands-on. Okay, so next one is select, select, it spits cell. What does select? Does it picks variables based on their name is same like a select query in SQL, if you know SQL or MySQL or any relational database, we use the select to select the value will select something from the data from the table using the column name. Okay, So that's what select also does here, fixed or variable based on their names. Then we have the filter, narrower filter. What it does is it picks the cases based on their values. So if you want to filter out your data based on some values, like on the ear or some on anything, any dress and you can watch, okay? Summarize, function reduces the multiple values down to a single summary. So what summarize will do, it will reduce the multiple values down to a single summary. We'll also see that then we have the orange function. What I didn't do, it changes the ordering of the rules, so it just arrange the data in a different order. The order of rose, twilight, anything okay. In addition, it also support the pipe operator. So this is the pipe operator percent tests and then greater than sign and then presented, this is called pipe operator. Okay. So now let me tell you how you can install the package in your artists to do or even in art. So you can simply go to the packages here and you can click on the Install and you can put apply here. Okay. Apart from that, you can use the tidy verse package also, the easiest ways to install the plot is to install the whole tidyverse. So if you installed the tidyverse package, it will include the D plan. And if you don't want to install this tidyverse because tidyverse will contain many packages. Okay? So if you want, don't want to, the packages which you are not using, you can go and put install.packages D plan, okay? So it will install one leader duplicate packets. Apart from that, you can also go this way. Now. You can come to the install package and you can put that package name and you can install it. And to invoke the library, we have to start using the library and then the package name that we give. So the other things you know already, but just to divide the things I'm telling you. Next thing is less explored. The basic manipulation wives of the player will use the dataset Star Wars. So we're going to use the Star Wars dataset that is available with the arc, which is pre-loaded in R. Okay? So if we put head, head, Star Wars, what it will give, it will give the first five. First type rose up the dataset that we already know and we have used multiple times, right? But this deep layer package will give you another function that is called glimpse that I have not used till it. I'm using it for the first time. So if you use a glimpse Star Wars, it will give you the all the rules information. And like this, see here it is saying is row, 87 rows and 14 columns. And it is giving, giving us the name of the column, name, height, mass, color, and what are the values for that? Okay. First three for collapse. It is giving us the, so this is the glimpse punks and it will just give you a glimpse of the data. So if you want to get a quick glimpse up the data, you can use the clean function. Otherwise, you can also use the head also. Okay. So if you comment out glimpse and if you ran head unit, head will give you a result like this. And a glimpse will give you that result like this. Okay? So this is the head and this is the limbs. Okay? So now We'll move to that pipe operator. Let me close this. So and similarly we have also seen the view. So you can give you a Star Wars and it will give you the view of the data lake. It will give you the full view of the dataset like a normal person you see. So this will give you the entire dataset, okay? Now the pipe operator, all the deep layer function take DataFrame as the first argument. Dataframe will be the first argument for all that the platform some, rather than forcing the user to either save intermediate object or nest functions, The plaid provides the pipe operator filter rows with the filter function. So filter allows you to select the subset of roads in a DataFrame. The first argument is the DataFrame. The second and subsequent document report to the valuable within the dataframe, selecting rows where the expression is true. So see here to select the character with light skin color and brown eyes. So what we can do we can use this is the dataset name Star Wars. And then we can use the pipe operator. And then we can use the filter, and we can give you the parameter for filtering. The skin colors would be light and eye color. So be brown. Okay? And you can also do in other way like Star Wars. Then you can use the Star Wars and then the bracket, Star Wars, bracket, Star Wars dollars can color. It means that we are going to affect the database on the skin color from the Star Wars dataset. And it's scaled, skin color will be light, and then star was taller. Eye color means we are going to face the database on the eye color and eye color should be brown. You can use the pipe operator like this. Or alternatively, you can use the dollar. And Star Wars. It means Star Wars dollar skin color means we are going to fit the data set based on their skin color from that Star Wars dataset. It means skin color is a one column in the Star Wars dataset. Okay? And then we can use simply this or you can use the pipe operator and use the filter bed under skin color and eye color. Both will give you the similar result. See here. Both are giving you the same result. There is no difference. So you can use either with the pipe operator or with their dollars. Okay? So these are the two ways you can filter the data. Arranged rows with orange function. So our infant son works similarly to the filtered except that instead of filtering or selecting rows, it reorders it. What did it do? It reorders the entire dataset. It takes the DataFrame and a setup column names are more complicated expression to order BY use this okay, and descending order. So if you use this DESC function, it will order the column in the descending order. If we use EEOC, it will sending out. Okay, so here adding elements in the descending order of height. So how we can achieve this, we can use Star Wars, then pipe operator and then arrange and DESC function and height. And simply you provide this and it will arrange the data in the descending order. See here, now the data is from the highest and it is decreasing. This is the descending order. So this way we can add in the data or filter the data based on the height in descending order. Okay, next thing is slice. What does slash two slides lets you index rolled by their integer location. It allows you to select, Remove, and duplicate rows. So slightly disagree slice the data from our dataset. Slice underscore head and slides on this called tail. Select the first and last rows of the data. Slice underscore sample, randomly select those rows. So slides under underscore sample will give you the random selected Raj from the dataset if you want your font to. So if you don't get a sample from the dataset, big dataset, you can use slides on this code sample and then randomly pick some roads and sold it to you, used often prop, to choose certain. Proportion of the cases. Slice underscore mean and slice underscore max select the rows with the highest or lowest value of our variable. So it will simply give you, if you provide slice underscore mean and here you will provide height. It will give you the minimum of the height and slides underscore math, max, height, it will give you the maximum of the height. So let's do the hands-on. So to get the character 5-10, we can use Star Wars pipe operator slice five colon ten. So let me simply run this. Okay? We'll give you five to ten, okay? If we use n equal to three, it will give us, see, only three rows, okay? And if we use n equal to 512345 lines, if you see the difference here, five to ten, it will give you 56789106 row state, including five n here, n equal to 5 min only 512345. Okay? Now, to sample ten per cent of the rows, we can use this. We can use Star Wars five operators, slides underscore sample. Prop 0.1%. 0.1 means ten per cent. And this will give you the ten per cent of the data. Okay? It will slice out the 10% of the data. Now using slides to get the lowest or highest value. So we can use Star Wars filter is not, not to hide. And then we can use slice. So we're getting next is select columns with select off when you work with a large dataset with many columns, but only a few are actually of interest, are few select allows you to rapidly zoom in on a useful subset using appraisals that usually One leave work on numeric variable position. So selecting columns by name. So you can use Star Wars pipe operators select hair color and eye color. So this will give you hair color and eye color. The queen and only this line. And if you select the columns by slicing, we can use the hair color and eye color, both are not same. Then we want to select that. This result you can see the hair color and eye color is not there. All other columns are there, but the hair color and eye color is locked and not included. Selecting columns with the specific criteria ends with color. So we want to select the data for each other, anything with that color. So let me run this hair color, skin color, and eye color. Three columns ending with the color. Now mutate, mutate from Sun allows you to, allows the creation of new variables. This is extremely useful for statistical analysis. So here are the Star Wars pipe operator mutate height to height by 100. So what do we add? Here? We are getting the height from the Star Wars and dividing it by 100 and creating a new column height underscore. And then select the height, height and everything else. So let me run this and see what do we get C here, we are getting the height underscore m new column here, and this is 172 divided by hundred, so 1.72 and then all other columns from the Star Wars, it's included. So this way we can mutate on new column height underscore m to the dataset. And we can mutate the new column and we can get the new value for that column using the mutate function. Okay? Now, fitted regression model with the height as exam, exam three valuable and Mars, either response variable including the outlier, display the summary statistics for the existing cooperation. So here we use a regression model. We'll use the lm function, height and mass, and then data values, Star Wars and summary. I'll use this. So let me run this. See here, these are the summary we are getting. We can also plot this regression model. Let's see here, we will get this residual for leverage. Okay? So if you want to go into the detail of this, you can see the water, the coefficient we are getting, the ReLu p-value, all those things you can see what is the minimum median. First quartile, the third quartile, maximum value. All those things you can analyze, you can see here normal Q-Q and standard residuals and theoretical quadrant tile location. All these things. You can analyze your agenda, getting somebody and you are plotting the regression model based on height and mass from the data was told this is the quick hands-on using the plot function, a D-flat library in the R. So I hope you got to know what is D flat and how we can use the different functions like mutex, select Filter, summarize and aren't all these things. The next lecture. 69. Simple Linear Regression Using Airquality dataset: Hello and welcome back. In this lecture, actually we are going to do another simple project where we are going to use the linear regression. And what we'll do with the linear regression to try to find the best fit line for our dataset. That is, air quality dataset that is readily available with the R. So this air quality data set is available with the r. And this is the dataset about Newark air quality measurement. Okay? So let's understand what this data is all about and then what we're going to do with that dataset and what we are going to achieve with the linear regression. So daily air quality measurements in Iraq May to September 1973. So this is the dataset all about. It is measuring the air quality in the New York City from May 1973 to September 19 hundred 73. The dataset is air quality. And I'll format the data format with the 153 observations on six variables. And what are these variables? Ozone layer, what is the amount of ordinal levels? So r dot r will be numeric number and it will be in the lungs. And this will be the PPB. Wind speed will be the MPH, and then temperature will be in Fahrenheit and the month one to 12 from Jan to December. Okay. So ozone means daily readings of the following air quality values. 1-30 September 1973 has been taken down and listed in the dataset. Don't mean or don't in parts per billion. 1300-1500 h at Roosevelt Island. Okay. So loud at what this represent, disrupting the solar radiation in the land lease in frequency band 4000-7700, Armstrong angstrom, from 08:00 in the morning to 1,200 h at Central Park. Wind. This is the average when the speed in miles per hour, 7-10 h at Land Gorilla airport means this is the maximum daily temperature in degrees Farenheit at this airport. Okay, So this is all about the data set. Let's move to the code. So I've already written the code. I'll explain you what we are going to do. The first thing is we need to use the dataset that is available with the art, that is air quality. So we can use data and we can use inequality. Okay? And here we can view the data with the view, function, view and then quantity. And we'll see the data. So this is the dataset. We have the six columns or John solar data. This is the solar radiation, wind speed, temperature, highest temperature in the day, and then month, which month? And this is the day. So they have the month, month temperature when he split solar, solar radiation and origin. Okay? So this is all about the dataset and it is containing one per three data, okay, one purpley, three rows in this dataset. Now, what I want to do, I want to simply plot this or Joan and solar radiation in a plot and see what we're getting. So let me just plot this data. The x-axis is about solar radiation and ozone layer will be represented on the y-axis. So on a particular day, the solar radiation will be this and the origin, let amount will be this. So this is the graph we're getting. Now, what I want to do with the linear regression, I want to find a best fit line that will represent this dataset correctly. So what this best-fit line will be? Something like this. Something which will divide the data almost equal. And with that, we can guess what will be the solar radiation on a particular day. So if you follow that line, like y is equal to mx plus c will be the straight line equation. So in that, if we draw that line on a particular day, we can guess. Okay, so what I'm going to do, I'm going to find out what is the mean of the ozone, ozone layer amount. What is the mean up to what I lose, I lose that are drawn from this air quality. Air quality dollar or John will give us the origin, the value up or down from the air quality dataset. And here what I'm doing, any data from each quarter true means if the values are not available, we just saw, not consider that. Okay? So with this we will find the mean of the origin. So mean value of the origin is 42. So it will be something like here. What I'll do, I'll draw a straight line which will represent the mean value of the origin. So let me run this thing here. This straight line is representing the mean of the origin. Okay? Now what I'll do, I'll use the linear regression to fit the best possible line. Okay, So how we can do that, we can use the lm function, linear model function, and we can use the origin and solar really sensitive here. And then I'll use the air quality data. Okay, so from the air quality data, use these two parameters or don't answer lot that are solar radiation. And I'll try to create our linear regression model. So I've created, now I'll see what our model is giving. So let me just put this on the right side. So see here, we are getting this coefficients. Y intercept value is this and so loud at artists this, okay? Now what I'll do, I'll try to, with this model, I'll try to find the best fit line that full represent our data correctly. So what I lose, I lose AB line, and I'll pass this model that we have created with the lm function and our collateral game. I want to give the different color to the best-fit line. So I'm using the color coding here, orange. So let me run this. Ceo. Now, this orange line is the best fit line that we are getting for this thing. So this will represent the value of the regularized value of the ozone layer buried on the solar radiation. This is the best fit line that we're getting from the linear regression. So based on this support one PIP3 days of data we have, we want to predict that. So we can see that what is the support I want to predict here? So this solar radiation, suppose here, and this will give us the value some delta yard that will be around 60 or 75. Okay? So this way we can find the best fit line using the lm function or linear regression. Okay? So this is how we can use linear regression to find the best fit line. 70. Dealing With Missing Values: Hello and welcome back. In this lecture and coming few lectures, we're going to deal with the missing values in R. So we'll see how we can deal with the missing values in our programming. Because whenever you start a data science project or machine learning project and you start working on the data. The very first step is the exploring the data. You have to read the data and you have to understand the data. What are the values there? What is that data all about that you have to really understand? Unless, until you understand the data, you will not be able to create anything or get any insight out of that data. So it's the very first step to understand the data. And once you have the understanding of data, what are the columns are there in the data? What are the values there? And what that data is talking about, what is that data for? So once you have this understanding, so once you have the understanding of beta, what the data is talking about, what the data is for. You can interpret that data, and once you have understanding, you can walk on that data. So very first step, you have to keep in mind that you have to understand the data. You have to understand what are the columns are there and what are those columns for? What kind of data is being stored on that on those columns. Once we have that understanding, you can start working on the data. You can start working on your machine learning or data science lifecycle. And the one very important step in working on the data and creating machine learning or artificial intelligence. Data science models are the dealing with the missing values. Whenever you see any datasets, you can see that there are many columns, many rows where the data will not be available. And what that mean. It means that whenever we, we collect the data through form, through surveys and through collecting input from the users. So whenever we collect data from the users radius you just large group of users. Sometimes they hated to share their data or sometimes they will not have a kind of proto level with them. So they skip that Dojo input points. And when we collect those data and make our dataset, and we may find that there are many, many columns, many, many values are there which are missing. And to make our dataset work properly for our machine learning or artificial intelligence model, you rehab to deal with the missing values. So it is a very common task in data analysis to deal with the missing values. In our programming. Missing values are represented by an a and it means not applicable or whatever. You can understand how missing values are represented by an a and in some other ways, it is also represented by 99. Okay? So the very, very important step is dealing with the missing values. So while dealing with the missing values, we have to follow the three steps. The first one is tasting for the missing values. What does it mean? It means that we have to look into the dataset and we have to test for the missing value. We have to see how many missing values are there. Where are the missing values? So we have to find the missing values. Then, once you have tested the missing values in the dataset, we have to record the missing value. The second step is decoding the missing values to what is decoding up missing values is that we have to put some, are other values in place up missing values. Suppose we have our age data where a few of the you just have not given their age detail. So what we can fill there, we can take the mean of that. You just ate all the youth justice mean or average is we can put on the missing value. So whatever font you just, whoever is missing the age, we can put that average age value of the group in there. That way we can record the missing values. So we need to record the missing values based on some algorithm like mean, all those things. Okay? Then the third step is we can, or thought, or the other way would be excluding the missing value. So another way we can, instead of recording, we can exclude the missing values rows. So we can simply remove those inputs which are not having the support, aid support you. We have name, you just city and age and employment details. And if some user is not given at his age, we can simply remove that user from the dataset so we can exclude the missing values. So these are the three steps in dealing with the missing values. And in the next exercise, in the next lecture, we'll do a simple hands-on activity where we'll see how we can taste for the missing values, how we can record the missing value, and how we can exclude the missing values. So next lecture will be all hands-on on the, dealing with the missing values. So see you inside the next lecture. 71. Test the Missing Values: Hello and welcome back. So in the previous lecture, we have seen how we can work with missing values in R, or how to deal with the missing values in R. We have seen what steps we need to follow. We need to follow these three steps. Or you can say two steps. Either you can record the missing values or you can exclude the missing value. So the very first step is taste for missing values. We need to test and we have to find the missing values in the dataset. And then the second step is required, the missing values or exclude the missing values in the dataset. So let's start with the very first step that is testing for missing values. So how we can test for missing values. So I have written this code and we'll do the hands-on width. So let me tell you what I'm doing here. I'm creating a vector with some missing values. So I'm creating a vector 1-6, and then I'm including one NA value that is missing value and then eight to ten and then missing. Well, when we run this vector, see what we get. Output is 123456 because one to six. And then when any value, then 89108 to ten and then any. So we have two missing values here. This dataset we are creating by ourself. Okay, So to tell you what is missing value, I'm just including the missing value in the vector. Okay? So now suppose this x vector we have, which is having two missing values. And we don't know here we can see, so we know that there are two missing values. But suppose we don't know, we're not aware, we're not, we're not seeing how we can. It's a large dataset is not only one row. There may be a dataset which will have multiple rows, thousands of rows you how we'll find the missing values there. So we can simply run a test that is, is Annie is any, and then we need to pass the dataset name. Okay, So here we'll pass the extractor is enabled, find the missing value. It will say, if missing values there, it will say true. Okay? So each NA and missing value, it will go through the this vector, x vector, and it will find the missing value. And a missing value is there, it will return true. So let me run this. Here. We are getting false, false, false, false, false and true. So this is for 1234566. False means there is no missing value till six. Then for seventh place there is any, and for that it is returning us true. Then 8910, false, false, false. And for up to ten, there is any, so it is returning true. So it will return you the true for the missing values. So in this way we can come to know that there are missing value for these two values. Okay? Now let's create a DataFrame with missing data. So I'm creating a DataFrame. Df data.frame is the way to create DataFrame. And here I'm creating four columns, column one, column two, column three, column four. And in column one, I'm giving 123 and a column to this NA is, so these are the value I'm putting into the DataFrame. Okay? So let me run this DataFrame and saw you the output. So see here, the DataFrame is like this. Column one, column two, column three column for column one will have 123.1 and column two will have this is a text. Column three, true, false, true, true. Okay. So and column four will be 3.55, 0.26, 0.2. And so this is the DataFrame that I have created. Now, I want to identify n is in full DataFrame. In this dataframe I want to find any. So I can simply run the test is an a and I can first the DataFrame V0. So each enter data frame will return true or false. So this NA, that's right here, column one, fourth row, we are getting through, then another entry here. So we're getting through here. And that is another and another two here. That means in this one, okay? So this way we can run our test is any via. Suppose you want to identify any specific DataFrame column. So I want to check whether this DataFrame is having any, any values in column two so I can run the test is any. And then the DataFrame name df dollar column two. So what it will return, it will return the true and false for the column. So let's run this and see your false, true, false, false. Why it is false? Because in column two, value is there, so it is returning false. For N8 is running true for is and text it is Dunning, false, false. It means there is one missing value in column two. So this way we can find the NA values are missing values in particular column. Now we can run the sum function and identify the count of NINR DataFrame. So we can use some off is any df is N A, and we need to pass the dataset name. So let's run this and see here now we're getting three. So there are three. N is in the data frame. Now, column sum is df, it will give you the sum of missing values in columns. Okay? So let's run this and see it is done in column one, row, one, column 21, column three. There is no data missing. And column for one. So C, column three, everything is there true, false, true, false, and we have no missing values in column three. Column three supports if I put something, some other values, instead of true-false, I'll put some brown. Some values we need to put 20, Okay? And now run the DF. Okay? Now run this. Column three is having zero missing values because it is having round down 905123 all the data available, right? That's why it is done in general for the column three, sum of number, number of total number of missing values in column three is zero. In column one, there is one missing value, that is this. In column four, there is one reason, well that is this. And in column two there is one. So this way we can test the missing values in R using is any, is any will return true if there is a missing value in the DataFrame. So this is how we can achieve that first step. In the next lecture, we'll see how we can achieve a recording of missing values. See you inside the next lecture. 72. Recode the Missing Values: Hello and welcome back. So in the previous lecture, we have seen how we can work with the missing data. And we have seen how we can identify the missing values in a dataset using is dot, right? So each dot and it will give you the missing values. And now, once you know the missing values are there in the data set, what we can do, we can either record them with some value or we can remove them. So let's see how we can record the missing value in a data set. So first thing is we can record the missing value. If it is a numeric data, we can Record with the mean of the dataset, mean of the values, okay, average of the value. We can port two, we can say. So here I'm creating a vector x, okay? And let's see what is there in the vector S. It is 123456, then one, then 8910, and then any. Okay, So here are all on numeric data. So what we can do, we can go ahead and find the mean of the existing data, adjusting our numeric numbers, numbers, and fill the mean in place of the values are not missing values. So what we can do, we can use the x, that is the dataset or vector and we can find is an a of x. We can find all the NA values. And here we can put the value mean of x. So we can find the mean of x using the mean function and where, wherever any that aren't equal to true. So we can put the mean value in place off. We will remove the values and we'll put the mean value in place of the envelope. So let me run this. And now, alright, so the Z here, now we have the 123456 and in place of any, we have 5.33 and then 8910. And in place of this and every have this 5.3. So let me run this again. So here instead of an a, we have 5.33. So how we are getting this 5.33? We're getting the average of 1234561 plus two plus three plus five plus six, plus eight plus nine plus ten, and divide it by number of values when you get the 5.3. So in place of antibodies, we are putting the 5.33. So this is the one way of dealing with the missing values, are recording the missing value with that average value, that is 5.33. Next thing is DataFrame that coats the missing value as nitrogen some, in some of the dataset, the 99 will represent the missing value. So suppose this is the DataFrame that I'm creating, which is having two columns. And let's run this and let me show you the DataFrame. The DataFrame column one, column 2123. And here instead of five, it is having 99. And column two is having all the wells. So this 99 is out of place. It is like a missing value. So it is treated as the missing value. So now what we can do, one thing, we can replace this 99 with an a so that the, it will be in place. So if we want to replace this d F equal to 99 within a, we can use DFT and in bracket df equal, equal to, equal to 99. And if it is 99, we need to replace with DNA. So let me run this. See you now in straight up 99, these 299, we have the N A values, right? So you can replace with the inevitable. If you don't want to replace with any value, you can simply let me show you the DataFrame again with the 99. Now, we can put any values here. If you want to put five, you know that there should be five and you can simply put five and C. Now the values replaced both the places as five. So this way you can deal with the missing values. We can simply put DNA, okay, So this way we can record the missing values in us. 73. Decision Tree: Hello and welcome back. So in this lecture we are going to learn about the decision tree, which is very important in machine learning. So we'll learn all about decision tree and restart from the ready scratch that. What is decision tree? So first, let me tell you, what is our decision. A decision is something which we take every day. Suppose we want to go somewhere and suppose I want to go to some event. And to support this is me. And I need to, I need it to go some water, some event. So what I'll say whether the weather is good or not, and based on the weather is good, I'll go up. If weather is not good, I will not go. So this is dependent on the weather condition. So here I'm taking distinguish them the way they are. So this is kind of one decision tree that I am making. I'm making, I'm taking decision based on the weather and support. I can add one more thing if that is good. And again, I want to add something like if I am well and not well, right? So if I'm well, I'll go it will be yes. And this will be no. So this whole will become a big decision tree where the first thing at the weather, weather is good, I'm going, weather is not good, I'm not going. And if whether it is good Also app to see whether I'm good or not. And then better than my health condition, I will say yes or no. So this is one big decision tree, okay? Now, so this is how we take the decision, right? When we're driving a car, we are taking a look at it, looking at the traffic condition, and then we are taking our move right? If trap, if traffic signal is green will move. If it is yellow, we'll wait. And if it's red, we will stop, right? So that will also one kind of decision that we are taking. So what is decision tree? If you want to understand, we have to go to one example. Suppose I have, suppose this is me, this is me. And I want to go to some go-to by some fruits. And I don't recognize fruits. Okay. So somebody in my family, somebody, he told me that the apple, apple, apple has two characteristic. It is red in color and rounding save, right? And then he said that banana, banana is yellow in color and it's not wrong, right? Not round. It's long actually. Ok. And then he said that we have got it. Got it is orange in color. And it is not wrong. Okay. Not wrong. So now I'm going to duck fruit market. And there I saw a soft keyboard. I saw a shopkeeper who is selling fruits. And he has many fruits like he has a bunch of, sorry. He has bunched up banana support. This is the banana. So he has many bananas that, I don't know it is banana. Okay, So something in banana safe, I'm planning to make, sorry for my bad drawing. And he has some fruits. Some drum set proves he has. And then he had some orange color thing. Got it. Okay. Support that I have few gathered there. So now I'm going to stop and I'm thinking how forcing water lost based on what ever description I got from my family member, I'll go and ask the soft key for what question I'll ask. I will ask that what is the color, right? So first thing I'll ask, like Egypt or red or orange, right? I lost you Gita. Orange color. So the protease of orange color. And if it is, I'll get to answer straight with it will be yes or it will be. Alright. So if it is yes. Got it will be classified here. The S, the carrots will come here, right? So all the candidates will come here because thereof, orange color and all the bananas, because they are not orange color. So all the bananas will be bananas, it will come here. And older. Apples also on this side, right? Because they're also not of not of orange color. So now I have made one decision. The fruits, I have classified the fruits into two categories, orange or not. So orange, I got the, all the carrots and not orange. I got the yellow banana and red apples. Now, the next thing I'll ask around or not right? Round. So I get yes and I'll get no. Right? So far is all the apples will be coming here, right? All the apples will be correctly classified, right? And for no, For know what will happen. All the bananas will be coming here, right? Because bananas are not round. So I'll get all the bananas here. So now see, if you look at this thing, we can see that this is this entire thing. This entire thing is a decision tree because it looks like a tree, right? Tree has the branches and leaves, right? So this, the main, this one is called a root node. The root node and child nodes we're getting and digital lips, right? Okay. So this is one decision tree where I'm first, I'm seeing all the fruits and I'm deciding whether they're orange or not, then I'm classifying data that is being classified correctly. Then banana unipolar here and then are their own, then the apples and bananas are rightly classified. Classified the fruits based on their color and save. So basically decision trees are used for classification problem. So the centuries are used in classification problems, right? And the mean of this node, this node, this node, the root node here. This is called entropy. So what our entropy means, the, how the more will be the number of items in this root node, the more dense your decision tree will be. Okay? So what I need to do, I need to The each decision with the each node. Each node I need to reduce the entropy. So here entropy is here, then trapeze will be the highest. And then with the each decision I need to reduce the entropy value so that we can classify those items. Okay? So this is called entropy. So each eye trace and we will be, we will be reducing the entropy, correct? We'll see in the next lecture how we will be reducing the entropy. So here we can, we'll see how we can use that and drop it also with the titration and reach child node, how we reduce the entropy in the next lecture. But inside, this is the decision tree and this is how we classify things in machine learning using the decision tree. And this is how we make the decision tree, will also do our hands-on in the coming lectures. First, we'll go through all the theory part and then we'll be moving on to the hands-on part where it will be classifying your machine learning problem using decision tree in our programming. See you inside the next lecture. 74. Entropy And Information Gain: So in this lecture, we are going to look at how decision tree, how does decision tree walk? Okay? Less than three. Work. Okay. So for that, let me tell you a few basics before we move on. So first thing, this decision, decision tree, we are taking various decisions. So this one is, this one is known as the root node, or this one is known as the root node, right? So this is a root node. And then these are called leaf node. So this is leaf node. This is another leaf node. They are called leaf node, okay? Root node and leaf node. Okay? So now we know what is a leaf node and root node. Let me take you to the, another concept that is called entropy and did see it's very, very important. So let me tell you what I'm saying. I'm saying entropy. So what is entropy? Entropy is very important thing. Because see here, now this one, this is having more number of items, right? So this is having high-end dropping. Sorry. This is having high entropy. High entropy, okay? And this one is having low entropy. It's kind of population. So when you have more number of items in a node, it is having high entropy. And when it is having low number of elements, it is called low entropy. Okay? So entropy is a measure for missingness of your data collection. Like more number of items, like here. On the root node, you have the items with, with three colors, orange, yellow, and red and off to save. So that is having more messy and right. So that is having high-entropy. And compared to that on this root node is having a low entropy and this is having very low entropy. Okay? So let me take you to another concept with this and that is called, that is called information gain. What is it? It's called, it is called information gain. So what is information gain? As we move to this decision tree from root node to the leaf node, we are gaining information. And with the gaining information, what it is reducing, the entropy is road-racing. Suppose these entropies, for this entropy is E2, for these entropies, E3. And for this entropies support entropies E3. So for each I trace and you can say that E2 will be less than one, right? So y1 value is E1, E1 is more and E2 is reducing. E2 is less than Y1. And similarly here, e3 will be less than E2. Okay? So what is information gain? Information gain is information gain, if we want to calculate, it will be information gain will be equal to y1 minus y2. Y1 minus y2 will give you the information gain. Information gain is the decrease in entropy by splitting the dataset based on some conditions. So see here. Now we have 1234567 data here. And we are putting one condition, Agent, Orange in the protists orange color. And based on this condition we are getting, we are reducing the number of number of datasets right here, two and here 345. So we're splitting the dataset. So here the entropy is reducing. Information gain is the decrease in entropy by splitting the dataset based on some condition. We'll see how we can calculate the entropy. That is another mathematical thing that I let you know. But for now, suppose when we are moving down the decision tree, the entropy is decreasing. Earlier it was e one, now it is E2, and further down it is coming down to e three. And e one is greater than E2 and E2 is greater than E3. Information gain for this, from this node to this node, it is coming to y1 minus y2. Okay, so that is what information gain. So what is our aim of the decision tree? When we split the dataset based on some condition, we are aiming to decrease the entropy, decrease the entropy to gain the information, right? So main phone dies to get more information. Okay, So in the next lecture, we'll see how we can calculate the information gain and how we can do with the dataset, how we can split the data and how we can calculate the information gain. And we'll see what is the mathematical equation of the information gain with, okay, so see you inside the next lecture. 75. Calculating Entropy in Decision Tree: Hello and welcome back. So in this lecture we are going to learn about entropy and how we calculate entropy. So in the previous lecture, we have seen how does our distinct walks. And yet I want to clarify. The leaf node will be, this will be the one leaf node. This will be one leaf node, right? And this will be another different leaf node, okay? Lymph node, the end node. Okay, that will get talkie. And we have seen how we are when we go down deep down the decision tree, the entropy will be decreasing zone main aim is to get to the data node, to the leaf node, where the entropy will be less than the previous node entropy. And entropy and information gain. We have learned that y1 minus y2 will be the information gain. So now we'll see what is entropy and how we calculate entropy. So the topic for this lecture is entropy. Okay? So we're going to discuss in detail what is entropy and how we can calculate the entropy. Entropy. Actually, suppose this is the support, this is our one decision tree. Suppose this is our one, this entity where they say F1, F2, F3, okay? And then, okay, so these are our leaf north, right? These are our lymph nodes. And this will be the leaf node. Can the sport or the lymph node. And this is the root node, right? So how we calculate entropy? Suppose at one more thing. Just to recall. This decision tree. Decision trees are used for classification problems, right? Classification problems, right? So basically it will be the binary cross classification, yes or no. Okay? So suppose when we add from the root node here, we are coming to this F2. We are getting, suppose we're getting three years and we are getting two nodes, okay? And here we are getting four years and three nodes. And then here we are getting five years. We're getting two years, and here we're getting three nodes. We're just assuming these values, okay, and here for node and support, here we are getting three. Yes, and he had a very good thing for nodes and supported this one is having the railroad like ten years, seven nodes. Okay? So now how we calculate the entropy, entropy is the parity of the split. So this is our dataset and we are splitting into two nodes, right? F2 and F3. So how pure and how good is this split? That's what we're going to decide based on the entropy. So, entropy. Entropy is basically how good is, how good is the split, how good is your split? That's what entropy is going to tell us and how to calculate entropy. That there is a simple formula that supports we want to calculate entropy of S suppose, so it will be minus probability of yes. And then log message to probability of yes. You can say probability of positive as well, plus minus Alpha. Probability of no log base two, probability of No, let me write it correctly. That's going down. Suppose supports entropy we are denoting with E. Entropy formula is very simple. Minus probability of yes into log base two of probability of yes. Minus. You can do plus, minus probability of north into log of log of Pashto, probability of node. So this is the formula of calculating the entropy. So suppose we want to calculate the entropy of this node. So how we can do that? So here the E will be probability of yes, yes will be three. So three by total counter, three-plus to find. So probability of yeses three by five-year. And then we'll do log base 23 by five. And minus sign minus probability of negative will be two by five. By five. Log base two by five probability of negative. So this will be the entropy of, this will be the entropy of the note F two. So this is the formula. You will get some value here. Some value when you calculate will get the value. So this will be dark and drop you off this F2. Note entropy of F2, you can, right? Okay? So entropy of F2 naught will be this minus three by five. Poverty appears minus probability of negative and will need to take the log base two of those, positive probability and negative probability. So this, yes, you can say probability of positivity, probability of positivity. And this will be the probability of negative, negative values. Okay? So this way we can calculate the entropy value. So we can calculate the entropy value for this also, for this, also, for this also here, if you calculate the probability value, that is the only one, there is no negative, right? Because this is the N naught, right? So suppose we are doing the split and we are getting, suppose here we are, instead of 4.3 nodes, we are getting three years and three nodes. So this is a very, this is not useful, this is, this is not a useful thing, right? Because we're getting both the things, three years and three nodes. So this is not going to give any insight, right? So this is very negative. This is very, very bad kind of split, okay? So your data will not be splitted in such a way in a decision tree. So this is discontinuity or hockey, because when you calculate this one, it will be three by six minus three by six. Basically the boat negative probability and partially probability both are same, okay? So we have to split it in such a way that we should not be getting this thing. Okay? So this is how we calculate the entropy with this formula. Okay? And we have to, and information gain as discussed, will be the, suppose this one is E one or E F1. And this is here EF2. Information gleaned from F1 to F2 will be E F1 minus F2. So this will be the information gained from node to this node. Okay? So this is how we calculate entropy and information gain. Information gain. 76. Calculating Information Gain for Decision Tree: Hello and welcome back. So in the last lecture we have learned about entropy. And the heel are in this decision tree that I have drawn. Abdullah, small mistake like Yolanda. Total cities are firing 17, 17 items then S. And there are no. So when you split these two nodes, three years to know. And for yes and three node, right? Four plus 37.5, 12 only, so five is not included. So you can so you have to take care of that. I like by mistake, I have not been given the correct numbers here, but that is not going to affect anything that I have taught in the previous lecture. So only the numbers are a total of 17 when you're splitting. Okay? So apart from that, everything is correct and the way we calculate the entropy, that formula is correct, everything else is correct. And that's what I wanted to clarify this to happen. Some other numbers. Okay. And like three or eight years it then and then okay. In this would be my mistake. I've written 38 yes. To know 10.7, 17. Okay. And then here is to be a support eight years. And yes. Or McDonald's right here, then it will be correct one. Okay. Okay. So now let's move to the next concept that is called information gain. So in the previous lecture, what I have told you, I told you in farmers and gain is like this in entropy minus this entropy. So this information game you'll get from, from this feature F1 to F2 rate. But that is not the total information gain. So when we get our decision tree fully formed, we have to find the information gain of the total decision tree. What information game? Again, we are getting in totality. So let me just toss like Dawn and what I'll do, I will create a decision tree. Suppose we have this decision tree. So supported this stage, F1, F2, F3. Okay? So I suppose this one is having n, yes, and seven. Okay? And when we are splitting here, it will be suppose ETS, nose. And here we're cutting. Let it be this three nodes. So level and here I'm getting three years and three. Nope. Okay, So this is one, our decision tree support a sample decision tree that we are creating here. Now, for this node, entropy is suppose E F1 support you. We are calculating the entropy for this, then. Yes, n, seven nodes, we're getting E F1. For this, we're getting and property as E F2. And for this we're getting E F 30. Okay? So now this is based on this information gain. And entropy will do. Entropy will just tell you based on entropy value, we can come to know that how is our split, right? Will not be knowing that whether the entire obese would go for this one or this one, right? This split, or this split, right? So suppose we can make another split, like starting with F2, start with F2 and then go to the F1 and F3. So this could be another split. So we don't know which one is correct, right? So how do we know that? Which strategy or which split we should go visit would go like F1, F2, F3 are ways, would start with F2, F1, F3, right? So how we can decide that? We can decide this by the term informed me some gain. So based on the information gain, we can decide. Which way we should go with, sort of go this way or this way. Okay? So this will be decided by the incremental gain up the total decision tree. So how we calculate the total, total information gain, and that's what I'm going to tell you. Information gain is how much information we're getting for this entire decision tree. So for this first thing, what we need to get, we need to find the entropy for each of the node, each of the feature, okay, so far supported, we have calculated based on the formula explained in the previous lecture. We have calculated and we got these values, E of one year, PUI of three. Okay? So now what information gain? Information gain. What will the information gain of this decision tree? It is like the information gain up the root node, that is E, f, f1, f1 minus sum of all those splits. Sum up all those splits like this one. And this one, the information gain and drop you off this. Okay? So what it will be, it will be those subset of all the sub splits like this is one split and this is one split up the same node, right? So some of this subset, the weighted weight total, okay? And then E, F support. This is going from z equal to one to n support. We're going from n number of split to E F1 fn, okay? So E F1 support, we have calculated E, F one. So what will be the how many subsets we have here? We are getting two subsets. One is this and one is this, right? So what will be the subset of this eight plus 31111 and total is 17, 11 by 17 into E of f, two. And then plus three plus 36 by 17. Total. Count is 17 to E F three. Okay? So this will come to the E F1 -11 by 17, E F2 minus six by 73. Okay? So this formula, this, this expression will give us, this expression will give us some value. Suppose I'm not calculating actual value, you can calculate. So suppose this value we are getting 0.53. And in the similar way, suppose we have calculated for this decision tree as well. Okay? What does this entry? And here we are getting the information gain as wind for one. So now we are getting, suppose this is again one and this is again to saw how our machine-learning algorithm, or how will decide which way we should go. We should go with this split. This split based on the information gain. So here we are getting, again, I support 5.53 and here we are getting 0.41 gain, information gain one is greater than the information gained, two. So we would go with the higher value. The higher the information gain, the more accurate will be the decision tree. So we'll go with that, go with this decision tree. So we'll go with F1, F2, and F3. Information gain will decide which decision tree or which approach we should take in creating that decision tree. And this we will not do manually or machine-learning algorithms. Or the library is they are going to do it for automatically for us, but we should be knowing how internally it is doing. What whenever you are trying to create a decision tree for some problem or machine learning, machine learning algorithms will create these subsets, these plates. And they will calculate the, that will calculate the information gain. And whichever information gain is more, it will go with that strategy. So this way, the information gain is useful in deciding which split or which a decision tree with follow-up based on what feature we should split first. Feature one are based on feature too. So that will be the very for information gain is one such important thing. We should be knowing the math behind the information gain. Support these three, yes and three nodes here, the entropy value of three will be one. Why? Because this is totally impure subset, right? Because this is not, this is not good. Because three S and three note that both are same numbers, right? So this is impure split and for this entropy will be 11 thing I forgot to tell you. Entropy value will always be in-between zero to one. Entropy value will be always in 0-1. If you are getting 10, your split is good. And we have to stop the split there. Okay? And if split is, your entropy value is coming one, that means it's a pure or impure split. Okay? So for zero we'll stop and on one it will be impure split. Okay? So that thing you keep in mind. 77. Hands on Decision Tree in R: Hello and welcome back. So in the previous lectures, we have learned about decision tree. We have also seen how we create a decision tree. And we have seen entropy. What is entropy? How we calculate entropy, and what will be the entropy value. And then we have seen how we calculate information gain and how information gain of decision tree will decide which approach we should move on, which approach we should adopt. Book Creator, best decision tree for our problem statement. And we also know that this entry is used for trees for classification problems. Okay? So basically a binary classification problems that we deal with the decision tree. So in this lecture we are going to learn about how we are going to, basically we are going to write our code to create a decision tree for our dataset. So here we're going to use in-built data set that comes with R, that is reading skills. So what this reading skills dataset is having and what, how we create our decision tree for that. That we'll see in this lecture. We'll write the code before that. Let me tell you some applications of decision tree. So decision tree, if you want to see in a real life, You, decision trees are being used when we are trying to find which email is spam, email, whichever email is not spam. So there we are taking disjoint, right? So that kind of machine learning projects, where we are, we need to decide an e-mail, it's coming from a source, whether it is a spam or not, there, we're basically creating a decision tree. We are taking this journey, spam or not. That kind of binary classification problem is spam or not spam. Then in the health care department or health care sector, if you see, detects an app cancer, whether it can be a tumor, can be, uh, cancerous or not, that we can decide with the help of machine learning and using decision tree. And in the financial sector in Europe, you can see, we can, like in the BFSI banking finance domains, they maybe use it for deciding whether they sort of give loan or not. And they can also decide whether how about or what is good about our water bad borrower based on their credit score. So like credit risk analysis, you can do with the decision tree. So these are the, some real life applications of our decision tree and machine learning problem that you can think of solving with the decision tree. So how would we solve this? Basically, we create a model and then we'll try to train the model. And then based on the trained model we tried to provide on the test data and try to decide all those things. So now, come to this problem, this lecture. Coming back to this lecture, we are going to use the reading skill dataset and that comes with R, The R package. So you're saying, well dataset and this dataset actually describe the score of someone, someone's reading skill, someone's reading skills. If we know the variables like age, suicide score, and whether the person is a native speaker or not. So if we have this information, is source I score and native speaker a naught, we can decide. We can find a score of reading skill or black person. So we are going to create a decision tree for this. Okay? So for in our programming, we have S3 function. S3 function we use for creating a decision tree. And this takes two input. One is formula and others did. So. C3 is taking two formula at the input and our data. We'll see in the code. Okay, so next thing is, next thing is let's write the code. So first thing is to like create a decision tree. We need one package called party. We need to install this package. So if you have not installed this package, you have to write the command install dot packages, installed packages. And you have to provide the package name. And then you have to run the code. And this will install this party package for our use. So I have commented this out. If you have not installed, you just uncomment it and run it. The party package will be installed. Once it is installed, you just comment it out so that this command will not run each and every time. Okay? So once we have the party package installed, this isn't necessarily for creating a decision tree because this S3 function comes with this package. We need to use this package. To use a package in R, we have to use library and then we have to provide the package name. So packaging Mh parties, and that's why we are writing library party. So now this library party will have this reading skill dataset cultured that so we can see what are the things are there with those datasets. So we can simply run this head. Head reading skill will give the first few. First few are datas from this reading skill dataset. So let's run this and see here. Now we're getting the first six rows of the dataset. It is saying native speaker age, suicides, and then the score. This is the basically the reading score. So now we have a glimpse of this data. We can move further and we can create a decision tree. So here we'll use input dot that to take the input. We can simply, if you don't want to put this name, you can simply use input. So here I'm using input dot data, and I'm taking the data from this rating scale. And then we'll move further. And here I'm just using a decision tree dot PNG. And this will be the image file that we'll have our decision tree printed on it. And then we are creating the tree here, output dot tree. And we're losing the S3 function here and here, we are providing the formula formalized native speaker. And this is the, this is the, this is the dependent variable. And based on age, suicides and its core, we will decide whether the native speaker or not, okay? And data is, is equal to input dot data. And then we are plotting the, plotting the output directory. So I'll put the tree we're getting from this function. And this is the formula, native H plus plus score based on these three predictor variable, we are going to get the native speaker value, okay? And then theta is equal to the input dot data. And we're plotting this, or productivity or whatever we are getting from this concern, we are plotting it. So let's run this and see how we're getting that decision tree. So see here now we have our decision tree file created. See here, this is the decision tree that we are getting. Okay? So we have not decided how this, oh, really, this leaf node, right? The system has automatically decided. And T are less than or equal to 3813 zeros x it is coming here, 30, less than 38.306 voltage coming here. And then it is equal to R7. It is less than six that is coming here greater than c because of this coming year. So this distantly is being printed, are being created with the S3 function. And here the information gain and all will be done by dy system, okay? So we need not to do, we just need to use the S3 function and probably the formula and the data, and it will create the decision tree for us. Okay? It is that simple. But why we have discussed the theory in the previous lecture, because we should be knowing the behind the scene what is happening with this S3 function. And we should be aware off, okay? Then only you can succeed in the machine learning and data science field. You have to know the mathematical intuitions behind the decision tree or any algorithm you are learning. Okay? Anybody can come here and use this S3 function and create a decision tree. But they may not be aware of how the decision tree is being constructed. What is entropy, what is information gain? And all right, so that's why you should be knowing the real intuitions behind the machine-learning algorithm. Okay? So with this decision tree, what conclusion we are getting? We are getting the conclusion that like anybody who's reading scale is 38.3, less than 38.3, and age is more than six, is not a native speaker, right? So with this, we're getting lake. If age is less than, reading score is less than 38.3 and age is more than six, then the person is not a native speaker. So this is how we can create the decision tree in R. 78. Advantages and Disadvantages of Decision Tree: In this lecture, we're going to learn about advantages and disadvantages of decision tree. So first we'll see advantages, and then we'll see this advantages of decision trees. Okay? So half the estimates decision tree, okay? So this isn't really, as we have learned, is a very popular machine-learning algorithm. And distinctly solve problems of machine-learning by transforming the data into a treat a presentation like this. Okay? Treat a presentation like this. And each internal node of the tree like this. And they present our denotes an attribute and each leaf node denotes the class level. And decision tree algorithm can be used to solve classification problems and regression problems as well. So it's not like when leaked classification problem can be solved with the decision tree. But we can solve the regulation problems as well. Okay? So this has been d t, Okay? So this entry can be used both for degradation as well as classification problems. So let me tell you what are the few advantages of the decision tree. So the first advantage of decision tree is that it requires less afford on it requires less important preprocessing operator. Let's apart on preprocessing of the data, okay? So for decision tree or compared to other machine-learning algorithms, you need to work a little less on the preprocessing of the data or data preparation. You can say, I'd like a pre preparation, or you can also put here like data preparation. So you have to put less effort on data preparation and data preprocessing, okay? So that is one big advantage of machine learning. Our product. The second advantage would be like a decision tree does not require scaling up. Scaling up data is not required. Scaling of data is not really required. Okay? So this is also not required and that is one big advantage, okay? And then the third advantage, we can say that this generally does not require normalisation of Beta. So even if you're not normalized the data, you can work with the decision tree. Normalization of greta. It up, not required. Okay? So this is also one good thing in machine learning and we'll see that decision tree. So in machine learning, the centuries and algorithm where the normal adjacent and Gil link up the data is not really required. If you want, you can do, but not really required. It is not going to have big impact whether the data is normalized or not. If we are working with decision tree. The next advantage I could be missing values in data also not have big impact on the decision tree. Okay? So like if you use other machine-learning algorithms that you have to work extensively to deal with the missing value. But in decision tree, this is not going to have, not going to missing value will not have impact on decision tree. No impact, okay, So not going to impact much. So even if missing data is there, You can go ahead and create a decision tree that is not going to, you're not going to miss on any thing that is going to affect your decision or impact your predictions. Okay? The next advantages decision tree model is very intuitive and easy to explain to technical teams agile at scale holder. So when you see the decision tree, it is very easy to lead you to understand. It is very easy to understand and any nontechnical or technical portion can easily understand. So this is the big advantage of a decision tree. So even if you want to present your decision tree to some Um, management guy, you can easily make them understand the decision tree. So that is one advantage of a decision tree. Apart from that decision tree. Or like very human-like intuitions you get with the Get with the decision tree. So it's very, very similar to human behavior. We also in our daily lives, we take the similar disjoint yes or no, based on some features or condition we take decisions. So it is very human-like. I'll be happy on a human-like algorithm. Okay, So these are the few advantages that we can count on for machine learning. Now let's look at the disadvantages of the decision tree. So the very first decision tree disadvantage that we can see is it is very, not, that part's reliable if your data is changing. Okay? So what we can say, like it's an instable, like in stable. And if the data is changing, if the data changes. So support you, you have made a decision tree and there is a small change in data. Small change in data can, can have big impact, big impact on decision tree. So that is one big disadvantage of the decision tree. Even the small training data can cause large amount of change in the structure of them. Decision tree support this decision is looking like this. If you change the data a little bit, it may have been entirely different structure of the decision tree and that is going to, that, that is one kind of big impact, right? The second disadvantage is like decision tree sometimes go for more complex compared to the other algorithm. So sometimes for the simpler data, your decision tree will be a complex. So sometimes we get very complex decision tree. And if you use some other algorithm, you may get the simpler solution. So you have to see whether your, for our particular problem statement, you really need a decision tree or not. And if you're using decision tree, it is simpler than the other algorithm or not. If it is not simpler than the other algorithm in machine learning, you should go with the other machine-learning algorithm and you have to not go ahead with the decision tree. So sometimes it's really get complex. Neck thing is like training of a machine-learning model with decision tree is huge, right? So it takes time, stick time to train our machine learning model using decision trees. So it is time taking. Process is take huge time for training decision tree. The time taken by distantly is more. So it is time expensive, right? It will take more time compared to other algorithms. And it is also like a complexity is also increasing in the decision tree. Other thing we can say that lesson or not really very impactful. So it is, if you use a decision tree for classification problem, classification problem, it is good. But for regression trees, it is not that impactful, right? As I understood, but it can be done for both. But regression trees are not that impact. So these are the few disadvantages of the decision trees that we can count on. So that's it for this lecture. These are the advantages and disadvantages of the decision trees. 79. Project 1 Introduction: Hello and welcome. So in this lecture, we are going to learn about the project that we are going to, to from. So we're going to do are very important. It simple machine learning project that will give you clarity of what machine learning simple algorithm can do. In few lines. We are going to use our programming and we're going to use our dataset, which will contain the previous data up the organization. And based on the data, we are going to predict the future stock prices. Yes, you guessed it right? We're going to do a project where they did on the previous data, previous StockData of historical data of the stock prices. We are going to guess the future stock prices like from now. What will be the stock price on 100, 500 days, or after one year or two years, after 30 days, after 40 days, what will be the stock price? So this will be the stock Prediction. Project, him machine-learning, okay? And we're going to use our programming for that. So simple XML file will contain the stock price. And for this we're going to use Google stock price data for Google. And we'll break the Google prices on the 400 days or 100, the days are days, okay? So stock price for X and project we are going to do. And for that, we have this data set in Excel format, which is containing the historical data of the Google stock prices 2019-2020. Okay. And see what are the columns here? The first column is date, then starting on 26, September 19. And we have data till 25th, September 2020, almost one year of data we have, okay, And then the second column is D, and then the third column is open. This means the open price. When the stock market opens in the morning, what was the price? So this is the opening price on day six, septum auto toggle 19, which is the day one. What does and then on that day it went 1,245, which is the highest in a day. And low wished it went to 1,232. So in a day started from 1,241.95. It went to the law of 1,232 and it went high up to 1,245 and it closed on 1,241. So these four are detail how very important and adjusted close is the same thing almost. And then the volume, like 15 lakh, 30,000 volume was there. So like that, we have all the data for 365 days. So based on this data, we are going to predict the future stock prices up the Google. If you plot this data on a graph, we can see how the prices are moving up and down. So you can see here. So we're going to plot out this kind of plot as well and see how the prices are increasing and decreasing. Dys, okay? So this is the project we're going to do. And let's see, in the next lecture we'll start importing this Excel file into our art studio and then we'll write simple code and we'll try to predict the future stock price up the Google stocks inside the next lecture. 80. Project 1 - Predicting Stock Prices: Hello and welcome back. In this lecture, we are going to start with our project that is predicting the stock price value of Google stock on a particular day based on the historical data that we have with. So we have this Excel sheet, which is having though past one year data like 26, number 20192, up to, I think September 2020. Yeah. So up to 25th, September 20 countries. So we have almost one year of data of Google stock price is with us all 365 days open price. What was the high value of the stock on a particular day? Lowest value. What was the closing value, and what was the volume? So all this data we have, based on this data, we'll try to create a model where we can predict the stock value on a particular date, okay? Suppose on a three day from now, or 500, 500 days, what will be the stock phrase? I'd say 100 days. What will be the stroke phase? So like that. Okay, So let's start the code. The very first thing is what we can do. We can import the Google stock price dot XLS, the Excel file into our studio. And how we can do that, we can go to the file. And here we need to go to the Import dataset. So we need to go to the import data set. And here you can see from tax, from the three NADH from the Excel, from various data sources has been listed here. So what we need to select here, we need to select that Excel here. From Excel, we just need to click. And then here we need to browse through the file where we have kept our data. So we have this Excel file. So I'll just open it and digital trimming the data. And once it's done, it will give us the import option. So this is one way to import the data file into the hardest studio. That is another way. And that also will see and see here now we can see the code preview of this import. It is coming like library. I don't need Excel and Google stock price will be the object. And here we will use the Read Excel function and will provide the bot to the Excel file with the filename. And then we can use the view, this object. Okay? And this will be so we can simply copy this code. And I don't know why it is taking this much time. There might be some MRI, so let me cancel this and import. See. Now, the data has been imported to our studio hand. Yeah, we can see the same data which we are seeing in this Excel file. Now, we can see inside this art studio. Okay, so now we have the, now, if we want to view this data in here, see yet there is no Claudia. But since we have imported, we can use the real view. And here we can use the solid object name your hand. We can run here and see the same thing will come. Even if you want to use summary of the data. Somebody we can run this and we'll get somebody up this dataset here. Mean, first quartile, median mean, third quartile maximum value on the quantity of September 2020 And it was 1,710, maximum is 1,733. That will be the highest. And this is the maximum of opening maximum. And this is the highest maximum nor local maximum and closing maximum. Okay? So this way we can see the data summary, okay? Now, instead of using this, we can also use the we can use the library. We can use the code library. I don't know why it's coming. If you don't know the code, you can simply go to the File Import Dataset and from Excel. And here you can see the code here. Just copy this and we just need to paste it here. And yet you need to put the Read excel, and yet we need to put the bot name. Okay, what will be the part here that we will put? Can just put this. So we are rearranging library Read excel, and then we're using object name rather than using the Excel function and providing the bad part XL file. And then we can use the real Morgan stock price and it will give us the same result. This and see, yeah, we're getting the same thing. Okay? So this way we have read and view the data in our next thing is we need to understand the structure of data. So if you want to understand the structure of data, you can use the summary function, summary of this Google stock price. And you'll get the summary of the data like we have already seen. This way. Next thing is what we need to do. We need to visualize the data. So next thing is what we need to do. We need to visualize the data, visualize the data. So for this, what I'm going to use, I'm going to use them lot. So what I'll do, I don't know why this is coming a huge plot. And I'll use that column open and use d and then data will be okay. And then just run this. The oven, we run this, we'll see the data. In your lesson. We will see the plot here. So here that is opening price and this is the day 1-2, purpley. This, we can see the data, how their stock prices are sending weekend. You visualize here. So this way we can reach later, later. Next thing is what we need to do. We need to use the linear regression to predict function. So what will use? I'll use the predicted price. And for predicted price, what I'll use a hilum Fung cell and this lm function will, I'll use on the same input I'll give open day. And it was done in this dataset, okay, Excel file ALU data, dataset. And then I'll just bring to the predicted price. So I'll just put in to the higher price here. So let me run this. See here it is saying the core present and this. Okay, so now what I'll do, I'll use the summary function to see what we are getting for this predicted price. So far, this predictive price, minimum, first quartile, median, third quartile, all these things we are getting. Now, what I'll do, I will predict the door. Predictor stock value on a particular day. Okay? So we can do that. I can use the predict function. And I can use the price. Yeah. And then what I can use, I can use data dot frame. Here. I can provide d is equal to 350, okay? And this will give us, this will give us the predicted stock price on day three. So let me run this, see here. On the death rate per period, it will be, the stock price will be 1,620 point for it. If I can this for 50. Thus stock price will separate. The stock price will be 1,700. Grantee, if I put 150, it will be 1,400. If I put a one, it will read 12, 30,000. See, the Google stock price will be almost doubled from the day one. So this way we can put the d value here and we can predict what will be the stock price on that particular day. So it is pretty simple project that you can also try. You can predict the support. I want to predict the value on one, please. Second day. One purpose secondly, this way, okay, So this way you can also try on your own dataset or you can use the same dataset that I can provide. You can download the dataset and you can try and you can predict the stock price. You can also make an Excel sheet of your own and put some random values. And you can put your own values here and you can predict, you can take any other dataset which is having the or some other values that also you can try and you can put into the same code and you can try to predicted value. Okay? So this is a simple project using lm function to predict this Google stock price. I hope you understood how we use the machine-learning and all these simple functions in R to predict the stock prices. Thank you. 81. Project 2 Uber Data Analysis using R: Hello and welcome. So in this lecture, we are going to learn about data analysis through visualizations in R. And for this, we're going to do a project in which we are going to use where data, what is the gap aggregator, service provider for getting the right on a cab x evoking app. You can see. So you either number one in the wall for a cab service provider. So we're going to use the UPA data for data analysis for this product. So the motivation for this project is to conduct data visualization, data storytelling, and how we can use the data to analyze it and get useful information from. This is an important component of machine-learning through which companies are able to understand the background of various operations. What does it mean? It means that suppose what is done in the gap aggregator service and mini cab drivers are attaching their gap two distinct Over app. Then I stream and the uptake in providing the right to the customer. So huge that is registering to the app and then he's trying to big gap based on their location. So what do you what does as soon as you log in and select your location and your destination where you want to go. Based on your locality. It will list all the nearby caps that can these to you in the fastest time. It will. It will. It will. It will store all the gaps near you and then be gender location and your requirement, it will look best for you, which can take you to the destination for the quickest time. So how does it do it? You just are lots of AI, machine learning and data science concept to analyze the data and give this a recession. So in this project we are going to use the data. And we're going to analyze like how many are happening in a day or many monthly rates are happening in a day. How many of your LEA that I start happening monthly lives and DY is what time of the day the VGS, that kind of analysis we are going to do in this project. So all these kind of operations we can do with the **** papa, do with the help of machine learning. With the help of visualize and companies can avail benefit of understanding the complex data and gain insights that will help them to crop this isn't. So The visualize part is very important because with the vigilance and we can quickly analyze the data and get really useful insights like what time of the day is the BGN? Which localities having less requirement of the cab where the users are not getting cabin time when it is lagging. All those analysis we can do with the visualization. This is more of a data visualized in a project that will guide you towards using the ggplot2 library for understanding the data and for developing an intuition for understanding the customers who are in the clips on what app. Okay, So this is the, this is the thing we are going to do. And the goal of this project is to learn visualizations in R and analyze the data. So let's get started. So before we proceed, we have the UVA data or files with us. And these are like this is this is the data for afebrile month. And this is having four columns, date, time, latitude, longitude, and based. So these are the information you buddies capturing. Okay, so date and time, longitude, latitude and longitude. And latitude will give the location of the gap and time or at what time they're booking and the base from where the best location for that. Okay? So these are the data we have. And this is the same way. We have a May, June, July, August, and September, the month data we have. And we're going to analyze this based on this data. This data is pretty simple. Date and time, latitude, longitude And so let's go to the file. Yet I'm using R for analyzing this. So before we proceed, we are going to import fruit library, that is GG plot two. Okay, then lubridate and tidy up DT skills. So what is GG plot to GG plot twist. The most popular data visualization library that is most widely used for creating aesthetic regulation plot. So GG plot is very popular in our programming for creating very aesthetically good-looking and very off-putting and very appealing visualize and gloves and plots. So we're going to use the GG plot two. Then we'll be using Lubridate. It is this leverage for the timeframe. Using the timeframes in the dataset. Then we have the deep layer is the data manipulation library that we're going to use. Dres and important library, it will make your data very tidy. Does our tidy data means tiny, determines the data which is easy to work with. The data. If it is easy to work with, we can use it in the other library this easily. So we'll use the tidy R library to make our data tidy. And then we'll use the nitty, the data tables in JavaScript library we'll be using. Then really reaching skills, skills. These libraries used to, with the help of the graphical, with the help of the graphical scale, we can automatically map the data to the correct skills with the ****, with well-placed x itself. So this will help us in that automatically map the data to the correct scale. So for this, we need to use this library. So before that, if you have not installed these libraries, you can go to the Tools here and then click on the install packages. And here you can simply popular packages named yuk, like you can put GG plot two. And then if you want to all together, you can put GG themes and you can put coma and write all the names, luminary dates. Okay? So like this, you can do, and you can click on the install this library, this for you. So like this, you can install the packages. So I'll repeat it again. You need to go in RStudio to go to the top menu. Here you will find the tools. Click on the tools, and here you will see the various options. In Rhodopsin. You have to select the first one, install packages, and then you have to give the package name GG plot two. And if we want to install multiple packages at the time, and then you can put comma and give the packages names by putting a comma between them and then click on the stock. I have already installed this package is so I'll leave it to agitate. And if you have not installed, you just click on the Install and it will be installed in rural Australia. So let's proceed further. So once our lab to use them and to use them, we we just we need to tell the hardest to do that. We're going to use up these library GG plot rho, g times lubridate, tidy, tidy verse, DT and scales. So for that we need to import the library. And then in the small bracket, we need to put the library name GG plot two. And all the libraries that we are huge, we need to write in this manner. Okay, So once you have written this, select all these libraries and click on Run. And it will be ready to Jews in Europe grounds. Now the next thing, what we'll do, we'll create the color vectors, vectors of colors for the plots. In our plots, we'll use the multiple colors for the plots. In our plots, we'll use the multiple colors to make it more appealing and aesthetically good-looking. And for that, we'll create the vector of colors. So here I'm creating a vector colors and I'm giving C, and then I'm giving the color name, the ACCC 10166 equal type. So all these gray Green, yellow, blue, pink, all the scholars we are going to be used. So these vectors will create and will run this. Okay? And then what we'll do, the next thing is we'll read the data from each timeframes and eat. What we'll do. We have the data file for each month like appraised May, June, July, and August. So we'll do this using a read dot csv. You know that for reading a CSV file in R, we read dot csv method function, so the dot CSV. And then here we'll give the path for our data file. So we have stored data file in this part, so we'll give that and the file name will be at the end. Okay, So now we'll treat all these monthly data. Okay. Just click on that and see. It is being successfully read for August, September, and see you running. So now it is done. And next thing is what we'll do. We'll combine the data together. Now we have read the data and we have stored in these variables. Mean June, July, August, September. Now what we'll do, we'll use the onBind method to bind the data together. So now all these six months dating one. Okay, so we'll use the data here. Odd bind method, and we'll just pass these objects may, June, July, and they will be concatenated together. Okay, so let's run this. So now we have the diamond sense of the data lake for columns and the many roles. Okay, so now we want to see how the combined data is looking. So we'll use the head and will provide the object data here. So it will print the first five columns of the data. Okay, Come my senior, now, we have this data for the time, latitude, longitude. And the next thing is, we'll use this data contains columns date time with this data contains columns, date time, which is a factor, and longitude, latitude, which are double, and basically factors. So we will format the datetime into them readable format using datetime, converge and functions. So here we are within the datetime convergent foxing function here to convert this datetime to a more readable one. So for this we'll use data and data date, the date, time, and then reload as dark CD, time format, year, month date, year, and hours, minutes and seconds. Okay. And then when you read this format is this one and Reform Act in the hospitalist. Excellent. Okay, and then we'll combine this. Okay, let's run. This. Script is still running. When this disappears, you can move to the next. Just run this. So it is running. Next thing what we'll do, we'll create individual columns per month, day, and year. And for that we'll get data day, month, year and a half week. And we'll use the factor here vectorally. And yet really take the time. In fact, our month lead time level through and factor weekday time on data. Okay. Okay. And we'll run this as well. So now it is running. Yeah. So what we'll do, we'll create the neutral columns one month. The next thing, what we'll do, we'll add time variables also. So part-time are able to do second minute INR. And here, the factor material than second Aspen a second and ask them the second noise vector. Second vector mina can take the RN window. This will run this as well. So it is running. Next thing when it is done, we'll look at the data. Data. We'll use the method to look at the data how it is looking. So let's run this and see now we have the daytime land, latitude and longitude. These are the four columns already there. Does it. Now we have created new columns like time, day, month, year, day of the week, second minute. And so these are the columns 123-45-6788, new columns we have created. And these columns will give more insight to auditor. Think it will, it will say like God, only, that now we know which day of the month, which month, and which he had a Richie Rich de, day of the week also will come to know. And then second created. Now. Now we are done with the Nita walking on the data and creating features, new features, new columns here. So next thing is, we'll move on to the data visualizer and part randomly create the plots and graphs and we'll try to utilize more. Here. What we'll do the first thing, first thing we'll do is plotting the drips by RT and RD. For that, what we'll do, we'll use underscore data are valuable and what we'll do data grouped by R and use the deep layer to summarize it. And when we run this, now, now look at this. Now look at this. Now we are getting this data zero or one or two or 3 h. We're getting data table we have eat did you can select how many trees you want to see at a time. Okay. Next thing is the source data in the table yesterday. But now what we'll do, we'll plot the data by our GG plot and GD plot here. The only data that we have created here. And as our li. And then we'll use geometric bar identities. Still, still feeling with this teal blue and the color will spread. Then titled review types by r and subtitle aggregated. And legend position none plot, title of the text and how 4.5 and 0.5 and a skill like continuous. Okay, So now we'll plot the data by our sun this C, and now we're getting the plot by our trips. Every RM here subtitle is aggregated products today. So you can see all three or 24 h we are getting that trips every audit this toy I've started, it was these many tapes. The second third, dark or dark like this. They swim club for 24 h and be done with this, we can see that the 17th are in them or having them more number of trips. So the beak are charges if they want to apply the middle applying from the Philippines are forming our product. Midnight the second or per liter. So these, these, these many things, these things, these times are very VCRs and here, when you see in the evening time, The Office and ends, they will apply that they will apply they will charge the bigger charges and they will increase the cabin. So this is how they do it. By this we'll find, reach out the Rs are ready VD and based on that, we'll get into rice. So this is how Uber does that. Now, do the plotting tips by r and month and aggregate our data and really use the group by function here, month are the declared to summarize it. So try this Then we'll do the GG plot to plot it. So C and now we are getting the plot here, which is depressed by n months. So see here, this one, this color, this is for real, this is for me. This is Module in July, August, and September. But our every month they destroyed. So which month is having more or less the pKa is similar in all the months. More or less, not exactly, but more or less you can analyze. So apparently September also, Muslim man up right here, Castile soap and water each month. The beaker is remaining synthesis. It is more or less same way it is going forward every month. So what summarized some guys as a group by month and you can override this bankruptcy. So now what we'll do plotting, I will plot data by trips during every day of the month. So for diet will use the data as a valuable and group by day. And we use the Deepak to summarize startups. Run this and then that easy plot to plot this. So this will soda and chips by the optimal. So now when you look at this, it will give you enough number of trips per day. There's many types second day like this, so deaths by day of the month. We can do like this. Next thing, what we'll do is collect data by day of the week and months. So here we'll use the group by day of week and month. And really the deep layer to summarize them, blood topic. So often run this. And after that. The DD plot took lot depths by day and month. So when we do this, we will get gifts by day and month. And he had to be happy with a color scheme that we have in it and at first here, okay. Now, with this, what we can get, we can get that for days of the week like that. For days of the week. Sunday, Monday. For every month. How many lives are there on Sunday? How many days are there? On Monday? How many days until day? How many days? Wednesday, Thursday, Friday to each day of the week will give you the number of rights, but every month. So with this, we can quickly analyze it for Sunday. Apple has the lowest number of fights and this month set them what is having the most number of rides on Sunday? On Tuesday, September, having more fun, more uprights. So like this, we can find the most popular day of the month quickly. Now we'll do a number of trips placed a month, a month, inner ear. So for that we'll use group by month. We didn't month a month in a year. So for that we'll use group by month. And then we'll plot using GG plot. Run this and see you're now we're getting that trips in a month. So did many trips but may they may, June, July, August. So it is so ET tube. By looking at the graph, you can say that September is having more number of trips up redact August and July and May and June not having almost similar types, almost. Even though Joni slightly better than me, And then apparently performing months in a year. This way. Now, what we'll do, we'll create a heat map, DR and month. So for deck will group by day are many trips. Excellent one like this. Now we'll plot the heat map by R&D renewed GG plot it. I will use this battery will just yet can you take a bite? And yet when they are filled with the proton on particular day, particular class, how many total, cumulative keep white. Let's just run this here. Now, we are getting this heatmap Our map by day and so our day. And for this art on the eighth day, this into 11 times 11 today. And for this we can analyze more. Now when the heatmap plot D and month. And then this, and then we'll GG Plot lot this. Now we're gonna kinda keep my fun month. And the next thing is we'll plot the heat map T of the week and month. This, so this is the heatmap for month and day and month, month. And I'm access to some lemon later today. With this also we can have now we'll create the mutilation relation for pricing in Europe. So minimum, maximum, minimum and maximum likelihood will get NYC. And here what we'll do, we'll try to plot these two agenda. Gg, GG plot. So here GG plot, our knowledge of dramatic lighting, total value inside one color and the skin, minlength max and Min max likelihood we are providing. And then we're back and we're using GG plot to plot this eigenvalue and lysine map based on Sundays. It is taking some time. It's still running. Let's wait for it to come off here. Just taking a long time. So let me show you the map, that map father NYC to type, but in a practical September. So here it is taking some time to so here we have used up all my data prior to September for 2014 to analyze and which later Uber rides up based on the time. Based on that, we'll come to D which month is better. So all these visualizations we can and using the data. So I hope this project will help you in understanding visualization and data analysis 82. Project 3 Customer Segmentation using R: Hello and welcome back. In this lecture we're going to talk about another project that is project number three for this class. And that is customer segmentation project using our programming. So in this project, we'll learn how we can do customer segmentations using our programming made on a data set. So whatever data we have, the customer data set, that is huge product customer segmentation. So we'll try to find the segment of customers that will be useful for now. The companies to find their best customers and the best-performing our best customers for their business. So what is customer segmentation? Customer segmentation is one of the most important applications of unsupervised learning. So as you know, there are two, you know, there are two kinds of learning. One is supervised learning and another one is unsupervised learning. Supervised learning is a kind of learning where free will, I suppose that is a kid and we want him to learn. There are two ways to provide where we will supervise his learning. Tell him what to do, how to do, and will give us setup instructions for him to learn. And there is unsupervised learning where we'll not supervisory team. It will be unsupervised and he will learn by his own experience. So there are two kinds of supervised and unsupervised. And customer segmentation is kind of unsupervised learning. You didn't costly gosh, clustering. Using clustering techniques, companies can identify the several segments of customers or lowering the potential user base in this machine learning project. In this customer segmentation project, we will make use of k-means clustering. Will learn what is k-mean clustering. And we're going to be with this k-mean clustering technology for this project, which is essentially an algorithm for clustering unlabeled dataset. So here we're going to use the unlabeled dataset whenever you need to find your best costumer customers in mentation, either ideal methodology. Because with this, you will easily find those segment of customers. And while looking at the segment of customers, you can find your best customer base. And you can have ladder Target, Who them to sell your product or services. We will perform one of the most essential applications of machine learning, customer segmentation. In this project, we will implement customer segmentation in our programming. So now we have the refunds customer segmentation and clustering technology we are going to use. So let's understand in detail what is customer segmentation in a little more different, little more in detail. So customer segmentation is the process or the region up customer base into several groups of individuals that there are similarities in different ways that are relevant to marketing, such as gender is interest and miscellaneous spending habits. Companies that deploy customer segmentations are under the northern that every customer has different requirements and require a specific marketing, afford to address them appropriately. Companies aim to gain a deeper approach of the costumers they are targeting. Therefore, there has to be specific and should be tailored to address the requirements of each and every individual has the requirements of each and every individual customer for them all. Through the data collected, companies can gain deeper understanding of customer preferences, as well as the requirement for discovering value valuable segments, that would be the maximum profit. This way, they can strategize their marketing techniques more effectively and efficiently and minimize the possibility of risk to their investment. The technique of customer segmentation is dependent on several key differentiators that divide customers into groups to be targeted. Data related to demography, geography, economic status, as well as rehab, real pattern play a crucial role in determining the company direction toward addressing the variable various segments. So what we understood by customer segmentation is this UP BY adding the customer base into several groups of individuals based on the similarity in different ways that are relevant to marketing. Like we can divide the customer base based on the gender, like gender, male, female, kind of thing based on the age groups like teenage Adults and the senior citizens to kind of groups we can make, made on these parameters based on the interest. Suppose we have the customer, large customer base where we can segment the customer base based on their interests. Like somebody is interested in music and dance, in arts, and titers in drama. So we can segment the customers and then based on their interests, we can recommend products and services. If somebody is interested in. Suppose in sports we can, can show them the ads are sold them the product or services that are related to that particular sports. So these are very important things in identifying the best customers to target. With this segmentation, the companies will gain a deeper understanding of their customer base. And based on the understanding and segmentation, they can target the customers in a more specific way. And that will increase their profitability. Because if you are just sender, sending some add to the customer and you are selling your product and services are for the support of senior citizens and you are sending your emails to everybody, then for the teenagers, it is not relevant, right? What adult? It may be relevant in some way that they have the elders at home. But it is more like, but it is more likely to be relevant to the people above 60 or senior citizens. So it is better to target only people who had the age of 60 and then it will give you the most profit. So this is what does customer segmentation mean? So before we proceed to the our project, let's understand what is k-means algorithm. While using K-means clustering algorithm, the first step is to indicate the number of clusters that we wish to produce in the final output. Okay, so first thing first, we need to decide on the number of clusters that we want to produce in the final output, the algorithm starts by selecting k objects from datasets randomly. So it will first select the number k number of objects from the dataset randomly that will serve as the initial centers for our clusters. The select star clusters means also numerous centers. So these are, like I said, there are k objects we have selected. So these selected objects or cluster means, and they are also known as centroids. Then the remaining objects have an assignment of the closest centroid. The centroid, the centroid is defined by the euclidean distance. Glue Euclidean distance present between the object and the cluster. We refer to this tape Azure cluster assignment. When the assignment is to complete the algorithm proceed to calculate new mean value of each cluster reagent in the data after the recalculation of the cluster, the observations or check if they are closure to a different cluster using the updated cluster assignment. This goes on repeatedly through several iterations until the cluster assignments stop. Altering. The clusters that are present in the current titrations are the ones in previous hydration. Summing up the k-means clustering. So this is what k-means clustering means. We specify the number of clusters that we need to create. Then algorithm select your objects at random from our dataset. This object is initial cluster or mean the risks and try it. Our key object or pin the assignment of a new observation. Rebased this assignment on the Euclidean distance between the object and doesn't read the k clusters in the data. Once our data center through calculation of the new mean values present in the, present in all the data points, that data point points, that data point of the clusters, the kth cluster centroid has a length of P that contains the mean of all variables for observations in the kth cluster. We denote the number of variables would be I try to minimize Jason up the total within the sum of the squares, then through the hydrate, minimization of the total sum of the squared assignment stopped wavering. When and when it achieved the maximum arbitration. The default value is ten that the art software uses for the maximum operation, so maximum address. And instead, we calculate the clustering algorithm for several values of k. This can be done by creating variations within k, 1-10 clusters. We then can calculate the intracluster sum of squares that is called SS. Then we waited on that number, k clusters. This plot denotes. Then we'd get the k clusters. And based on that, we'll see, so this is the theory part for this project. In this project, in next lecture, we'll start doing the project when we start discussing the code for this project and we'll do the actual project up. Customer segmentation. See you inside the next lecture. 83. Project 3 Part 2 Customer Segmentation using R: So let's start with the project. So here the very first thing is we need a data file which contains all the customer information. So here we are using model, model underscore. Customers start to CSV file and this CSV file is here. And if I open it here you can see the the customer number, gender, age, and income, spending score. All this information has been Given. We have the five columns customer numbered, and each annual income spending spending score. So this is the information all we have. And based on this, we are going to know that customer segmentation using. So let's get started. So first thing first, we need to read that file. So I use the app data are valuable or customer underscore data and I'll use the function, read dot CSV and Mazda part, part the fight let CSV file and it will read the now from that CSV file. So you can see that the custom ID, gender, age, annual income, and spending this course. So here you can see their own datatype, char, char columns. Okay? The next thing is what we do. We named customer data and then really, hey, try to print the first five columns. Crushed favorite or study the customer ID, gender, age, and willing Commodus things. Okay, You can see Next thing is really summarize this with the customer is, so let's run this and see. Yet when we use the summary Wong Sun Belt on the edge, it will give us the minimum age. First-quarter me, first quartile, median, mean third quartile and max. So this is for the age data on this and made any statistics, okay? Then customer age, annual income, and spending score for all those things will handle the same thing. So we get this information, standard deviation, estimate, gender visualization. Here, what we'll do is we'll create a table customer database on the agenda. And then we'll try to plot a bar plot. And this will have the x-axis and gender and y-axis as the count and colors capability with that. And let's run this. And you'll see a bar plot, which we saw the gender female and male here. If we do more. So using bar plot to display the gender comparison. Okay? Next thing is this formula here divided by sum of a in 200 LBS, male, female and male, female. And this will give the depicting this you have male and female. So C and now the purpose X percent and male is 44%. So now we don't want that in our dataset. 56% of the female end, 44% of the male. Now we want to do, we'll try to visualize age distribution. So try to draw a histogram made on the glass and frequency. So see here, now, this histogram will show you the contact each class. So here you can see zero to 20 frequency starting for 20 to 30. All these things you can see, this is the histogram. This is the age class 2030, 30 to 40, 40 to 70. All these things we can visualize and see a bar plot with the age. And this will give us the descriptive analysis is a boxplot. So here you can see that it is mostly between the target and then a C sub N1 income. So here for enrolling company, try to plot a histogram or the annual income and frequency we'll put on the y-axis. So see you for the annual income plus the frequency, you can look at your hockey. Then we'll put our customers Plot, another plot here, density plot for annual income. So all these things you can analyse then we'll draw a box plot for spending score. See here, this is the end, this is the box plot for that spending score. Then again, our histogram for the spending score. And then we'll start our K-mean algorithm library bar, and we'll set to seed. And then we'll use a function to calculate the total intracluster sum of square ISS and put to one to ten. Okay? Next thing is I assess value, report, map underscored AV. And then we'll plot the number of clusters on x-axis and y-axis, total intracluster sum of squares plus the number of clusters. You'll see one to ten. And here you can see the total y-axis. Clusters, sum of the total cluster, sum of squares, okay? Number of cluster for these, 4464814. Okay? Evidence, select my third library. You'll use cluster extra. These three library you will use. So if it is not installed, you can go here, tools, installed packages, and give the name of the package and it couldn't be. Let's start. Now. What I'll do, we'll do, we'll create the clusters k2. And k2 will use the K-means Clusters. And then we'll give that I dress and max actress and 100 and start and start from D algorithm like. And then we'll plot this cluster and customer data values. And the Euclidean meant that, okay, like this, we'll put the k3. This is further, this is for the A2. Now we'll do the thing for k3, k4, k5. And we'll see you in the plot is knitting. Now we'll use the envy cluster and no extra. And we'll do celebrity and you will see it one-to-five and customer data, k-means. And then K6. Know we have this optimal number of clusters and number of clusters. Cluster one ever wrestled with this for 3.4, 647. And then we have the optimal number of clusters. We, then, what we'll do, we'll try to visualize the top principal components. The PCC lashed up with PR function for this. And let's run this. Here. You can see the annual income spending score. And then we use the GG plot to plot this. Now you can see the segment up model costumers using k-mean clustering data, the cluster Santa costumers. This is one cluster, this is another cluster. This is one plus, this is one cluster. So the gentle clusters that the customer based on their behavior. Now. Now try to put the cluster one, cluster two cluster three, hydrogen gave me. So now you can see six clusters. Cluster one, cluster two, cluster three cluster for cluster five. So these are the points, these are the clusters. We can see the data. Now we put that and we'll try to run this again. And now it is pretty clear. Now we have the k-means clustering here. So here you can see this red one is the cluster one, yellow, one, cluster two, cluster three, the green one. This cluster, cluster five is blue. And this is cluster for. Now, what you can see, this is the other customer clusters that we have created based on the k-mean clustering. Cluster four and cluster one to cluster consists top customers, medium one and medium PC scores. And cluster six. Classic discussion I print the cluster having PCA, high PCO2 and low PC. One, cluster five in this cluster there are customers with the medium and low PCO2. So these are the bed on the PCA one and PC2 values we have. We can apply this glabella done the clusters and behaviors. We can target the customers with the help of clustering. We can understand the variables much more dead prompting us to take careful decisions. With identification of questions, companies can release products and services that target customers. We don't say that, but I would just like income, age, spending patterns, etc. Furthermore, more complex patterns like product debuts, uptake into consideration for better segmentation. So this is the simple explanation. You can go in detail and learn about these things. Further, I hope this project will help you in understanding how we do the gaming clustering. 84. Project 4 - Introduction - Movies Recommendation: Hello and welcome back. So in this, we're going to do project number four, movie recommendation, system login and machine learning. So now we have the basic understanding of programming and we have done few projects also in this class. So this project will give you a very broad experience working for machine learning algorithms, judging our programming happening in this, we will be doing the very popular movie recommendation system. You didn't add programming and machine learning algorithm. So what is this project all about? This project focused on the development of a movie recommender. I'm using R and machine learning techniques out and machine learning techniques. The goal of this project is to the recommendation system. That recommendation engine that suggest movies to users based on their preferences. So what is actually happening on all the OTP platforms like Netflix, hot star, T5, Amazon Prime videos. Whenever you are watching movies or put your, you have watched a comedy movies. You have Mazda was romantic movie of Acts and movie. So based on your preference, what you have watched in the past, all your preferences will be stored. And based on your previous behavior, like you're watching a comedy movie, commonly genre or acts and genre. Next movie we'll be recommended to you. So all flex and all they're losing the same metallurgy. To suggest to you the next movie. Support. You have watched the committee and the X-Men movie in past weeks. So next, what they will, they can do. They can suggest you a movie which has both the things comedy and excellent. So they can suggest you X and movie with the comic elements in it. Are they, They will definitely suggest to Duck comic movie and the movie along with that, they can suggest to you the movies which are containing both comedy and exon together. So all these things we do using the machine-learning algorithms. So this is very popular these days. So by implementing item-based collaborate to filter what we will again, we will gain practical experience in applying our data science, applying our data science and machine learning skills to real life project such as movie recommendation system. What we're using here, does it MovieLens dataset, which consists of one lakh, 5,039 ratings in the ratings dot CSV file. And it is also containing 10,329 movies in the movies dot CSV file. So these two CSV files, data files that we are going to use, one will contain the movie in commission, like 10,000 plus movies. And other than that, I think CSP will store the spot towards particular movies by the various critics and audits. So now we have the basic understanding like what we're going to do. Next thing is, what are the libraries? What are the libraries we will require for this? So we will require the GG plot to lab data.table and receptor, the other basic libraries that we need for this project. So GG plot these for the digestion part data.table. What table and the commander lateral views. Recommend lesson thing. So what are the steps we are going to form in this project? The first step will be data pre-processing. Then we will be having the next step Elaborate to filtering are exploring the similar data. Then there will be a database. And then we'll go to the data preparation. And data preparation involves a few steps and then at last will be doing the standard one by one. What are the things we're going to do in data preprocessing. Data pre-processing is what supported. We are getting a raw data up movie dot CSV file or reading drugs, yes. We need to preprocess that data. So once we extract the data from the movies dot CSV file and ratings or CSV data sets. What I observe, what we will observe that user ID and movie ID columns, integers. So these are the things we'll see when we look into the code also. But for now, I'll just let you know. In this file, we'll see what are the things we can do. Let's flush. So let's first understand what we are going to lose. So this movie's ID and user ID. It consisted of the integers. And additionally, we need to transform the movie genre in movie underscore data, dataframe into more user-friendly ****. Movies or not, we need to transform into a more user-friendly format that you do. And to achieve this, what I have done, I have created one hot encoding metrics that represent the genre associated with each field. So to achieve this user-friendly data frame, movie genre, what I have done, I've created a one-hot encoding matrix that referenda genres. Each movie subsequently are such patterns were generated to facilitate each searches based on the John journals. Since movies often have multiple genres, we all know that one movie can have, we categorized in multiple genres support. One movie can have x and also it can have drama. So it can have the comedy also. One movie can have multiple genre as well. So what we'll do, we'll use that metric class. And for our data, we'll, what we'll do, we'll convert the matrix, the genre matrix into this sparse matrix to make it so table for the movie recommendation system. And for this we will imply the real mad at the yellow rating matrix class. So next step will be the collaborative filtering or exploring the similar data. So collaborative filtering involved for recommending movies to you just based on the preferences of other users. So what does it mean? Collaborative filtering is, suppose you are watching a movie, which is a comedy movie. And you have not watched much of the movies on a platform support. You are watching Netflix and you have just started using your neck. And you have watched just one movie, which is at the Comedy genre. Now, how Netflix will recommend movies to you, because Netflix says no data, you accept that you have watched a comedy movie. So either it can keep on, recommend new, recommending you the comedy movie rate. And that will be a very weird thing because a person will not depend. We'll watch only one genre movie accommodate only he may be interested in the x and also drama. So suspense, thriller as well, right? So for that, what this split function will do, they will use Collaborative filtering, which in what they're recommending movies to users based on the preferences of other users. So there are millions of users using the platform and they are watching the committee also x and also. So based on our users who have watched the same movie and what they have watched next. Based on their data. Netflix will predict that you may also like a particular movie which the order you just watch the movie was. Netflix can recommend you movie X because you have watched the movie. Okay? So if you jump in X and so does User B, then the movies watched by huge a week. And we recommended to huge array. The same thing I have explained you and vice versa. Therefore, the recommendation, recommendation of movies relies on establishing our similarity relationship between them. You will what we're actually doing here, we're establishing, we're establishing a relationship between the yield similarity based on the similarity of their accent or their interest. Using the recommended lab library. What I have done, I have computed the similarity using various operators such as cosine, Pearson and Jakarta. And then construct data visualization lesson. We'll also see the similarity in data. So here what I have done, I have visualized the similarity between users and also explore the similarities among the films that previous. To get. Databases are most views movies will see than what I have done. I have investigated more in the dataset. Prior to this analysis, I calculated the number of huge, huge for the each field and organize them we knocked able in descending order. Total number of views for the Gulf films will visualize using the barplot and from the vigilantes. And it was written that fixin was the most watched movies. So these are the data visualize and things will do. And after that. After that, followed by forage to come into every lesson, a heat map of movie dating to gain insight of the movie ratings, I created heat map that displays the ratings of the top 25 rows and five columns in the data set. Next thing, select selection. So what I have done, I have done, I really like the golf. You just send movies through a heat map, also examine the distribution of average ratings for future. Next thing is data normalization. We're dressed up potential bias caused by users who consistently provide high or low ratings for all the movies they watch. I normalize the data. Normally suddenly the procedure to standardize numerical values in a column to a common scale, ensuring there is no distortion in the value. Then in this case, normalizing transform the average rating, where I hit my voice and plot it. We will see that when we do the hands-on binary in the final step, data was banned, right? Assigning discrete values 1.0. This step improve the efficiency of the recommendations. What I did, I define the matrix with a rating of three correspond to one. And otherwise, it is just saying if the movie rating is about three, it will correspond to one. Otherwise, if less than three, it will be assigned a value of zero. Collaborative filtering system. In this, what I have developed item-based collaborative filtering system that determines item similarity based on the huge ratings. The algorithm will our table of similar item purchased by customers and by a recommendation part of it is the steps involved with determining similarity between the items are as follows. For each item on the agenda, item ID1 present in the product catalog purchased by customers see for item I to j by customers, see guilty of what I had done. I create Jade by customer, see guilty of what I had done. I created A record indicating customers see purchased I, i1 and i2, calculate the similarity between the items I1 and I2. Okay? Because customer has bought I1 and I2, there might reach answered that i1 and i2 items are similar in some sense. So that similarity we will calculate. And then we will split the data set into 80% for the training dataset and 20% further testing for a recommendation system. The 80, 20 rule is always applicable in machine learning algorithms. The next and final step will direct model system. For this export the various parameters item-based collaborative filter. The default value of parameter k, which denotes the number of items is used to compute. 30. Algorithm identifies the k most similar items and stores their corresponding numbers. Recommender recommendation model was retrieved using get model function and the glass or diamond sense of similarity matrix we had analyzed or heat map or generated to visualize the top crunchy items of the same lighting. Next thing is exploiting that a man distance system model. By summing the rows and columns of similarity above zero, I obtain the distribution of some columns. This distribution was visualized to gain further insight. To create the recommender system. In this slide, the top recommended by the vertical ten is specifying the number of films recommended by each user. And the predict function was then used to identify similar item accordingly. Each rating was treated as a weight, which was multiplied by the related similarity tastes. Finally, all the weights with added to generate the recommendations. So these are the things we are going to do for movie recommendation system using R and machine learning. Our goal is to recommend movies for the users based on their likes and dislikes and what are the users are watching. Underplayed prom based on the similarity thing. Okay? And for library GG plot TO data.table received through and recommended lab. We're doing steps really looking for data preprocessing and data processing will do then we'll do the collaborative filtering out, exploring the similar data that we have already discussed. The AWT lesson. Next, we'll see how we can revise the data. For data preparation, there are three steps to data select some data normalization and data binarization that we'll see. And the collaborative filtering system that we have already understood by this example and what I have already discussed initially. So I hope the project I have explained. Well, in the next lecture, we'll do the project by writing the script. So see you inside the project 85. Project 4 -Part 1- Movie Recommendation System using R : Hello and welcome back. In this lecture, we are going to do the project of reading a recommendation engine that tech movies to you just done their preference. They have watched earlier. And also weight on the other users on the platform or watching movies. Or they have similar kinds of movies which you judge. This huge debt is watching our watch window. If you're doing if you are using the streaming platforms or TPU for fall for the first time, they will still be able to recommend more risk to you. Because they have the usual database, which is having millions of people who are watching the similar kind of obey that you have just so based on their interests and your interest of watching a movie. That is the similarity between you and other people, like just think on the platform so that their preferences are what they have worked there. The heavier. It is. Recommend to you, the movies. So here what we're going to do it, we are going to be where it's item-based collaborative filtering recommendation system. Okay? So what is the item basically, I can reach me based on particular items. We will consider dark collaborate to print our similarity. So far this reality region, we are going to use the recommended lab library, GG plot for visualization data.table, what tables and receptor. So these four libraries. Okay, next thing is we are going to retrieve and display the data. Movies and the movies data. More read dot csv. Friday we'll see androgen source CSV. So let me show you the stock price. So this is more Vidar CSV, which is containing the movie id, which is one to 10,000 something. So more than 10,000 plus more risk has been added here in this dataset. And then go to love the movie, movie ID title of the movie genre, which journals you have washed off for me. So there are more than 10,000 movies in this dataset, their title and their genre. So here you can see one movie genre, the genre adventure. Before this genre Edward Jeff animation, children already in parentheses. We're done that three nights having comedy and drama. That is, I really love now and then children and drama. Drama, fantasy, mystery, safe way. So one movie can belong to multiple multiple genres of movies. And then we have the structure Yes, CSP, which will have the user ID, the movie ID, for which movie, which it is given that rating. Then what rating? You guys, even to the movie that it's one or one to five. And between onto high rejection portrayed the movie. And then we have the diverse types. So let's get to the core. So he identified will go to the path of the file. Then we'll create a movies underscore data variable and then restore the data. So read CSV file function will use and dumb movie dot CSV Which is being stored in this thiamine be rigid us it and and string as fructose, false. And dendritic underscore data, dot CSV and things start. And then we'll see the data is dark CSP less than this. See here. Now we have the movie id that the teacher can jump, right? Scott actinides the job and writing. Them always. Naturally see the summary of the data by using summary function. The summary of the data you can see, yeah, movie, first quarter, second quarter, third quarter, I mean genre. So here they can see the ones the movie I did see the head up open the score data entities coming 123456. Okay. So now we can see the summary of data and then really see the kidney it out one heart. And coding to create the matrix that comprises costs for new Kendra Roth, each outs of freedom. So this will meet the data processing thing. We've created. The one-hot encoding movie underscore gin Chandra and data as.data.frame movie underscore data and Jordan rivalry. String as structures, strings as factors is equal to false. Then review the library data.table and yeah, creating movies underscore genre. And here we will add dark data.frame. And he had DST, the STR, split movie genre, and type dot cardboard true strings as factors files. And then we'll call her name movies genre. And we'll use C, one to tell. And they stop genre at some of those things where report to your left knee. Hello, I'm from here to here. No column name and the leash stuff. John, john drop and fought index one tool. And I trekked, handwritten. I don't run this code. And for calling in. And you John her my two younger, I like to call them and add Argentina. Chandra, broken. Want Dakota does to a string. Then right-click on this. And then can you hear to start my tricks for such infant base plus plan Chandra and such five tricks. In this quarter, we're using editing my tricks and my take on this views. Some important parameter for within that system per mole easier when you do that. Though like one station model registry and get entry data type for the L matrix, a matrix. And he entered you the I received my tics and recommendation Mark Moore, I underscored the rating matrix and our trust. Cnidaria. You just similarity can see using this my graph and portrayed us in lightly that is between the fields. Okay, that's it for this lecture, we'll continue in the next lecture. 86. Project 4- Part 2- Movie Recommender System: In the previous lecture, we have started with the movie recommendation system using on this project. So let me give you a quick recap of what we have done in the previous lecture, part one of this project. So we need these for libraries, recommender lab. You plot two data.table. So we just stand these libraries and then we need to get the data from the place where you have kept the data. We'll then store that will read the CSV file using the read.csv function and restore into movies underscore data. Similarly, we'll store the rating data into rating underscore eta by the similar between the dark CSV. And we'll pass the final numerating starts CSV, then, then STR, lungs. And we'll pass this move underscore data. And then we'll see the summary of the movie underscore data. And this will give you the movie ID title, genre. And here you see the statistical details here. Minimum, first quartile, median, mean, third quartile and maximum values for this data. Then we'll see the head of the movie underscore data. It will give you the first five rows of the data. So see here the flush movie ID and title, and then the genre of the movie. And then we'll see head of the ratings. Yet also you can see the minimum first quartile, median or all the columns, okay? Then we'll see the head of the ratings data. Yet user ID, movie ID and a rating and times tamp is given. So now we have the idea of what is there into the idea of what is there in the CSV file or the data file that we have and that are on these data we are going to walk on. So the next thing is what we'll do. We'll create a one hot encoding, creating a matrix that comprises corresponding general for each proof. So for our diet encoding, we need to create a matrix. So we will create a variable here more than underscore genre and here as.data.frame, reuse them movies underscore data. And then we'll use that genre column. So if you look here that is genre column in the movies data, movies of data file. So we'll use that and then strings as factors false and then library they will use later. So here we create a matrix, movie genre matrix, and then just use the data.frame and movie genre. And then we'll be done matrix here, then this, and then column name. And then they stop genre. Genre. What are the list of genre is your x and adventure. All those things will make a list here. And then we see the genre, Mach one and column names. And for index in one. And rho movie genre to four will read the column and for each genre will get the movie name. And all. So let's turn this then, genre mat two. And then same here. Then we pass it to STR here. Then we pass it to STR. And now what we'll do, we'll create a search Patrick's. So I think by specifying the genre. So John, right? So let's run this using yes, C bind. Okay? And then we go ahead and see here now we are getting on this again. So CN movie ID title, Excellent Adventure Animation, children, all the genres As a column in this matrix, okay? Next is, this is a matrix. Next one is rating matrix. And now what we'll do, we'll convert that rating matrix and toward recommend lab sparse matrix, that matrix and the yellow rating matrix, okay? And then the rating matrix C is 668 into ten times the quantity five or n columns. The rating matrix of plus one lag 5,003, 13 and ratings. Okay? Next, we will use names up, recommender, recommendation model. And then we'll use the play Recommendation mode. And here we'll use the description. Implementing a single model in our item-based collaborative filtering, we have 12. And here we are using AI VCF, okay? Now, collaborative filtering involved suggesting movies to the attacker based on collective preferences from many other. You're just, with the help of recommenders lab. We can compute similarity between the Eucharist. So now we're going to create a similarity metric and method we are using ARCore shine. And farther you're just okay. Now we'll see that you just similarity will throw this off. Again. There was not enough space C. Now you just similarities, you can see you just similarity matrix. Similarly, we will do, will seem like it at least several between the films that we use. And we will run this and then print that image. Now, you can see this is the movie similarity. Now rating values as director rating matrix data, we'll see. Then unique ratings extracting the unique ratings. You use a unique, then require the rating values. Table of ratings. Creating a movie rating. And then we'll see the table rating values and evaluating. Now, what we'll do. We'll do the most viewed visualization. And for this we're going to plot two movies underscore views and Qualcomm's rating matrix called count will use and then TableViews a data.frame movie called her names. We'll take the names from the movies viewers and take the names from the movies view and go to movies viewers. Here dataframes of future through this table views. Similarly, then TableViews index will iterate over all the entire 325 movies. And we will. And then we'll see that table views running. Wait for it to complete. So see here now, complete fc here you can see that movie reviews and title. Next thing what we will do, realize you're using the bar plot or total number of views of the top page. So far this we'll use the GG plot, TableViews and aesthetics. We'll use title and views and geom bar, geom texts and team and delete. I will give you the title here. So I hope you know all these things. I'll not go in detail. So now you can see the forest comes as 331. Your classic back to 94, views and bulk. Okay. So this way we can get tough viewers of the turf fields. Total boots up the top firms. Now. Firms. Okay, now we'll create a heat map for the movies. So see here the heat map out the first $25.25 columns. Okay, Next is what we'll do. We'll go into the data preparation. So movies ratings, ratings are greater than 50 and current contract is improved. Then. See the movie rating for 22, for 1,400 in class. Okay? So these many are there for more than three, okay? Now, driving metrics of relevant you just rating 0.9. So minimum movies and minimum you just will see we'll use the content for this. And then we'll create an EMS, see a heat map of the top, you just add the movies. Next thing is visualizing the distribution of average ratings are huge. So for this, we'll use the average underscore rating. And form means we didn't have the movie's rating and will plotted. Now you plot C and now we are getting the distribution of average rating, but you're just okay, So this is the distribution of the average rating, but you're just then, we'll use that to that data normalization. For this, we'll use un-normalized ratings, normalized function reviews and other movies rating. Then some of the row means normalized ratings greater than 0.001. And then see this normalized rating, the top users. So here you can see you will get an item columns. Okay, next, data binarization that we have discussed already in the enterotoxin. So here we are using quantiles and row counts, movies rating and then more base wash. And then rated movies, binary movies, more, minimum rating three. So minimum rating a3. All those movies will get. So let's run this. See, these are the movies which are having minimum rating as three. Now we'll go into the collaborative filtering and it's splitting the dataset into a training set and 20% of the test set. So sample data, we have your sample method here. And here what we have done, we have even 80% for the training and test. Split the data. Now we will see that training data and testing data. And then we'll build a recommendation using a recommender registered registry. And here we'll, what we'll do, we'll get entries, datatype, real rating matrix that we have created. And then we'll use the eye vcf, underscore real matrix and parameters, and then recommender will use the recommender meters. And here we will pass the training data method. We'll use IV CF, and we'll get started. Okay, so let's run all these things. Now we have the recommender recommendation model. Already. Run this class of recommendation model. You can see the recommender lab and now exploring the data science, the coordination models. So let's turn this class model in, forget model and model. And for C, D, D, C met afterwards, afterwards brackets and Patrick's next is gone. This top items. These are the top items. And then we'll see heat map up the first row and columns of the. Then when you run this sum of draws and table up some of those 30.47, then GG plot, we'll see this is the qplot and bar chart. You can see distribution of column number of items to recommend. You just put less than ten and predicted recommended one day. Some recommendations for the first urogenital one movie of 1 bar, one in ten movies. And then recommender matrix, matrix with the recommendation for EasyJet flight, kindness items. And we're valuing the column names from the movies reading. I'm going to buy items and then dark title, distribution of number of IVC up and qplot number of items since t, distribution of numbers of the Item, number of items up by BCF and da Vinci, the top recommended movies. Wallace and Gromit, God Father, son died and so are the YuJa. So this way, we can get the recommended movie for the YuJa. So I hope you understood this project. If you have any doubts, you can by commenting out, by asking the question in class. Thank you. 87. Project 5 Introduction Credit Card Fraud Detection: Hello and welcome back. So I welcome you to another project that is project number five for this class. And that is credit card fraud detection, judging our programming. And here we're going to use some machine learning algorithm as well. So what is the goal of this project? You may be aware off online fraud and data happening these days like somebody's grade card is being used. Without their knowledge, somebody's Social media has been hacked. So these kind of frauds are being done online. Somebody's banking formation is being stolen. Somebody's bank account has been stolen and they have taken the money out of their account. In India, there are few incidents where people, without knowing they have shared their guardian permission or their OTP is on a phone call. Ah, the imposter is saying that they're calling from the bank and they're asking for the debit card number. And then they're asking for the CBP number. And people are unknowingly say, sending all the details, thinking that the person calling is from the concern Bank. And they are just setting that our details. And once this OTP, all their money from their bank account has been taken away. So this kind of frauds are going on and we have to be very, very careful. One such online fraud is credit card fraud. Well, people use their kid cartoon probation or to buy something, or they store their credit card information, or they say, are there Kate card numbers and some even number unknowingly to the people and people use them without their knowledge. And there are several incidents where Kate card is being used without even though Kate card being physically present at with the impasto. This for credit card fraud detection. So we need to do a front sorted. Whenever a grade card being used by somebody, the company will know that this transaction could be a fraudulent transaction and they can immediately inform the customer that grade card holder, that you are performing these kind of tangents. And right now, the customer or the grade card holder says, No, no, this is not me. Somebody else is doing this transaction, then immediately they will decline that transaction. And in this way, that grade card we'll be stopped by using and not a lot of money will be saved. So the goal of this project is to develop our classifier capable of identifying fraudulent credit card transactions. For this, what I have done, I have you, I will be using several machine learning algorithms, such as a decision tree, logistic regression, artificial neural networks, and ultimately gradient boosting classifier. So we'll see what are the algorithms we can use. And with this, we can classify what are the fraudulent and non-product or Jackson's. For this, we'll be using a dataset which will be having the information about the cake card and the type of credit card transactions in your prevalent or non-problem for the training and moderate level up men. Then we'll see data exploration, manipulation, data modeling, fitting the logistic regression model, decision tree, artificial neural networks, all those things will do. But before that, we need to understand in detail fraud and its type. So if you are not aware of the DOM or the problem in your hand, you will not be able to give the proper solution. So before jumping into the project, we should be knowing that What problem we're going to show. So for that, we need to have the very clear understanding of what each fraud and what kind of fraud are being done nowadays in the era of the digital. Okay, so first thing, when the Internet was not there, this much of our digital transactions were not there. Then people used to go to the bank and they used to take the money by fitting a form and then the ATM card scheme. Then people started using the ATMs to withdraw the money. And then people started doing the ATM frauds by just manipulating the ATM card holder scarred and just seeing the password of somebody's and stealing the card and using the card at IPM by this particular character password. So this kind of fraud sweater happening earlier. And with the increase of the Davison and digital banking now a days the online frauds are being done. Very rarely batch. So what is fraud? Fraud is defined as any intentional act of deception for personal and financial gain. To fraud is defined as the intentional act of precepts and for personal and financial gain. So if you are doing something, if you're doing this epsilon with somebody for your personal gain or financial gain, then it will be classified as fraud. It involves missed the presentation, concealment, or manipulation of fat to deceive others. And what what types upgrade card fraudster happen these days. So types of credit card fraud, Identity, account, takeover card, not present, fraud and counterfeit cards. And then comes the skimming. So we'll understand one-by-one. What are these type of card fraud? So the very first thing is identity theft. So what is identity theft? Fraud us just steal for fraudsters, steal personal information such as social security numbers or login credentials to impersonate the victim. So in this situation, this tilde, apart from information like source, some security numbers or the other, our card number or banned card number or the login credentials of your banking. And the Kcat Dunbar's in some cases, to impersonate that they are the real customer. And then they will login with the details and they will use your banking information to do the transactions. They use the stolen information to open fraudulent account to make unauthorized trajectory. With those like your other cardio or PAN Card, they can open a bank account on your behalf and then they start doing the impersonating yourself and they will be doing the transactions, which will be on your name but you are not involved. So that is called identity theft. Then come the account takeover. The account takeover is another kind of fraud where criminals gain unauthorized access to a legit meet account by stealing account details such as passwords or personal identification number. So in this, they will just hack your account, steal your password, username, password, and they will be having your account taken over. And then they will do whatever they want with your account. They take control of your account and use it for fraudulent activities such as making purchases, Art, transferring fonts. So once they have the control over your account, they can do anything. They can transfer the money from your account to any other account, or they can purchase things online and offline. And you will not having any access to your account because your account has been taken over, they can change your password as well so that you will, you yourself will not be able to login to your account. The third kind of fraud is Card not present fraud, which occurs when fraudulent transactions are made without the physical presence of the grade card. And the very common is online or over the phone transaction where card details or manually entered fraudsters, huge stolen credit card information to make unauthorized purchases. So in this, the grade card will not be physically present, but what they will do, they will call you and they will ask you to do a particular Jackson and they will ask you the kid card information Over the call and then they will ask for the OTP. And as soon as you give the OTP or tangents and it's complete and your money will be gone, or they will be buying something on your behalf and your credit card will be charged. The next stage, counterfeit cards, fraud as just create counterfeit grade cards that meet once. So in this kind of fraud, what fraudster will do, they will create a fast, they will get some information about your cage card numbers. They will be numbered all those things. And they will create a similar looking card with a similar cage card number and the CBP number, which we'll be looking very legitimate to the original one. And then they may encode the stolen credit card information onto the counterfeit card. And then Counterfeit cards are then used to make purchases. Often without the Carter cardholders knowledge, you may not be knowing that your grade card has been counterfeited and somebody else is having a duplicate credit card off the same number which you have been assigned and they will be using it for the fraudulent transactions. So this is another kind of credit card fraud happening nowadays. The next one is scheming, is skimming involves, legally, sorry, skimming involved illegally capturing credit card information without card cardholders knowledge, suppose you are giving your cart store charging and without your knowledge what they are doing, they are just noting down your credit card numbers. If we number all good information, expiry date and all this kind of information when the store they can do of fraudulent transactions. So whenever you are setting your car over the counter for dy dx and you make sure that nobody is taking any information from your KD card without your knowledge. And then protestors, what we'll do, this tile installed the scheming devices or payment terminals or ATMs to capture the credit card data. Recently in India, one kind of fraud came to notice last month that they were using the favorite coal into the ATM machines. And with that, they were using some techniques where the when you insert the cage card or your credit card, information will be captured somehow. And then later on, when you go away from the ATMs and they will do the tragic cells on your APM. And you may not be knowing this was scheming up for serving happening nowadays. Then they use captured information to create counterfeit cards and performed turn probability. So this is the kind of identity theft account takeover card not present, counterfeit cards and skimming or fraudulent activities are being done online. There are more kind of frauds are also there like over the phone, they will pretend like they're a bank employee and they will ask you for the sum fortunately information, your credit card information, social security numbers, bank card numbers are darker. And then they will pretend like they're giving you some award or something. They're transferring some money to do your account, but actually they will be demanding the money from you and they will send you a link. And they were asked you to click on the link. When you click on the link, harder, you will not be noticed that you stood up transferring the money. They're deducting the money from your account and you'll just enter the OTP and your from your account money will go to the prevalent account. So this kind of fraud happening and we need to know how to stop them. And that's what we're going to do in this project by examining the guard Jackson's. And with that, we will develop a model which will detect the kind of transactions and it will stop the fraudulent activities 88. Importance of Online Fraud Detection: Now we have the brief understanding of what is fraud and what kind of frauds happening these days. Now, we have to also understand why plot detects and it's important. So fraud detection is important, that is one obviously then is to prevent the financial loss. But there are another region is also there, there are few other reasons also they're knocked one and that's what we're going to understand it. The importance of fraud detection. There are three major fraud detection important that says are there. And the one is financial loss prevention. The second one is products and customers. And third is maintaining the trust. And these three parameters are very, very important for any banking or financial institutions. Because if you will not provide the financial loss per taxon, your customers customer is not going to stay with you. If you're not protecting the customer interest, then also probably not with you for a longer time. And if you miss to do this, prep, financial loss prevents and end products. And our question was information, then you will obviously lose the trust of the customer. So maintaining trust is also important. So these three are things that are very important. So first thing, we understand, we will understand this financial loss prevention. The fraud can result in significant financial losses for individuals, for businesses, and for financial institutions as well. Fraud detection helps identify and prevent fraudulent activities minimizing this pregnancy losses. So the very first aim for the product axon is to prevent it, the financial loss, financial loss part of the individual. People like you and me, for the businesses, the people who are doing businesses, and the financial institutions itself. Like if somebody will come and they will have the bank, bank account itself. And then he, the enforcer can do the hacker can do any transaction on any of the banking customers, right? So to protect their account also, it is important, okay? So fancy loss prevention is one such important thing. The second is protecting, protects enough customers. Fraud can lead to identity theft, unauthorized transactions, and other form of financial financial harm to customers, detecting and preventing fraud and seals the protections of customers financial assets and personally permission. So protects and a question mark here. It means that we need to either banking institute, if a bank is there, other financial institutions are there. Their primary responsibility is to keep the customer information safe and sound and private. To ensure that they have to implement that fraud detection majors so that customers, financial assets and personal information will be c, meaning safe. The third is maintaining trust. Trust is also very important if your bank, your bank institution and constantly a fraudulent activities are being done on the cones. Then the customers will lose trust in you, and then they will seek to other bank. Fraud undermines trust in the financial system by activity, activity detecting and combating fraud. Financial institutions and Venus's can maintain the trust up their customers and stakeholders support. One-off. Suppose you are a bank, X, Y, j, and one of your grade card holder, customer will get a notification when somebody tries to do what unauthorized tange Exelon, his or her grade card. And before confirming that transaction, you just call the customer and say, Hey, there is, there is one arc tangents and $99 is being done on your KD card. Egypt you're doing on this particular website. And if the customer immediately says No, no, no, I'm not doing that, please stop. And you blocked that transaction, you decline that transaction. And in this way, you have saved $99 for your customer. And this will Surely increase the trust that person object customer into your, in your institution and he will surely recommend your instructional tool. Others also, because he knows that you have just saved $99 amount. This way, implementing the fraud detection wages to the financial institutions through the businesses. It will improve that crashed also for your costumers. Now let's understand what kind of statistical data on the fraud, What kind of fraud is happening and how much the fraud is cos costing the businesses and the customers. So global fraud losses, according to a report by the Association of Certified Fraud Examiners, ECF free, it is a global organization. Global organizations lose and estimated 5% of the annual revenues to the fraud. See, global organizations and the big, big MNCs are losing 5% of their annual revenue, which, which could be a very, very large amount to the broad activities. And this translate to trillions of dollars lost each year. So if you implement the fraud detection majors using artificial intelligence and machine learning, then you can save these trillions of dollars for your customers. The second one is Arcade card fraud. That Nielsen report states that global losses due to the credit card fraud exceeded $27, $27,000,000,000 in 2020, which is a huge, huge amount, and it's worth nothing that grade card fraud, one of the most common types of fraud. Given the widespread huge upgrade card. This is a fuse. Trade cards ports are very measure lid on. And you can see the stat C0, $27 billion in 2020 And now we're in 2023. So it could have been doubled with the increase of the digitization like India in last 34 years, they going all digital and gases being used very less. So in this case, it could have been raised footprint or billion-dollar as well. So credit card detection is very, very important to implement. The next is identity theft. In the United States alone, Federal Trade Commission, FTC received over 1.4 million reports of identity type in 2020. This highlights the prevalence of the impact of identity theft, which often leads to financial fraud. Then online fraud, with the increase, increasing adoption of e-commerce and online transactions, online fraud has become significant concern. The LexisNexis, this solution contains 20 true cost of product report revealed that every $1 of fraudulent transactions cost witnesses $3.36 due to the charge back squeeze and lost Mark and dice. So this is the impact. Businesses losing $1. It is not that they are losing $1, but indirectly it will cost them 3.3, $6 according to this report. And they said because of the chargebacks, fees, and the lost merchandise, these statistics underscore the urgency and importance of implementing effective fraud detection measures to mitigate the financials and possible risks associated with fraud. And with this, knowledge about the fraud is type and stars, what impact it is according on the financial institutions, on the customers, individual customers and financial institutions and weaknesses. Now, we are good to proceed with our projects. So in the next lecture onwards, we will start doing the date card products Excel project site. The next lecture 89. Dealing with Imbalanced Dataset: So before moving to the writing the code for the project, Let's understand few more things about the fraud detection. And this is important to do the project as try before starting the project. Including the theoretical part that is very important to enhance your learning. So in this lecture, we'll understand what are the challenges associated with fraud detection models that we may create and how to overcome those. So let's get started. Challenges associated with the fraud detection models. So there are a few challenges when we are going to our table up or fraud detection models. And one of the very important are very, that occurs very frequently. That is, imbalanced datasets. So imbalanced dataset or the data sets. Suppose you going to do fraud detection for Kd card and you know, Arcade card fraud activities are very less like 0.1% or 0.001%. Chances are there that one credit card activity or transaction will be the fraudulent. So when we have the dataset, which are having most of the transactions for our four cars around the world. Will we let him teammate one leaves few lake in one lakh, 1 million cartridges x and there will be few hundreds transaction which will be fraudulent. So when we collect the data set, there will be most of the 99% of the dataset will contain the legitimate transact cells, will only few of them will be the fraud. So when we create a model, or models will always favor to the majority. So in this case, it will, it will always, whatever transaction will happen, it will trigger that as a legitimate because the fraudulent transactions are very less, our models will not learn much about the product. Jackson's and because 99% data set containing the legitimate transactions, it will always favor to the legitimate transaction. So it will not trigger it very rarely it will trigger any fraudulent activity. So imbalanced dataset is one of the challenges associated with our fraud detection model. Fraudulent transactions are relatively rare compared to legitimate transactions. So the jolting, imbalanced dataset, the majority Jackson's alleged met, while only a small portion of transactions are fraudulent. This class imbalance poses classmates the fraudulent and legitimate transaction. So this class imbalance poses challenges for training models as they may become biased towards the majority class, leading to poor work detect set up product instances. So now we understood what is this E, So this is two is related to the majority of the dataset will contain the legitimate transactions because fraudulent transactions are very less. So. This will lead to the word it takes enough fraudulent instances by our model. Another one is adaptive fraud techniques for us to constantly evolve their techniques to bisphosphate the detection systems. So whatever detection system, how we develop the fraudsters are one step ahead and they will develop and they will evolve their techniques to bypass that detects system. They adopt their methods, making them harder to identify. Using traditional rule-based. Our pattern based approaches. As charged, fraud detection models need to be regularly operated to keep up with the emerging Fraud pattern support. You have Deb Love on rule-based approach or a pattern based approach. And we have developed one for detection system. And you are not updating it. But over the time, the protesters, they will study, they will develop techniques and they will just set a false detection systems and they will Start doing the fraudulent activity. So it is very important to adopt the techniques and enhance your models every few months so that you will be, I head up the protesters. Next problem is evolving fraud pattern. So fraud patterns are not limited. Support. You have a dataset where you are developing a model and there are few patterns that you have recognized the model recognized and it has developed a mechanism to identify and trigger a fraudulent baton while Nixon is going on. But what will happen that fraudsters will re-invent or they will keep on inventing new patterns that is not there in the dataset. And because of that, or models are not trained on that. So if our model is not turned on that it will not recognize that pattern and it will trigger that Azure legal legit activity and fraudulent activity will not be detected. So evolving fraught pattern. Another challenge in fraud detection. Fraud patterns change over time, make it challenging for fraud detection models to capture new and emerging Fraud techniques. Models need to be able to detect unknown or previously unseen fraud patterns without relying solely on historical data. So we have to make our model in such a way that it can detect a fraudulent transaction, even though it is not happened at that foot tangents and has not happened previously, or it is not historical data. So it should recognize the new pattern as well. So that is another problem challenge that is being fed by the fraud detection systems. Scalability and real-time processing is another very big challenge in fraud detection. Large-scale financial system process of all the amount of transactions in real-time. Fraud detection models need to handle high volumes of data efficiently and provide real-time analysis to identify fraudulent activity really promptly. This requires robust infrastructure and optimize algorithms to ensure scalability and real-time processing. So this is another very important and very big challenge, scalability and real-time processing. There are asked for the data. If we want to detect a fraudulent transaction and you want to trigger that to have only very few seconds, six to 8 s of time is there to detect, trigger, and identify the transaction that is fraudulent. If you miss that time frame, six to 8 s, it will pass through. So that is timed real-time processing is very, very important and this should be very quick can prompt. So that is another thing. Then Feature Engineering, featuring in developing effective and fraud detection system model requires expertises and feature engineering. Identifying relevant features that capture fraud detection. Fraud or electric patterns is crucial for accurate detection. However, selecting the right set of features and creating meaningful representations of data can be challenging and require domain knowledge and really active. I tried to experimentation privacy, constant fraud, detect, involved processing sensitive customer data, reading privacy concerns. It is essential to ensure compliance with the regulations and maintain data security throughout the entire fraud detection process. Next cell in this false-positive and false-negative, strike the right balance between the minimizing false positives, legitimate transaction flag, fraudulent, and false-negative fraudulent transactions meshed. And the triggered as a legitimate transaction is a challenge. A high false positive rate can inconvenience customer, while high false negative rate can lead to financial losses. Fine-tuning the model's parameters. And it's always necessary to optimize the performance. So what is false positive? False positive means legitimate transactions are flagged as a fraudulent. So what will happen in this case? Support on user who was using his own credit card and doing some transaction Which your model will trigger Azure fraudulent. And you immediately call that customer and say that you are doing on your credit card some fatherland projects and is being done. So what will happen if that customer he himself is doing that, transacts and he will get irritated because he's tangents and has been stopped. So in this case, false positives will create very much inconvenience to the customer. And if it will happen again and again, it will very much irritate the customer. Similarly, the fault positives, false negatives can also lead to the finance. If if a fraudulent transaction is missed and it is going through the system, then it will Gordon lead to financial losses. The board, we have to balance the false negative and false positive in such a way that both of these cases will occur very little. Inter, inter fretted ability and explainability. Modern machine learning techniques such as deep learning can provide highly accurate fraud detection models. However, they often lack interpretability, making it challenging to explain the reasons behind model's decisions. The ability to interpret and explain model output is crucial for gaining trust and acceptance from stakeholders. And addressing these challenges require a combination of advanced modelling techniques, continuous monitoring domain expertises, collaboration between the data scientists, unleashed and domain experts to build a robust and adaptive fraud detect sunfish dumps. Okay, now, we will discuss some techniques with which we deal with the imbalanced dataset. Because in our project we're going to use a dataset or a credit card transactions which is imbalanced in nature because most of the transactions are around 98, 99%, Tange excellence are as legitimate and one leaf 1% and the exons are there, which will trigger for the fraudulently dealing with imbalanced dataset. Dealing with imbalanced dataset is a crucial step in developing effective fraud detection models. Here are several techniques commonly used to address the challenges posed by imbalanced dataset. The first technique is these sampling technique resampling with water. We'll sample data, we have, we have two. We need to resample that so that the imbalance in the dataset can be managed. And there are a few techniques in resampling also that we can apply. The first one is undersampling. Undersampling techniques involves random play removing instances from majority class to balance the dataset. It reduces the dominance of majority class, but may result in loss of valuable information. So if we keep on removing randomly the majority class, it may result in loss of valuable information also, but this is also being done. Okay? Next one is oversampling these checklists involves creating synthetic instances of them minority class to balance the dataset. In this case, what we do, we fill the dataset with the minority class. It can be achieved through techniques like random oversampling or smart synthetic model minority oversampling technique. Smart oversampling increases the representations of the minority class, but may lead to overfitting. So this lead to the overfitting problem. Hybrid model. These models combined both oversampling and undersampling techniques to balance the dataset effectively. Next one is algorithmic techniques. The first one is cost-sensitive learning than n. Then both methods, anomaly detection is another technique, evaluation metrics and featuring in every so these are the techniques with which we will deal with the imbalanced dataset. So in our project we're going to deal with this imbalanced dataset with all over sampling, in which we will over-sample our dataset with dopaminergic cloud that is fraudulent activities. So see you inside the next lecture. 90. Fraud Detection with No Model: Hello and welcome back. So let's get started with the project. So here I'm using a credit card dot csv file, which is containing the credit card transactions details. Okay, So we, the first step is to read this CSV file. So for doing that, so to do that, what I lose, I lose out the door CSV function and pass the path for the CSV file and TSV file name. So the dirtiest we will rotate it and then I'll store that into them. Grade card variable. So important that it does, it is like this for do you know it? Very well. So far step is to run this. So now we have the grade card details in this grade card, okay, in this variable. So when you click on that, now you can see here there are a few columns times V1, V2, V3. And this V1, V2, V3 is going to read 20th. And then the column is amount, and then another one is glass. So you want to be 28 and then timestamp and then amount column and class column. So what are these columns? Timess, timestamp and this V1, V2 up to V 28, Arthur Jackson card transaction details of users. And why it is like this, because this is not the actual transactions for secularity region. And to keep the information of the huge Arcade card information safe and secure. By using the PCM. We have. These columns are the reduced version of the actual data to keep the user information sector. And for this, we use BCM and we use diamond modality to reduce it to this form so that the judge information will reject your there is an amount and then there is a class classes is for euro the presenter ledger transactions and one represent fraudulent transactions. So this is the dataset rehab, and we're going to walk on this dataset. So next thing is, we'll see the structure of the dataset. And for that we use the STR function and pass this card variable. And when we run this, we will, we will be giving them, this will give the result. Still running. Get started for the structure of the data. Okay? Next thing is this is the structure of the dataset timestamp. So here you can see all the columns are numeric. And among these numeric and the glass here, which is here it is numeric GTO one. But actually this not be numeric, but so be, it is a categorical data, right? Because it is, the determining factor for the euro is for the legit and 1 " further. One is for the fraudulent transactions like so this would be in the categorical format. So what we need to do, we need to make this into the factor. We need to convert this integer to the factor. And for that, what I'll do and use the credit card dollar, dollar classmates, we are going to use the this class. Okay? This column. Okay, so can you read dollar call class will take us to this color. And then we'll use the factor function to convert this into the factor. And then here again, we'll give credit card dollar class and then comma levels. You can see 01. So we are assigning G21 factors. This to this. So this way it will be converted to the, convert it to the factors. So earlier it was Integer. Now, if we see the structure of the dataset, if I run this again, now you can see here all other columns are nomadic and the glass is no factor of two levels. 01. Okay? So now we have successfully converted the glass column two factors valuable. Okay? Now again, we'll see the structure of the data set and notice glasses factor. Next thing is checking for that missing value. So whenever we get our dataset, the first thing we check for null values. If there are none values, we need to tackle them. We need to handle the null values by putting some losers, removing the columns, floors, which are containing more number of null values. So there are a few techniques that we follow to handle them missing values. And here to check the missing values we use is dark any and reparse the dataset. And thumb off is an a will give us the total number of missing values in the dataset. Some of is dark NA, and then the dataset variable here will pass. So let's, let me run this. Now it is coming zero. It means that our dataset is containing no missing values so that ignore missing values. So we need not to worry about that. We need not to do anything for that. Next thing is getting the distribution of fraud and legit transaction in the dataset. So now for that, we'll use the table function and we'll pass the glass column because class column is the distribution factor that will tell which one is fraudulent and which one is legit transaction. So credit card, dollar, class, and tables. So this will give us the number of transactions and number of fraudulent transactions. So zero is 28,004 to like 84 times three we obtain is the ledger transaction one lift 492 or the fraudulent transactions. So here with this week, gamble know that this is the imbalanced dataset because most of the dataset, each having their tangents and agile legit transaction and lonely two-legged 34,300.492 as a fraudulent transaction, which is very less compared to the ledger transaction. So this is one kind of imbalanced dataset. Now We'll see what is the percentage of legit and for Berlin dataset, fraudulent transaction in the dataset. So to do that, we'll use prop dot table function. And we'll pass this table, and we'll pass this column class. Okay? So let me see. Your 99.8% is the fraudulent, sorry, later Angie, Excellent. 0.017% is the fraudulent transactions. So next thing is, let, let me put it this into the pie charts. So to create a pie chart, what we will do first we'll create a logic and product two vectors here. And then we'll assign it to levels, and then we'll create another level. And we'll use the paste function based on these labels. So we'll take the levels, this legit, and then we'll use the round hundred into prop table. This will give us the percentage of the percentage of the legit and fraudulent transactions. And here comma two and we went because I want that here, it is a many digits after decimal. I want to restrict it to the digit and then Levels, levels and percentage. And then we'll use the pipe function. Here. They'll pass the table K plus this column name on which I want to plot up ICE table and then labels this legit and fraud. And then column A collapse or each color has a green Android. The greenish part, the leptons excellent read further, fraudulent and manage equal to pi. This will be the heading for our pie chart. So let me run all this together. And he had really get done by chart here, legit transaction in 99.83 and fraudulent and wine 17. Let's run this again. Here. The pie chart is your ledger clinics and his 99.83 and that's three, everything is coming green. And the red one is for only 0.17% ready less. So this is a truly imbalanced dataset. Now, what I'll do, I'll just try to predict with normal model prediction will not use any machine learning algorithm and religious try to predict without a bottle. So for that x and y label and yet a huge repetition or function, they're brought into GTO. So what I'll do and row credit card. So in our dataset for all the rows and ROI will give that number approaching that Kate card dataset. And for all the rocket will repeat zero, it will fill in zeros. So all the project will be becoming as a ledger transaction. And then we will use the factor predict sense and levels G row one. And then let me run this. Okay? And if I put predict sense, Okay, and predictions, okay, and then I'll use the character library here. Because I'm going to use the confusion matrix. The confusion matrix, and a boss, this data, as you call it a prediction, this one and references the grade card dollar class. And if I run this confusion metrics, see that it's charged up prediction. See the accuracy of this model is 99.83. So all the all the transactions has been classified correctly as a legit. All the ledger transactions are correctly classified as a legit. And this disease, the additives and this is negative. So this for 92 or the fraudulent transactions that are not classified as fraudulent transactions. So all the transactions in the dataset has been classified as a Leggett and exons, even this for 92 tan x and recharged fraudulent. Our prediction model has not got endured as a as a fraudulent transaction that has gone to the ledger transaction. So because this is truly imbalanced dataset, our prediction model is predicting everything as a legit because it is not getting trained on the fraudulent data because it is very less and that's why everything is going into the true positive rate. So this is the prediction based on the normal model prediction. We have not used any machine learning models and algorithms. So in the next lecture, we'll use a machine learning algorithm to handle this imbalanced dataset. And we'll try to predict in a better way. See you inside the next lecture. 91. Creating Training and Test Datasets Sampling: Hello and welcome back. So in this lecture we are going to move further into our project. And what I'm going to do here, I'm going to take the thing. What we'll do, we'll take the sample data from the gate current data that we have. So what we'll do, we'll take the ten per cent of the data from the existing dataset at the sample. And on that, we'll try to create a model. And then when it is giving correct predictions, we will apply that model to the data set. So to do that, what are the things we can do? I'm going to use a D-flat library. So you just download it. If it's not download it. And then you did library, we're going to use and then set seed. One, this report, because when I take 10% from this credit card dataset. So if you don't put set seed pool, then what will happen every time it will randomly take the ten per cent from the whole dataset and each time it will be different, ten per cent. So for that reason we want the same setup, 10% every time we run this code. So far that I lose one. Okay, so let's run this. And then Cape Cod. Okay, I refreshed it. So let me try to run all the code again. Okay, so see now we have taken the ten per cent from this card. This is the whole dataset from this, I'm taking sample, underscore frack, this function I'm using and I'm giving 0.1, 0.1 min, 10% of the whole dataset. So I'm taking the flux of 10% bracket them from the dataset and assigning it to this gate guard. Valuable. Okay? And then when I run this, you can see now we have done same setup, rows and columns. Okay? Then I'll use table, and I'll use this class. So let me run this. Now. We have the grantee here toggle back and forth or something. So 10% of that. Okay, and here the legit is 28,000 and no Protestant and sell 44. Okay, so now we have taken the ten per cent from the whole dataset. What are the things? Now I'll use GG plot to just use this. And if not downloaded, you download it and install it and then use it. Okay? And here what I'll do, I'll try to scatter plot here. And for this year though, this dead person that we have created just now, anesthetic can use the x and y-axis. We do column and colors and again, made under class last-minute zero or one. A legit dot front of length, okay? Geom points, not, not rent in black and white. Team up your graph. So that will look black and white. And scale color manual here in blue and black. Blue and red, sorry, not black, blue and red. Bluish color. The Euro and war for one red and blue each ledger transactions and red or the fraudulent transactions. So let me run this tool Okay, so see here, now we got this scatter plot. The class representing 0.1 representing, but still it is imbalanced. There are so many Jackson who only few of them on the fraudulent. Okay, so now we've got the 10% of the data and we have plotted it successfully. Next thing is we need to create the training and test set for the fraud detection model creation. Okay, for this, I'm going to see tools. If you have not installed this, you can install by using install.packages. And you can see towards the back-end for it, you want to install the library. They all use library. And a buys this name Seattle's. And I've done this, then I'll use that to seed 123. And then I'll take the data sample. And what I'll do, I'll split this sample data that we have taken 10%. Now, this 10% dataset and split into 80, 2080 for the training and Bronte for the test. Yet, I'll give the sample dot split the credit card class based on this, you know, and one the classifier and then split race you, I'll give pointed, pointed 80% for training and 20% bought the dashed. Okay, and next thing is trained data. And take the subset of this data sample and data sample subset, functional use and grade card. And did a sample is called a tos 2 min. Let me run this first and then I'll tell you why you didn't get true and false. Okay? So when we run this data sample, that doesn't police containing true false, true, false, true, false. True is for ledger transaction and falses for the pulses for the fraudulent transactions. Okay, so now we have the sample here. So what I'll do, I'll take the two ALU and I'll get the false values here. So let me run this trend data and this will be the test data. So training dataset and this dataset. Now, if we want to know the diamond sense of that train and training dataset and the dataset you can use a dim function name, train underscore data. 22,007, 85 rows and 31 columns. And for theta 5,696.31 columns, the rows and the columns. So this way we can get the train and test dataset. The next lecture we'll move further into the creation of the predictive model 92. Random Sampling Methods Over and Under Sampling: So now we have the test data and training data. Let's handle the imbalanced dataset. So there are two, there are three approaches that we're going to see in this project. The first one is random oversampling. And then we'll see the random undersampling, and then we'll see both of them together. Okay? So let me start with the random oversampling. So what is random oversampling? Random oversampling means we need to put a 50, 50 per cent of the legit and fraud cases. Okay. So for that, what I'll do, I'll use the train data and over this glass table. And let me run this. So see here now we have the legit rose, 22,007, 50 and for our landmark 35-year leak. Okay, So this is imbalanced, so we need to random oversampling this dataset. Random oversampling means we need to put both of them as per pupil agreements. We need to make both of them equal. So for that analytic number of transactions are called collagen, Ashley 22,007, 50. And what I'm want to do new frac sand often legit, well is how, what I want 50%. Okay, so now I'll collect the new and total, new and total number of rows required in our new dataset over sample dataset. So here what I'll do, I'll new legit allergic to this one divided by the fraction of logic gates as we want so that we'll do and we'll get a new number. Okay, so let me run this. So this is 45,500 rows. We need as bus this, okay? So now I'm going to be used Roche package here. So if you have not installed, you just install it. Otherwise. Just huge. It is pretty solid installed here and create a valuable oversampling and that's called a jar. And he had a huge boon dot sample method. And here I'll give the class as a variable, independent variable. And this sign dot will give all other variables available in the datasets. And then comma and then data is equal to, we are using that train data, train underscore data that we have created. And then method is oversampling. And n number of roses, new and total that is coming to 45,500. And seed I'm using here, so that will get the same result every time. So let me run this. So now it is run. Now, if we look at this or look at the random oversampling, they charge, you can see here 45,000 Raj. Click here and you can see the next thing is now, if you want to see the data, we have to use the oversampling and that's called result dollar sign data. So it will give us the dataset. Okay? Now, if we look at this dataset table, oversampling, credit will see there 45,500 rows and legged cases at 22,017, 50 and fraudulent cases are also printed tutorial. Right? Now, what I'll do, I'll just plot this with the help of GG plot. So here, theta is equal to, I'll take this oversampling great data and v1. Why? We do class color, class, class and geom point and thin blue, the same thing which we have done earlier. So let me plot this and see how our data is looking. Now, I've got oversampling. Okay, Because I click price. See here now our new or word sample dataset has been like this. This is the scatter plot and why we are seeing still, we're seeing very less number of farmland cases as read. Most of them are looking at blue only because for each Fraudulent cases. We have created a duplicate entries while oversampling. And that's why each of these will contain more number of duplicate values. So that's why behind this, there are more points. Okay? So to look at, to plot it better, we can use the t-test. The t-test will give you more visibility on this. So let me tell you here. Geom point for Giussani is quadruped position, underscore detail and width. We can put 0.2 or I can put even 0.5. And let me run this. Now these points will be a little bigger details. So that will come to know that there are more points. Not only this point, because while oversampling, not duplicate, it will create the duplicate of the existing data. Could see here. So now you can see there are more points, bigger points. So this is, this is because of the duplicate things. Okay? So now we understood what is oversampling and how we can do that. Okay, oversampling is there is a problem with something that we may create the duplicate points. Now we'll see the random undersampling. For undersampling also use the same. Then underscore data. And then here, number of fraudulent rows are there in this. Just start if I write N, fraud will be 35. Knew what I want. I want 50 per cent of the total dataset. And new and total will be n fraud divided by 0.5. So that will give us the total number of roads required for the random undersampling. We're now on this. New and total will be 70, So there will be one list, 70 roles required. If we do under sampling that is quite low and this will lead to the loss of beta. Okay, Again, for others applicant, so I'll use the package. So here I'll create undersampling, underscore, adult and own sample. Same thing I'll use. Everything will be same except this metallurgical to under method will be under sampling. And here we'll pass the new total and seat. And let me run this. Now we have done under sample data. Now you want to see the dataset, USDA, under sampling the data and undersampling great writer will alkylate. We run this. Now we have this undersampling. What Lisa Monday rolls out there. And half of them are legit and half of them are fraudulent. But this will lead us to the loss of data if you plot this same thing. Let's see. Here. Legged and the fraudulent cases are almost same. But here we have lost almost 22,700 rows. That is not good for any analysis or any prediction data. So in the next lecture, we'll combine both of these undersampling, oversampling together and we'll try to predict it 93. Using ROS and RUS together for Data Balancing: Hello and welcome back. So now we have seen how we can work with the random oversampling ROS. And then we have seen how we can work with that random undersampling, oversampling and undersampling we have seen. But both of these has some disadvantage like oversampling will also cause sores and undersampling will cause the suave or deletion of most of the rows, which may impact the decision-making on the prediction and our models will not be accurate. Okay? So approaches using both of these together, it means that random oversampling and undersampling, both of these will do together. So this method is called board. So everything will be same here. What we'll do, I'll use and new and Andrew, the number of rows in the training dataset. So that will be, it will be 12,000 study grantee tutorial 785. Okay. So here we can put in command and new intelligence 785. Now, frac sand fraud, new, what we want to do is find 5%, 0.5. Half of that would be the fraudulent and half of the data source ledger transactions. Now, we'll take the sampling, so I'll use the dot sample and the same thing which we have used underclass parameter. And then all others will come up with this dot solve other columns really come then data train underscore data method. Here. Earlier we have used the method under and over. Now we'll use both. So this will do both under and over sampling. And you can do an underscore new and be equal to flex and underscore fraud or misconduct knew the wind frame. Then we're still seed. So that will get the same retarget. Next thing, let's run this. And if you want to see the dataset sample of kids where they will alkylate and sample, sampling underscore dollar data. So this will give your sample because they jumped. So this is the data. Okay? Now we'll create our table. So see you now. 11,004, 31 is less than or equal to five. So almost same number up legit and fraudulent cases. So this is kind of little balance. Right? Now we'll create the prop table. And here you can see the percentage of the, so legit, tragic, they're not appropriate person almost. And 49% is the fatherland transactions. Now, if we plot this distribution, we'll use the same exam part, this one or this dataset. And we want to and redo x and y. And cold colors are made on the glass and geom point G and 0.3. Okay, let's run this. So C yet this is the scatter plot. And see you on the blue dots are still, you can see it is coming more from, but actually it's not that thing because we have a robust sample and another sample or sample it. So the legged cases with less in the dataset, so it duplicates of those admin create and that's where it is overlapping. So red and blue are almost same, but rates are overlapping. One on the right is looking like less one, but you can see it. I've used the detail here. So that's trait is coming as a little jittery. Okay? So now we have seen how we can do the oversampling, undersampling and using both together also. Next thing, we will use the synthetic minority oversampling technique to balance the data. That it's called smart technique. And for this, we'll use the Smart family library also. And in the next lecture we will learn more about sport and then we'll write the code for the smart technique for synthetic minority oversampling. Okay, so see you inside the next lecture. 94. Advantages and Disadvantages of SMOTE: In this lecture, we are going to learn about small synthetic minority oversampling technique, which is a very popular over sampling technique, used to address the problem of class imbalance in machine learning. It works by creating synthetic minority class samples that are similar to the existing minority class samples. This helps to balance the class distribution and improve the performance of machine learning models on imbalanced dataset. So this will actually solve the problem that we faced in oversampling and undersampling methods. That's why it is called synthetic minority oversampling techniques. So that transaction which is in minority, it will over-sample that minority class, but in a very synthetic week. So still this is also an oversampling technique only, but little different. So we'll see how it is done. Here are some of the advantage of smart, smart technique. It is really simple and easy to implement technique. It can be used in variety of machine learning algorithms. It can be effective in improving the performance of machine learning models on imbalanced datasets. So the first thing is it is very simple and easy to implement. And it can be used with a variety of machine-learning algorithms as well. And it will improve the performance of machine learning models. On imbalanced dataset. There are some disadvantages of smart as well. It can create synthetic samples that are too similar to the existing magnetic class samples, which can lead to overfitting. So this can lead to the overfitting because it will over-sample the database on the existing mandatory class samples. So it may lead to the dataset in such a way that it would eat it can over-fit the model and our model will not predict the correct outcome. It can increase the noise in the data set, which can also lead to overfitting. It can computationally expensive, especially for large dataset since we're oversampling. And if the dataset is already large, that it does, it will be more larger and computational time. And the expenses will be really high because it will take more time to do computation to any apply any methods on that. So overall, more synthetic minority oversampling technique is powerful technique that can be effective in improving the performance of machine learning models on imbalanced datasets. However, it is important to be aware of its limitations and to use it carefully. So now we'll see what are the disadvantages and advantages are first, smart technique. Once again, the advantages EEG and simple to implement and disadvantages it can create or overfitting can be used on a variety of machine learning algorithm can increase than rice. They said one test. And can we factor in improving the machine-learning pump performance models performance and computationally it will be expensive. So now we have the basic understanding of smart and how, and what are the advantages and disadvantages are ripped through. In the next lecture, we'll implement the smart technique in our project 95. Applying SMOTE Technique on the Training Dataset: Hello and welcome back. In this lecture we are going to write our code further. Smart technique, synthetic minority oversampling technique to balanced the dataset or the technique used in GitHub dataset which is imbalance. So here we need to install the backend install.packages multifamily. So I'm going to use the Smart family back in here. So if it is not installed, you just install it by running this command and then use libraries spot family. Okay. So first thing we need to run this. Okay, next is use the train data on the glass. So see here now, this is our oriental, our training data. So 22,007, 50 is the number of the cases, 35 is the broad land cases. Now, we'll set the number of fraud and let him make guesses. The desired person disciplined med gases. Okay. So so number of guesses is this much in the art, original training data and the prevalent cases at 35. And what we want our gyro, I've given 0.6. What does it mean? It means that I want the, our new dataset. After this smart technique. I want it to be like 60% of the legitimate guesses and 40 per cent of the fraudulent cases, okay, So 60, 40 this year I want to ashtray, I am giving yet Arg2 is equal to 0.6 min, 60% of the cases, and 40% of the frontline guesses will be that in our new dataset. So how to calculate top sites? So when we, if you, if you look at this mode function, synthetic manner, you can put F1 and you can see this synthetic minority oversampling technique here. So this is through dendrite, synthetic brush to instances, instances using smart algorithm. And what are the parameters? Tx, dx, dx, that is our dataset. Target, is that the column on which we want to target, like in our case, is the glass. And then k. And then we have the dub, dub, dub sizes. Number of times c of x is the DataFrame of the numeric attributed dataset. Target is a vector of the target class corresponding to what it does if dx k, the number of nearest neighbors during the sampling process, and then double underscore size is the number or the vector representing the digit times up synthetic minority instances over the original number of majority instances. Okay? So how many times you want this smart technique to we run that we need to decide and how we can decide. We can decide by using this formula. Here, I'm using formula one -0/0 into n divided by n, n1, and that will put minus one. Okay? So this is the formula. This formula will give the n times which will be dumpsites equal to the dub stage. So let's calculate this. Let's run this and let me see how much. 422 times it needs to run. Okay, 432 open, 334433. Okay, so now we've got the dub stage also. So now we'll use the smart underscore output variable to store the data will get from here. So smooth function we'll use and then x, x will be our training dataset train and thus coordinator. And then here I'm using blank comma and then I'm going to do minus c one comma one. Why am I doing this? Because in our data set, in our training dataset, in our training dataset. If you look at here. So v1 to first column each timestamp that we don't need. That's why I'm moving this first column, timestamp and then we have another one. The class. So class also I'm removing Chrome. Yeah, so that's right, 1.31 first and last column I'm level. We then target is train underscore data Target column is class known as class. I'm giving five and upsized is equal to n time. So let me run this. Okay, now, if we run this and if you want to see the data, it will be on the data that they want or dollar data. And we'll run this to look at the data, can see here. Now the smart new dataset is having V1, V2, V3, up to read 28 and then amount and then the class, okay, So now the class here is lowercase c. Earlier we have used it was in the capital. So to remove confusion, we'll just change that to uppercase C. So to do that, we need to use the length function, which will take the data set name and then the column number. And then you can give whatever name you want to get to that column name I'm giving your class. And when I run this, it will be changed. And if I click on again, so now the class is in uppercase, Okay? Now, if you want to see the percentage of whether it has been disturbed, the oversampling or sampling has been done by S2 for two days. You are not. So we'll use the prop table that we have used earlier. And here we'll pass the credit underscore small datasets, the class column, and we'll run this. See, now d rho is 60% and one is almost 40%. So now our data set is distributed like 60% of the cases and wanting percent is the fraudulent cases. Now, we'll compare with the original scatter plot. This is, this is the original scatter plot that we have done on the training dataset. So let me show you how it was looking. See it looking like this. I didn't know the distribution of the data. The fraud cases were really little. And now I'll plot the new one, correct. And that's because smart, smart technique we applied. Now you can see now our data set is more balanced. And this more technique gas field though it does it with the with the more number of fraudulent guesses here. And these are not the overlapping or duplicate values. These are the synthetic point that has been filled with the smart technique. So now we have the dataset, imbalance dataset. Next thing what we need to do, we need to create a decision tree and we need to predict the fraudulent cases that we'll do in the next lecture. 96. Predicting Credit Card Transactions Cases with the Model: Hello and welcome back. So now we are good to predict fraud cases based on the data set that we have created thus mode. So let's predict the value based on our model. So for that, we are going to use the rpart and rpart plot packages. So you have not, if you have not installed, install it by running these two commands. And then library rpart and library our dotplot. Okay? And then here I'm usually guard model as a variable. And here what I'll do, I'll use the rpart function. If we want to know what is rpart function, you can put rpart F1 and it will give you the recursive, our partners recursive partitioning and regression tree. So if we want to know more about this, you can read the documentation and what are the things that takes it takes the formula laid out with subscript x and our part and model faults and all those things. Okay? So you want to read in detail, you can go and read it. Okay? So next thing is bark, and then we give them the columns or the dependent variable. So in our case is each class and then sign and then dot means all the independent variables. So dot will take all the, all other independent variables. And then here we'll use the dataset. Great honors course. More data. We have, we have made through applying the smart the market. So let's run this. So now you can see the GARCH model is here. And okay, next thing, we need to use the R. Now, I'll make the decision tree for this. So let's see how our model predict, classify things into them. Legitimate cases and the problem says, so it takes the lesion. So our plot, our part dotplot and then we'll pass this. And then extra is equal to zero type integral of fiber. And quick 1.2. Okay, So let's run this and see, yeah, this is the decision tree. This 1.1, 0.2, it will boil this thing. Okay? So this is the decision tree, and this is how our model will decide which one is. So suppose this is before. If the value is greater than or equal to -2.5, it will be classic applied as a legitimate case, and it produces less than 0.5 -2.5, it will be as a fraudulent guests. So this is the decision tree it will follow, okay? And then if you want to see the predicted values, so we can use the predict and then we'll pass the GARCH model. Then Daesh data on data that will predict your type class. It will classify zero or one. So we are going to run this model on our test data and see how it is predicting. So let's run this. Okay? So now if we look at, can see here, now, for a play thing, it has predicted for fourth row, like this, 0.1, it has been classified. So now, if we want to see what is the accuracy level so far that we can create a confusion matrix by using the Keras library. So huge carrot and then he either knew the convergent metrics. And I'll pass this predicted value here and here. Data on the glass dependent, dependent variable. Okay? So let's run this and see here the confusion matrix. So out of our total nine fraudulent cases where there and hold up 97 has been seven prevalent cases has been classified correctly and to wear not classified correctly. Similarly, for the legend red cases, almost all the cases classified correctly. So the accuracy of this model is 98, almost 99%. So this way we have implemented the grade card fraud detection model using our programming. And we have learned so many things like how to handle the imbalanced data and what are the algorithms we can apply? What are the techniques of water melon balancing the dataset? So I hope you understood if we have any doubt, you can ask in class commenting or by asking a question. Thank you. 97. Introduction to ggplot2: Hello and welcome back. In this series of lectures. In coming lectures, we are going to learn about GG plot. We have been using GG plot too many times in our lectures. But what I opt job that many of us reuse plots like bar chart, pie chart. And there are many jars or the right-hand men libraries be huge. Like GG Plot tours, GG plot. But we missed to know the basics behind that. And we should be knowing that. Keeping that in mind. I'm creating a few lectures on basics of GG plot two. Similarly, I'll be creating more lectures covering all other topics as well, the basics of all those things so that we can have a clear understanding of things. So DJ ggplot2 is an R package for creating data visualizations. It was developed by Hadley Wickham, has an implementation of grammar of graphics. Will see what is grammar of graphics, which provides a structured approach to creating an customizing plots. The package is built on the principle of letting aesthetics, mappings and allowing users to create highly customizable and publication quality graphics. So GG plot to Deb loved to make public keys and 3D graphics. So before that, we use to create graphics, but that we're not publication ready. So if you want to create a public isn't 3D graphics unit to go and use the tools like Power BI or Tableau, some other graphics tool. But GG plot two has the capability to create publication quality graphics. And it is very customizable. So whatever layers you want to add that users can add based on their requirement. It follows the grammar of graphics approach that is very, very important and that's why it is having so many capabilities to create awesome looking in graphics and charts. Okay, so let's understand some of the features and characteristics of GG plot to plot two follows the grammar of graphics GG plot two. As I said earlier, problems that grammar of graphics framework, which means that plots are built by combining different components of layers such as data, aesthetics, geometric objects, and statistical transformations. So it not only just plot the data, but it takes care of the data, aesthetics, geometric objects, and statistical transformations as well. Statistics, aesthetics and mappings. Gg plot to allow you to map variables in your data to different aesthetics, attributes such as x and y coordinates, color, shape, size, and many more. Did mappings define how the data will be visually presented in the plot. Next feature is layering. The first one is grammar of graphics, and then we have the aesthetics and mapping. And then we have the third key, features and characteristics. That is layering. Lots in GG plot two are built by adding layers of graphical elements such as each layer represents a different component of fluids such as points, lines, bars are text. So it is built by adding layers, one top of the other. And the layers have different components like plot, component of the plots such as points, like geom point to use right? Geom line reuse. Then bars we use our text layer will also be there. Layers can be added and customize independently, allowing for complex and layered visualization. Statistical transformers and GG plot to provide such a wide range of statistical transformation that can be applied to the data before visualizing it. These transformations include aggregating data, computing summary of statistics, smoothing the data, and minimal, highly customizable GG plot two offers a high level of customization options, allowing you to modify almost every aspect of the plot, including access, labels, titles, legions, colors, and themes. So it not only provide you to like a Teams, but it can also provide you to customize things like axes, labels, titles, legends, colors, and it has many themes which you can apply This flexibility enables you to create plots that meet specific requirements on match your preferred visual style. Support for faceting, GG plot two supports faceting, which allows you to create multiple plots or panels, bid on, belt on one or more variables in your data. This is useful for exploring and comparing different subsets or categories within your dataset. Integration with our GG plot to seamlessly integrates with our packages and data manipulation tools. You can easily combine GG plot two with data manipulation libraries, such as the playa and tidy R to preprocess and transform your data before visualization. So overall, the GG plot provides a powerful and intuitive framework for creating a wide variety of medulla Jason's, ranging from simple scatter plots to complex multi-panel faceted plot. It has become one of the most popular and widely used data visualization packages in R due to its flexibility, aesthetics, and emphasis on good design principles. So now we have the basic understanding of GG plot too. Let's dive into that Recording and let's create some basic plots using GG plot two. So I have created a GG plot two dot R script file. And here we will try to use the GG plot to create a basic graph and plot, and we'll try to understand the different layers added to it. The first thing we need to do is we need to install the ggplot2 package. So if it is not installed in your RStudio, you can do that by running the script install.packages, and you can give the package name GG plot two. Once packages installed, you can use it by writing library and then given the package name here. So this will enable you to use the GG plot package. So first step is to load the package, install and load the packages. The second step is create a basic scatterplot. So for that, what I'll do, I'll first create a DataFrame. So to create a DataFrame, I'll create a variable here data. And then I'll use data.frame to create a DataFrame. And here I'll provide the x points, x vector, c12, 345, and y vector and y-axis for y-axis and gifts, see 246810. So 12,243,648.5 ten will be the coordinates for our. So this way, I'll create a DataFrame. Now. I'll create a scatterplot using GG plot two is pretty simple. We'll use the GG plot function here, GG plot. And then we'll provide the data, that DataFrame that we have created. The data on which you want to create the plot. So data will give you, then the next thing is aesthetics. Aesthetics will give x and y axis on which you want to plot the points. So x is equal to x and y is equal to y. So from here it will take, and then we'll give the plot, sorry, plus geom point. So geom point, we'll plot the points. It will take the points from urine plot. So let me run this. So see here now our scatterplot is ready. See, the first point is one-two, x-axis, and y-axis two. Then 242.4 to four is plotted here, then 36, and then 4.5 tenths. So these points have been plotted correctly using GG plot two. So let me explain you a little more about these things. So we create a DataFrame called data with two columns, x and y. Gg plot function initializes a GG plot object that texts the DataFrame in its first argument. And then the aesthetics function here we have used that to take function specifies the aesthetics mapping X to X axis and Y to Y axis. So here we're mapping the x-axis to x and y-axis. Why? And then the geom point fonts and adds the points to the plot. So geom point function, what it will do, it will take the points from this and it will plot the points on the plot. Now, the next thing is we'll try to customize the plot. So what we'll do, we'll try to customize the plot by appearance. So customize the plot appearance. So here we'll use the GG plot function data aesthetics will be same. And then plus geom point. And then after the geom point, what we'll do, we'll give seepage equal to 21 And fill color will be blue and the color will be black. And sites, I'm giving you three. Then we'll use the labs function to create title. So title of the plot will be the scatter plot. And then what x-axis, we'll give the name x-axis and y-axis will give them y-axis. Then again plus. And then we'll use, we will use the theme here and we will use theme underscore, minimal, minimal thing we're going to use, yes. So let me first run this and then explain you the user various points. Now our scatterplot is ready and you can see Once I've been plotted here, but this is the, since here we have here in blue, the points are filled with the blue and the outer part is black. So here we have given black. If I put it orange, and if I run it. Now, alter parties orange. So let me make a little bigger to nine. The IP and port grantee and size and 31, sorry. So now you can see here the size has been increased. Okay. So this is the YuJa see pants I feel is blue if you make it yellow. So this will become yellow. Okay? So this way we can customize and you can make it run. And see this is the scatter plot heading down here, and x-axis, y-axis are coming year. And team we are using minimal. So let me run this again. So size will be okay. Next thing is geom point font, some texts that is no arguments to customize the appearance of the points. Here we said the grantee one is full circle and a fill color to blue, outline of color to black and size three. So this was the earlier. Now we have changed two sides level called outer color is orange, fill color is yellow. Okay. You can put F1 by selecting geom point and it will give you the description on the geom point, the Algoma and data mapping. Okay? If you want to learn more, you can go and learn more about the geom point from you. Save. You can see here if you want to learn more about the CEP, you can come here and you can learn about those safe. The safe aesthetic can be specified with an integer 0225 or single character, which uses the characters at the plotting symbol to dry smallest rectangle that is visible about one pixel. And it will draw nothing mapping to a discrete variable. So if we put here 25, let see what is being plotted here. The other triangle has been plotted if we put two. So as you see, the triangle without filling, okay, then one will be the rectangle. One is circle five. If you put five different, say that rectangle again. Then for eight, let's see what is being plotted here. Okay, Lines 11. So you can see, so you can explore and you can use other things, okay? Then we have the labs that will add the title to the x-axis, y-axis, and the name. Then Tim minimal is the function applies to minimalistic theme to the plot. We can use another theme that is black and white. So next what we'll do, we will learn how to add multiple layers and annotation to our plot. So here everything will be same geom point. On these things will be same geom line. We will add a geom line so that one line will be drawn and then we will write some text here. So eNodeB decks x is equal to three, y equal to zero, and level 999 labs titled will give the same and team will use them minimal. So let's run this. So see here now, one genome line has been drawn by joining the points. And see here for 3.6, 3.6 points, we have written texts, so text align is written here, labeled line. You can change it to anything. Suppose, suppose you want to write anything here and just run this. Now it is written anything. So you can put anything here. So line. So we'll run this again. Whatever you write, it will be coming here. So here by 3.6, the x-axis three and waves is six. We have annotated it without texts, and we have given the texts and level name as line. Okay, and team we are using minimal. Okay, Next thing is, so geom line from Sonata line to the plots annotate once and reuse to add extra notation. Here, the specified text to display as well as x and y-coordinate. And you can modify x and y levels augmented by annotate function. Okay, next ditch fast setting and themes. So we can use facets to add multiple plots to a lot. For that, we'll use the data.frame will create a variable data underscore facet, data.frame and x axis. X and y point will give and then group. Here I'm creating group by each point will be attached to a group E, a, B, and C. So now we have groups here. Now let's run this. And then now create our faceted scatterplot. So first let me create the scatter plots. Now we have the points. And then other ABC categories or groups are there. So geom point, everything except for, except rap. And here we'll provide the group, okay, this sign and group will take the group from here and it will facet. So now we have ABC groups and everything else is same thing. We are using minimal, again, facet underscore wrapper function is used to create faceted plot based on the group column, each group Harris on panel hockey. Now, we'll change the theme to black and white. Theme underscore v, w. We'll use, and we'll see what it's getting to see here. Now, the theme has been changed and now it is showing clear, whoops, ABC. It has to point B to point C has one point. Okay. So be themed punk song changes the plot, him to black and white. And this GG plot two has various built-in themes such as underscored minimal, theme underscore classic and theme underscore gray. So let me copy this and try to use the t underscore gray. Now it is in gray. And then team underscored classic. Classic is like two groups here and then points. Okay, so let's make it today. Okay? So this is the basic of GG plot to I. I suppose you learn and I encourage you to do more practice by creating some points and plotting various plots on that. And playing with the geom points, adding the labs, I think the same playing with the same leg segments. Wondered 25. So you can put various shapes and dry what numbers mean to the same, okay? So these are the things you can do 98. Scatter plot and jittered plot: Hello and welcome back. In this lecture, we are going to learn about another important plotting thing that is scatterplot and do dot plot. So let me tell you we're going to see some example. And for this I'm going to use the inbuilt dataset, that is Iris dataset, which is available with the R. Okay? So let me briefly tell you what a scatter plot you must be knowing what a scatter plot. But to just refresh your memory, I'm telling you scatter plots are similar to line graphs, which are usually used for plotting. The scatterplot store how much one variable is related to another. The relationship between the variables is called as correlation, which is usually used in statistical methods. Okay? So this is the definition of scatterplot. And for this gambled, for this lecture, I'm a handwritten accord. And here I'm using the iris dataset, which is very well-known dataset that contains measurements of your print, iris flowers, sepal length and petal length and all its contents. So for this, we need the iris dataset that is inbuilt with the data how our package, and then we need GG plot to that we have already downloaded. And if not downloaded, you can use install.packages and the package name GG plot two. And it will reinstall downloaded and installed on your RStudio. And then to use the GG plot two, we need to use library and then we have to pass the backend name. So library GG plot to really enable us to use the GG plot functions. And then to load the iris dataset, we just simply need to write data. And then we need to pass the dataset name datatype is the data iris. It will load the iris data set. And when you click on this iris dataset, you can see here it contains the eye, this Florida tails that is sepal length, sepal width, petal length, petal width, and the species. So there are species. Okay? So in this the one property objects are there and private labels, one per p rows and five columns are there in this dataset. Okay, So let me go back to the code. So now what we'll do, we'll do the, we'll create a scatter plot. So to create a scatter plot, we'll use the GG plot function and we'll use the dataset name here. That is dataset to Iris dataset. Then for aesthetics, for X axis, we'll plot sepal length and y axis. We'll plot the sepal width. And then we use genome underscore point function to plot these points. It is pretty simple. Gg plot function then dataset name than aesthetics. We need to pass the x and y coordinates. So x bar, X coordinate will use the sepal length. And for y coordinate will use the sepal width and geom point function to plot these points, geometrical points. So when we run this, we get this scatter plot here. So now all the points on sepal length and sepal width have been plotted here. Now we use GG plot function to any slider GG plot object and specify the dataset. That's what we are done here. Then the aesthetic function here, we are losing the aesthetic function. Inside GG plot define the aesthetic mappings with sepal length. What we're doing here, we're mapping the sepal length, sepal width. Sepal length is mapped to the x-axis, and sepal width is mapped to the y-axis. And the geom point function adds points to the plot. And thus we get the scatter plot so your distort. So we are using the GG plot function inside that we are passing the dataset name. Then we allergen the aesthetic function to assign the coordinates. Map though, column name or to the x's. So x is equal to so-called outlets and y-axis sepal dot width. And then we are using Jin geom point function to plot the points on the scatter plot. So we are getting this plot. Next thing is, now we will customize the scatter plot. So to customize will pass again. We'll use the GG plot. First we draw the plot. So for that GG plot, then dataset name that aesthetic. Here, aesthetic, same thing we are reading. And what we're customizing. We're customizing by adding a color to it. So color is equal to species. So All the species will be a specific species, specific length and sepal length and sepal width will be colored. So for each species there will be one color assigned. So color is called the species. Then we are using geom point. And here, and here we have just plotted geom point fonts. I'm using geom point function here we'll customize by adding arguments to it, like psi is equal to three and alpha is equal to 0.7. Then labs here with the labs function, we assign the title, so title of the scatter plot, we will give a scatterplot for sepal length. Sepal width for X axis will give the name sepal length, and y-axis will give sepal width. And then theme here we'll use the, we will be using the t match theme, minimal theme, and that's called minimum function. We got to use the minimal, minimalistic theme. So let's plot this and then we'll discuss further. So now you can see here, since we have given the ecological species. So all these species, there are three species and three colors has been assigned. So by looking at this, we can say that the green is for versicolor, for the pink is for setosa. Virginica is dark blue one. Okay, now let's understand little bit more. So we add colors you call two spaces argument inside aesthetic function to color the points based on this precision of the eye. These plots that we have already discussed, the sides argument in the geom underscore point sets the size of the point. So now it is three. If I make it supports our 15, then what will happen? This, the dots will be bigger insights. If you want to see, see here now the dorsal bigger in size. So as per requirement, you can, if you put one or five, it will be like this, okay? Now, there is another thing, alpha equal to 0.7. So what is this alpha, alpha argument controls that transparency. So if I put, if I increase it to one, what will happen? Let's see. See it is more darker, right? If I put wind, one, see it is not that dark. And if I put 0.8, it will be darker. Martin, darker. Okay? So I'll fire that ammeter will control the transparency of the points. Okay. Next thing is left function is used to set the title of title and axis labels. So for the access levels, the x-axis we have sepal length, y-axis, we have given sepal width. And the title of the scatter plot, we have a given scatter plot of sample versus sepal length versus sepal width. And the theme underscore minimum points and minimalistic thing. But the plot. If we can put here, if I repeat this and if I use theme underscored be black and white VW once and then what will happen? Let's see. Nothing much has been changed. Now we will get into jitter plot. So what a jitter plot? Data plots include special effect with the scatter plots that can be depicted. A specialist effects with which scatterplots can be depicted. So it will add the special effect to the scatter plot. Detail is nothing but a random value that is assigned to the dots to separate them. Okay? So you can see if your dataset is containing the duplicate values and when you plot the scatter plot, the points will be overlapped. So in those cases, if you want to know which are the points are duplicates, then you can use that jittered function to highlight those points which are having the duplicates. Okay? Jitter is nothing but a random value that we assigned to the dots to separate them. Okay? So now GG plot function, dataset name aesthetics, then geom underscore, jitter and width will give 0.2, height will give zero alpha 0.7, and all other things are almost same and your team will use black and white. Okay? So let's run this. See here. Now, the points has been separated. Okay? So this is the third plot. So we modify the x axis, a species and write access to map the petal length and the genome return from an ad jittered point to the plot. The width argument. Control the width of the jitter and height arguments and the height of the height set to zero. This regional thing, the vertical jittering money. And I'll parliament controlled the transparency as with this cult and all others we have already discussed. Okay? So this is how we can create normal scatter plot and we can use a jittered to create a jitter plot 99. Bar Plot and Hostogram: Hello and welcome back. So in this lecture we are going to learn about two more important plotting. Things are like a bar plot and then we'll see the histogram. So these are two important routing techniques, type of plots. So let me first tell you what is barplot. Barplot is also known as bar chart and it is also known as column chart as well because it is so neither column. Okay? And it's a type of visualization that represents categorical data. You do a rectangular bars. Each bar correspond to specific category. And the height or length of the bar represents the frequency, count or proportion of that category. Bar plots are commonly used to compare different categories or display concrete how Saudi displayed this group data. So a bar chart is basically used for the categorical data. Okay, so let me tell you how we can do that GD plot package we'll use. We have already installed it. If you have not installed, install it by using install.packages. And then did it not to. And then it's been a huge IT library, ggplot2. And we will simply run this. Sorry. Okay, Then it, for this exercise we're going to use diamonds dataset. Okay? So this is a built-in dataset in GG plot to load data. To load this data in this dataset, we need to use data and then the dataset name, that is diamonds. And so when you click on this diamonds dataset, you can see the carrot of the diamond and then cut, then color, clarity, depth, table price, x, y, z. These are the columns in the dataset. There are 53,009, 40 entries in this dataset and total ten columns are there. So this is a big God, it does it. And now step two is to create a bar chart, draw. So to create a bar chart, we'll just use GG plot function and here we'll give them, what do we give diamond the dataset name then aesthetic x is equal to cut will column. You can see here the column is the column on the x-axis. And then genome and Escobar function will use to create a bar chart. So let's run this and see the result. So now you can see here on the x-axis the tape up there, and on y-axis, automatically the number count for that cut will come here. So fair, good, very good. Premium and ideal type of goods available in our dataset. You can see here in the dataset, gut ideal premium, good premium, very World's Fair, very good. Ideal premium. I didn't like this. Okay, so now we can see our bar chart is ready. So to create a bar chart, we'll deal with that function Genome Atlas Group, bar in GG plot and aesthetics, religion x is equal to cut, so it will use the column or bar chart. We use the GG plot from the central image slider GG plot object and specify the dataset. Doesn't have these diamonds. And aesthetic function. Inside GG plot defined the aesthetic mapping with gut map to the x-axis and geom underscore bar functions as the bars to the plot, creating the bar plot. Okay, next thing is we can customize this bar chart or bar plot by using the function geom bar inside that we can pass the four diesel is equal to fill, so it will refill. And here x is equal to cut and fill will fill with the clarity. So we will use the x axis to plot the cut type of God, and we'll fill the bottle with the clarity. So the more that clarity, it will remove friends with them or color. Okay? And then geom underscore bar for dissent. And then labs. This will give the title for the whole bar chart, bar plot of diamond cut frequency and x-axis, we'll give the name cut and for y-axis will give proportion. And then team, you lose minimal. Okay, so let's run this. Now you can see here it is filled with duck clarity. Clarity is like this. The clarity has been given with dark colors, yellow, green, light green, blue, the blue. Okay. So like this We add field clarity argument inside the aesthetic function to fill the bars made on the clarity of the diamond. The poor, decent prolonged went in the geom bar, tacks the bottom, making it easier to compare the proportions. So this is one above, that is the huge off the field. Okay. You're sending people to fill. And then labs function. We'll add the title of the bar chart and the access levels and Tim minimal, create dark minimalistic team for the plot. Now comes the histogram. So histogram is graphical representation of the distribution of continuous numerical data. So now we have, we have seen the bar chart with each for the categorical data. And histogram is for the continuous or numerical data. It consists Topsy Regia bars, where each bar represents a specific range or being of values. And the height of the bar indicates the frequency or count up the data points Pauling falling within that range. Histogram provides insight into the central tendency and spread up the data, aligning the visual analysis of the distribution. So this is all about histogram. Let's create one. To create a histogram, we'll use GG plot from sin and dataset diamonds. Aesthetic x is equal to price. So X axis we'll plot the price and then geom underscore histogram function. Inside the GG plot. The histogram. Histogram for your bin width will give 500, and then labs for giving the labels and the title bar, the bar, our title farther, but sadly histogram. And then the team will use black and white. Okay, so let's run this. This is the histogram. You can see price on the x axis and frequency on here, extra rough diamond prices. So here we modify x-axis for that price. And Yom underscore histogram function, add the batch to the blood creating the histogram been without wound control the width data bins in the program. So if we put this like 100, let's see the changes. The width is getting reduced. If I make a Tao Qian, then the red will be increased. So reported 400. Okay. So this is how we create a bar chart and histogram lifts to understand a little more about bar plots and Instagram. The key differences between the bar plots and histograms will understand in terms of blood that I bought, Placement, Model, presentation, and use cases. So in terms of data type, bar plots are suitable for categorical data, where each bar represents the category or group, whereas histograms are approved for continuous numerical data. So this is already a key difference because datatype, the datatype bar plots for categorical data and for numerical data or continuous data, we can use the histogram. Histogram bars represent that ranges or interval of values. Whereas in bar, plot, bar represents the category or group. In terms of bar placement in barplot, the bars are typically even evenly based along x axis with a gap between each bar to represent different categories. In histogram, the bars that are distinct to adolescent and touch each other as they represent continuous ranges or interval of Beta values along the x axis. So this is another very clear difference. If you can see the bar, plots, the bars that are along the x-axis and there is a gap between each bar and that represent different categories. Okay, and he managed programs, indices plotted on a continuous data, numerical data. There will be no gap between the bars and they will be adjacent to each other. Because the other brilliant that continuous ranges in dumps of data representation in a bar plot, the height or length of each bar represent frequency counter proportional specific category. While in histogram, the height of each bar indicates the frequency or count up data points falling within the specific range are green. So this is, this talks about histogram, talks about the number of counts are falling in a particular range. Whereas the barplot talks about the number of counts in a specific category. And the huge guess, which is very much clear now that bar plots are commonly used to compare different category, display concrete, discreet, sorry, display discreet data, or illustrate the relationship between categorical variables, variables. Whereas histograms are frequently used to visualize the distribution or identify patterns, central tendency and spread of continuous numerical data. So these are the key differences between the bar plot and a histogram. I hope it will help you in understanding and you will be knowing that we are to use the bar plot and where to use the histogram 100. Pie Chart with ggplot2: Hello and welcome. So in this lecture we're going to do something unusual, which we don't do. Gg plot too much, that would be quite interesting to do. So what I'm going to do here, I'm going to create a pie chart using GG plot. And before doing that, let me tell you. Gg plot two is primarily designed for creating layered and grammar of graphics bit based plots does not have a built-in geom for pie charts. So GG plot does not support bicarb. However, we can still create a pie chart using g, g plot two by manipulating the data and utilizing other geoms. So let's see the example. So step one, load the packages and create a sample data. So here we'll use the download blocked we've not done. And then we use the library. Did you plot two? And then we create a sample data. For that, we'll use the data.frame category and the ReLu categories ABCD and value is this vector C, 13th, 20th, 20, be 35-day preempting the 27.10. Okay, so let's compute the sample data. This is a categorical data, okay? Now step toys and manipulate the data for pie chart creation. So here what we'll do, we'll calculate the proportion for each category. So data, when you create a variable that dialogue in yellow and then transform the data. And the data will pass for the transformation again, we'll calculate the proportion ok, value divided by sum up. Well, okay, so let's do that. And so now you can see here the data is like this. Now category value proposition, we have created a proportion, we have created, proper proportion. We have created, okay? So now sort the data in descending order up proportions of it. So for that, we'll use data order, data. Dollar profit will. We want to sort by the YouTube prop because that is the only numerical data. So to that end in decreasing true. Okay? It will be in the descending order. So let's do that. Now if we look at the data, okay, that is a new order. Now, step three is creating the pie chart, stacked bar chart. Okay? So GG plot two, then we'll pass the data. Then aesthetics we'll use x, will use nothing. Why? We'll use prop and fill the category, okay? By the category. And then geom bar, human Escobar function will give stat is equal to identity and width will give one. Then what polar will give Y starting at zero. And then you have to really keep that title. My chart and fill with the category. Team will use theme underscore wide. So let's see. You know, we have a bar chart showing the category and these are the leads. This pie chart, the pie is this. These are drawn by the help of the proportion that we have created. Okay? So let's, let me explain you a little bit more so that it will, I'll give you more clarity. So in this approach a week and what the pie chart into a stacked bar chart. By manipulating the data, we calculate the proportion by dividing each value by sum of all values. The geom underscore bar function with the stats is equal to identity. Create the stacked bars with the, with the highest corresponding product proportion. Okay? And then quad underscored polar function. What it will do function converts the bar chart into a circular shape to mimic up pie chart. Okay, and then labs will give the title and legend and theme underscore word fonts and remove the unnecessary background and human providing a clean appearance. So this is how we create a pie chart using this two. And I'll give you a disclaimer here. As I have said earlier, the GG plot is not per pie chart is not supported because it is made on the ground myself graphics. So you have to note it down that pie charts are generally not recommended for data. We learned some due to the difficulty in accurately perceiving differences in the angles are areas of the slices. Or that chart types like bar chart or stacked bar charts are often more effective for representing the vertical data. So whenever categorical data is there, we cannot use, though we have not used the bus pie chart. Instead, we can use the bug jars or the stacked bar charts. Okay? So I hope this will add some value to their learning 101. Line Plots using ggplot2: Hello and welcome back. In this lecture, we're going to explore how to create line Plots using GG plot in our programming. So line plots are useful for visualizing trends and patterning continuous data, continuous attach time series data. And we will learn how to create basic line Plots, plot multiple lines on the same Plot, and customize the appearance of the lines. So what basically we are going to learn about, learn about the line Plots. Line Plots, why we use we use to visualize trends and patterns in continuous data. Line Plots, we cannot even afford the categorical data. We can use it further. Continuous data, numerical data, or what a continuous range. The range should also via continuous. And the best example is time-series data. And we will also learn how to create basic line Plots. And we'll also see, will also see creating a basic line plot. How we can plot multiple lines on the same plot. And how are we can customize the appearance of thumb lights. There are three lines on the same plot, how we can go straight up yet in some of the lines like colors or gender now, okay, so let's get started. So the first thing is creating a basic line plot. So to create basic line Plots, we've got dataset with two continuous variables. Typically they present exits. That will be the independent variable and y-axis, that will be the dependent variables. Okay? So what are the two things we need? We need that Dataset. Obviously we need a dataset. And Datasets would have at least two continuous variables. And typically, the x-axis will be the independent variable and Y axis will represent the dependent variables. Okay? So our dataset with two continuous variables, one for gender, independent data, valuable and other will be dependent. So independent, brilliant on the y-axis. So for this, what I'm going to use, I'm going to create a simple plot of a line that will represent the sine curve. Okay, so far that let's sample leader for that I'm creating that were dependent on the x-axis, definitely with the independent variable. So here what I'll do, I'll explain it. Well, I'll create with our sequence Fontan. And he had sequence, I'll give zero comma two into pi and left or daughter, I'll give hundred hockey. So this will create the dust random variable X, that'll pop sequence and then Y, which is the dependent variable, which will depend on the x-axis. And since you don't want to Canada, same curb, the sine function you and both the X variable here. So for every exhale, so for every excel will get assigned curve, that algebra and on the y-axis, okay? So X, I'm using the sequence function, and here I'm passing the zero comma 2.2 pi and lengthened or doubt I'm giving hundred and that X value when I'm passing two sine function. So that led into sin, god. And this time, whatever, it will come based on the X. So X is the independent variable and dependent variables because, why is value-added depending on the X that we are passing to the sine function. So why is the dependent variable and X is independent variable? So this way we got, we will get the X and Y. Now, the basic line Plots ggplot2 library, so we have already installed it. So let me first give you, run this line, two lines so that Okay. Okay. So see yell if we try to print the X values, so see the data X value that we have kidney attempt with the help of the sequence function. And if I print Y, so it's going to be the Y value depending on the X value. So with the help of this sine of X will get the y-value, which is the by next Friday. Well, so now we have the And we have the waypoints. Okay, so next is we'll use the library GG plot two. And then what I'll do, I'll try to configure line Plots. Can either line, let alone the GG plot function. Both data. You're going to data.frame. And he had passed the X and Y coordinates are X and Y values. Annuli are the aesthetics function. And exit equal to X and Y is equal to Y. And plus I here I'll use the allyl tried to plot a line, so I'll use the geom underscore line. So let me run this to C. C, and now I'm getting assigned golf. So on the x-axis hundred blending, now, plotting the exploits and Y and Y. So this way we're getting up same curve made on the X and Y. So X is the independent credible and Y the dependent. For every X there is a y-value. So this is the same curve we are getting. The way we can simply lot line Plots. Simple, basic line Plots to bring into cosine, go like. So. If you want, you can go to the explanation once again. So data.frame X comma Y will create a DataFrame with X and Y variable. So X and Y variable that we have created here by using the sequence lungs and the reopen it and X values and my passenger extra the sine function. We have got the Y and X and Y values. We have seen me add water. And then we have used the DD part library and you didn't get Plot and Data is equal to data dot frame X comma Y. This will create a DataFrame with X and Y variable. And then aesthetic function, X is equal to XY. Just go like this, what this will do, this real aesthetic mapping. So X will macro the X and Y will map y-axis. And then review the geom underscore line function to add the line to the plots. Okay? So this will get done line. Okay, So this is the explanation for creating this simple line Plots UG, X, and Y, that sample later we have created. Now we'll see how we can plot multiple lines on the same blocked. Okay? So for this lecture, okay. Okay, so now we can add multiple lines to the plot by providing different Datasets are grouping the data using grouping variable. So now, let's plot both the sine and cosine curve on the same plot. So here I'm going to plot Sine and Cosine curve on the same plot. So X will be the same yet same sequence function, doodle, I'm not going to buy land that outage called 100. So X value will be the same here. Why sine and white crocheting these two variables, alkyd here and here, I'll pass the extruder sign sorted. I'll get the Y underscore sign Values. And Y underscore cost will pass to the cost function cos of X will give us the way underscore cost Values. Okay? So this way I'll get to do while loops for every X. So each X points will have sine of X and cassava X value, right? Sudbury for X, we have two Y's scores, sine and cosine. And those two points, I'm going to plot on the same plot. So to create the line plot with the multiple lines will use the GG plot function. And here you can see here rehab GG plot. And we have by the data, data.frame X comma Y because he had, why it was only one. So we have mapped VX is equal to X and Y squared Y. Here when you the GG plot function. And we'll use the plus operator here. And then we'll use the geom underscore line function. And in the line function we'll pass the data. Frame. Data is equal to data.frame X comma Y. Here Y will be Y underscore sign, right? So here we have even X comma Y simply because only one way was the only one. Why was there? Since there are two y-values. So here will give me X comma Y, X will be the same. So gamma, why each quarter, why underscore sign and then aesthetic function will pass. X is equal to X and Y is equal to Y. And Lateral gifts, sign. And then for the same thing we'll do with the why question. So data is equal to data.frame X will be the X and Y reader. Why endoscope costs? And aesthetic X is called reacts OH, or the why. And color It's called recursion. Here will give the title Azure sine and cosine curves. And X is equal to capital X and Y is equal to capital Y color function and Tim will be the minimal. So let me run this again and see the outward. See here now we have the sine and cosine curve. This is the temp1 here. And then we have the sine and cosine lines on the theme plot. See you, there are two Plots now. Two lines. One is representing or shame and another brilliant in the same. So this way we can plot multiple lines on the same blocked. So I'll just repeat it again. Now here what we have done, we have created a X by using sequence that we have used already. By creating the basic line plot. Sequence and zero comma two into pi and then dot, dot hundred. And here why underscores in? Because we are going to plot the sine and cosine lines on the same plot. So I need two points, Y points to wage why underscore sin will be the sine of X and Y underscore. Cosine will be Y underscore cost will be the cost of ZR Cosine of X. So these, these, these may regard these three points. X comma Y signed an X comma Y equals. And now I need to plot these points. So for data value, the GG plot function. And here you stood up bossing the dataframe to the GG plot function, the plus operator to pass that anything. And I'll use the geom underscore line function. And yet advisor data is equal to data.frame X comma Y is equal to wake sign and aesthetic values. The wax is equal to X, Y is equal to white and colored. Yet allergic color is equal to same, okay? And color is equal to or shine. And here, the same thing. Only YOLO will weaken Nicaea. And then title will give labs function to give the time. And X is equal to X and Y is equal to weigh a lateral view function and minimum. And when we run, we get this sign-in or shamed go when the same plot. So I've already explained we create two separate dataset for Sine and Cosine curve using data.frame X comma Y, X comma Hawaii, hawaii sign and X comma Y. Y is equal to X comma Y. You can do why costs, okay? And then each is Yom line layer. Here we're basically creating two layers of layers of separate lines. Okay? So each geom line, geom underscore line function will create a layer correspond to separate line, Okay, One Park sign and one for cosine. We use the color aesthetics to distinguish them. And then labs function we'll use we will be used to set the title and axis of excess levels, okay? And team underscore minimal, tended overall appearance of the plot to a minimality. Next thing, what we'll do, we'll try to customize the line properties. So now we can customize the various property of lines such as color, line, type, and size. So here the sample data will be the same sequence from San Values and recreate X where loop. And then we'll create why underscore sign and Y underscore cost by using sine and cosine functions here and passing the extra illusory exit the independent where they will anyway, underscore sign and wireless go. Crocheting will be the dependent variable because these values are depending on the X. And here we'll create line plot with customizations. You add Plot plus geom line here, DataFrames, same thing which we have used earlier. And here we'll use colors. It's going to sign a line type is called a solid and size here and give 1.5. So line type solid and say so yet I'm using size to determine the thickness of the line. Okay? So I'm leaving 1.5 here. And same thing when a white wash. And then everything else will be the same. Lab Title. X is equal to XY is equal to I color. So let's give you this, run this, and now we're getting them. This is dotted and this is that thickness. Okay. Together we have you're done line type as Dashboard, it is coming as a dashed line. And here we have your solid, so it is coming as a solid support. They'll make it 111.5. And if I run this again. So let's see what you see here. Now. The dashed line is coming. This way. We can increase or decrease the thickness of the line. Okay, so I hope so, I hope this part is clear how to create a line Plots into GG plot to the next lecture. 102. Data Visualization with ggplot2: Hello and welcome back. In the previous lecture, we have seen how to create line Plots. And we have also seen how we can plot multiple line Plots and how we can customize the line Plots. In this lecture, we are going to do some more data, GG plot to add programming. So what this, I'm going to heat our Dataset, which will have name, a few names and the gender, male, female, and then their age and the test score. The score will be repeat 100 and of course, okay, so and based on that, we try to visualize this data, which is having name, age, gender, and Test score. And we'll try to visualize this dataset with the help of radius plots or live in GG plot to, and we'll try to do customizations on that, like really faceted plots. And we'll try to use multiple things on this simple Datasets. So I'm using the simple dataset so that it will be each to understand. Before that, let me clear this. So let's start the data Visualization with ggplot2. And with this simple dataset, we will try to understand better what we have learned till now in Guinea ggplot2 hogging. So this will obviously will load the necessary libraries, that is GG plot two. So library and we'll pass the bucket name library named GG plot. Next thing is we'll dendrite the sample dataset that we are usually continued for this exercise. Okay, so here I'll use set seed One-two-three. And this is for the reproducibility. Reproducibility of seed. Because see, we are going to use sample by using the sample function. And it will be 18 to 25-year-old. It will move 18-25. So that will be the random generator. So what I want to do whenever I want to use this sample Data, if I use set.seed, it will, the data will not be regenerated. It will be, once it is generated for this exercise, it will really meaning same. So it will reproduce the same setup random values. Okay, So far. So set.seed will reproduce the same values even though we're reaching the sample function, okay? Otherwise, if you not use set.seed each time, it will create a random new values, new values. Okay? So now first thing we'll use set.seed and then we'll create our variables students. And here you use data.frame. And here frame will create the name variable and there will store this vector which will contain the names Alice, Bob, Charlie, David, Eva, rank, Frank, grace, Hannah, Yan, and Jack. Okay. And then we'll create another factor which will store the gender and gender will use the vector yes, male and female. Okay, And Part ten by ten, okay? Ten to ten and replace true. Then for Angel, use the sample 18 to 25. And how many I want, I want end values to which ended up reading to quantify and Test score up 210 Discord, I want to generate by using the sample function. This we will get the data sample datasets. So let me run this. So now we have the sample dataset created. Okay? So see here now, if we print the student, we can see your name, gender is, and Tesco. So he had the name, we have given names. And the gender has been generated randomly by using the sample function. Okay? Male, female, male payments on hands-on, Okay? Male, female, male, female. So random liters generator, the age 18-25 husbands randomly assigned to these names and Test scores has also been than namely generated. This way we can create Sample dataset. Next thing, what I, what I want to do, I want to create a scatter plot between this age and Tesco. I want to see how the test score is wearing based on what each is I want to plot or scatter plot which will source the score for that corresponding age. So for that, what I'm confused scatter plot, age versus test score. So for this alludes, I'll create a variable scatter plot. And here I'll use function and pass the student dataset here. Okay, students. And then I'll aesthetical huge x-axis and X is equal to H and y-axis. Score. And neodymium point, I'll use the geom point function to plot the points on the X and Y-axis. And your lab, X is equal to is and why it's called r2. Score. And title of the plot will be the scatter plot, age versus discord. And then I'll print this scatter plot by using the rand function to display down here. So let me just keep it here. And okay. So see here, now we have a scatter plot here, which is on page 18 to 25, and the y-axis and destroying the score, test score. So for 18, it is somewhere around 85. So for 18, if you look good here. What identity at four? So see here thymidine or at 400 and plot it. This way. We can lotta scatter plot by using the ggplot2. So here, on aesthetic layer, we have given X and Y axis, we have passed the test score. And we use the geom point function to plot these points on the plot and your lab function, we have X is equal to H. And why it's called to test score. And tighter, scatter plot versus scatter plot, age versus the scarf. So this way we can get it up plot or scatter plot. Okay, next thing is what I want to do. I want to same explanation I have written here and let data file, you can go through it. And if you want, I'll explain again. So here we'll start by loading the GG plot pull library. Next, we'll create the sample dataset by using data.frame that we have created here, data.frame and name, age, gender, and Test scores generated. And then we'll use the GG plot function to initiate lot and provide the data. Students and aesthetics. Aesthetic function X is equal to graduate from Tesco, which maps the variable to the x-axis. So x-axis is will be mapped on the x-axis and the score variable will be mapped to the y-axis. The geom point function is used to add points to the plot. These points will be added to the plot by using geom point, geom underscore point function for each data point for creating a scatter plot. And finally, we customize the labels and title of the plot using labs function. So here Lab function will be used to customize the labels and the title of the Plot. And then we'll use the print function. Henry, pause the scatter plot, the name which we have assigned to this plot, and we'll pass it to the print and the scatter plot will be printed on our un-solo. This way we can create each versus the score sample dataset that we have generated. Next thing is, I want to add the faceted faceting to this. Scatterplot is what's the test score for each gender separately. Okay, so this will be the first block. So for this I'll create a variable first step plot handle, you add Plot. And he had a user. Same DataFrame. Student dataset. Aesthetical, you'd X is equal to X and Y equal to Tesco. And he'll, since I want to this a lot based on the Each versa test score based on the for each gender separately. So here color is huge. Color is going to gender, so that the gender will be specified by the color. And then geom point. I used to plot the points and labs function we used to give. Next is test score and title of the Lord. And here, and use the facet underscore wrap function. And here I'll wrap this one. X1 by Mao was in the tender. You still use gender four if I subtract, okay, So let me run this and see the output. Okay? So now I'm painting the paint and see here now we have the faceted plot, age versus test score by gender. So x-axis is, again it, and y-axis is deaths code. But here, our Lord scatter plot has been divided into two parts, female and male. Female or female. Female gender has been plotted here. And the for male gender it has been plotted here. So it is a clear cut. We can see that digital few minutes or female desk called Values and on me. Okay. This is the way we can use the fester plot to use for gender are classified data. Okay? And we want to know what about the explanation? I'll repeat it again. So let me just outline it here. What we have done, like we have that before. We have used the GG plot function to save the plot and provide the data, students data that we have created using the sample function. And then the aesthetics. The aesthetics fonts and X is equal to SY squared called score and Gartler. We have your agenda to map the age variable called the agenda to map the age variable to the x-axis. Tesco valuable to the y-axis, and save that color up the wind speed on gender. So color of the points will be the based on the agenda. And there are two genders. Naturally the two color husband use red and blue. Okay? So this will result, the result in different colors for male and female students. So Cl for male and female on the different colors has been used. And that is because we have huge color is called to gender, okay? The geom point from Sandy years to create the scatter point for each data point that we have already seen. And we customize the levels and title and the plot. You didn't last function that we already know. Then the facet underscore lab. And here we'll pass the gender dysfunction will create a separate panels, facets on these two panels, male and female. You can see here these are called visits. Facet, underscore wrapper will create two separate panels are first set for each gender. So here we have passed gender. So based on the gender it escalated two facets are two panels for representing the points. Okay? This means that plot will be divided into two subplots. Okay? If there are three genders like male, female, and neutral, then it will be divided into three parts, okay. Subplots. One firm Mileage to runtime, another female students. And the print function will use to display the display scatter plot on the screen. Okay, So this is the explanation for creating a scatter plot and facilitate blocked. Next thing, what we'll do, we'll create a bar Plot. And here we'll use gender. What we want to plot, we want to plot gender versus Test code using mean value at the summary. So here we are going to create a bar Plot. And bar Plot will be created based on these two variable, gender versus Tesco. We're going to plot. And what we'll use here, you will use the mean value, the summary. So let's see here alkylated wherever bar and is called plot. And I'll use the GG plot function as Datasets. Students aesthetic X is equal to gender wise square root test score. And I'll use the plus operator. And here I'll use the geom underscore bar to create the bar chart or bar Plot. And here, I'll give you static is equal to summary. And the function will be the main function here and fill each equal to stable. So it will rebuild with the steel blue. And the function will be made on the MDA, made on the mean or average value of Somebody and stack will be the summary. So summary, I'm going to summarize it by the mean values and laugh functional, huge, ecological, x-axis, gender and Y, X, X is really with a mean test score on average test score. And the title will with a bar Plot, gender was mean test score and print bar Plot. So let me run this and see the result you see. And now we have bar Plot for male and female. And here on the y-axis we're showing the average test score, one male and female. So far female average test score is this. For me? I'm just test score is this. This way we can create the bar Plot. So here, if you see the explanation weekly, so we tell you. So we will use the GG plot function to initiate the plot. Aesthetics. X is equal to gender or y-squared test score map the gender variable to the x-axis and test score variable to the y-axis. And Escobar function is to create a bar Plot. And we sit. Stat is called two summary and phon is equal to mean two main function will be calculated and display. The score for each dome. Average of the they score will be displayed and the field which will restrict glue, will be the fur color at the Barstow. Blue, the color of the bars. And we've customized the levels by using the last function that we already know. And then we'll do the print function to print the plot on the screen. Next thing is, we'll create a boxplot. Gender versus discourse, same thing. So here GG plot, function, stranded asset, aesthetic X is equal to gender away on y-axis test score and fill freely by gender. And we'll use the geom underscore boxplot function to create the boxplot and the latch function will get used to give the title of the bar and the x-axis and y-axis naming and will simply print it. So let's run this. Now we have a boxplot, gender versus test score. The x-axis is gender and y-axis test score. And this is the boxplot for male and female. So you can see, okay. So based on this, you can do the further analysis, okay? Whatever you do with them, boxplot, first quartile and all those things you can relax plot. The same explanation Gorgias, DG plot for initiating the plot aesthetics for mapping the winds and young boxplot function used to create the boxplot. And we customize by using the last function print. Okay? And here you can see the two boxes, one for each gender, male and female. The boxes Entropy in the inter-quartile range that I already told you. The line inside the box representing the mean, median value and the whiskers external to the minimum and maximum values within one FY15 quartiles, inter-quartile range or IQR. And points out are the whiskers are considered as of last point outside this risk because when we called out less, so those are the analysis you can do to find outliers and all if they are, these dataset is not enough for me. There. This dataset is not enough for analyzing dog class. And all. This is the sample dataset that we have created through just create the different plots. Okay, So next thing we'll do, we'll create a histogram. And here we'll use the age distribution of students. Here. I'll use the histogram. I'll create a histogram where they will Year GG plot function. I'll use the same dataset and yet aesthetic exit equal to H plus I'll use the geom histogram geom underscore Hostogram function. And here, binwidth, I'll give one. I'll fill with this Tableu, the color to white, and I'll use the left function to give the name of the Plot and X and Y-axis, and then simply print the histogram. Here, this is the histogram. We have created them being say, okay, so this is the histogram for the age distribution of the students. Okay. I'll quickly again the explanation. So here again we have here the GG plot function to save the plot and then the Dashboard and strain data to the GG plot to. And we'll use aesthetics function and the age because we are going to undo the age distribution of students using histogram. So X is equal to H and map the history valuable to the x-axis. Geom underscore histogram function values to create the histogram when each bar represents the frequency, count up the students specific age group. So here, what I'm trying to say, that each bar represent the frequency of fish to rent in a specific age bar. Okay? So at that frequency of the strain for 20 is frequency of this going to rent how many strands are there that you can see? You see here problems for the age. At 24, there are no students, so it did not showing any barrier. And white using full-color argument, a bar that is white, the gap between these white, so this collateral weightage cannot see it outlined, won't be there. Then customize using the left function. And the printer using the print function is an x-axis. And the president, the different age groups and by accident brands that frequency not but I fished rent in each group. So what 181 is to run for 20, there are students, so I'm critique or to the Dataset CF1 18, only one is 20 there. And for 2,012.3 strands, that's fine. If you look at here, for 23 students, what 25? That is Nostradamus. So when we, when we, you look at the data, Part D for that is totally fine with them after 24, then Nostradamus. Okay? So this way we can create various net average license using GG plot 103. Add color aesthetics to the plots: Hello and welcome back. In the previous lecture, we have seen how we can get various Plots using ggplot2 in our hand rehab creating this simple dataset which contains name, gender, and discord. And with that, we have seen how we can plot does scatter plot, which will represent each word says Tesco. And then we have seen how we can use geom point fonts and how we can use the labs function. And we have seen all the explanation. We have seen how we can create a faceted plot which will. So the logins are two parts like age and the test score. Based on gender. It will sort on the gender. It will also in the two parts, male and female. Okay? So all those things we have seen like this, it wouldn't be how fast the in between male and female. And so this fascinating plot lot, scatter plot also, we have seen how we can do that. Then we have seen how to put up bar Plot. And between age and gender and discord. Discord that also we have seen. And then we have seen how we can put her boss, mosque, lot, boxplot and between gender and Test code and how we can, we have also seen the explanation of how we can analyze the outliers, but it on the inter-quartile range and all those things. So not all distinct then we have seen the histogram. Many more things you can based on your requirement. Next thing, what I'm going to tell you about adding aesthetics to the plots. So let me assume you've watched, we have created this plot, scatter plot here. So when we see the scatter plot, this is the scatter plot like this. Okay? Now, what I want to do, I want to add color later on the agenda to this plot. So the same thing will U-Dub library and add ggplot2. And here are why I wanted to modify this aesthetics of this plot. Because by looking at these, I don't know which this point is male or female. Made on it. We made on each agent desk, could we have plotted, but we don't know, this is me. This score is male or female. Okay. I want to put the points in a color so that we're looking at the plot. I can know that, okay, this being belong to female and the blue is for me. So that kind of thing we can do with the aesthetic change by adding the color. So what we're going to do, we're going to add color aesthetics to the plots to differentiate between male and female students. Okay, so here the same setup sample Data will create. So first we need to run the library and then it took me at this data set that we have already created. And then we will create a vector here that will contain the male, will assign a blue color for memory will assign the pink color. And this rector will assign to the color variable. Okay, so now the same week, what we have created here, scatter plot, the same way will be dark. Scatter plot here will give a variable named scatter plot to will you the GG plot function. We'll pass the dataset as a students. This one we have created, then we'll use aesthetic function. Yet X is equal to H and Y is equal to test score will be the same. The only thing is we'll assign another parameter here, color each call to gender. So we are going to assign the color based on the agenda. And this color will come from this color vector here we have created. And then we'll use the geom underscore point to plot the point on the Plot. And then we'll use the scale underscore color in the second. So this function will be used to color the points made on the agenda So color is going to gender. And here we are passing though, this fonts and color scale underscore color underscore manual Values is equal to color. So this value will come from this vector. Male and female. Male will be blue and green. When we're looping. From here, it will pick the color and it will be plotted and lab function. Now we already know. So let me just done this thing and see here. Now we have the scatter plot we have created. Now let's simply print to the scatter plot here. Now the scatter plot has pink and blue. The points which were earlier one Leap black. It is being shown as a blue and pink. Pink winds belonged to the female and the blue for the males. So this way we can customize the scatter plot by adding color aesthetics. So let me explain this again. We have created this sample dataset. People having this table discount dataset is Eric name, gender, age, and Test Score. And here we want to add a color aesthetics. So we have created our color, we vector, where we have assigned a blue to the male and female color. This color we tap bars to the scale underscore, underscore manual function. This function will data values from this color vector. And it will actually, it will assign based on the agenda for if it will assign blue color and if it is free milliequivalents NDA, I think, okay, then we have plotted that scatter plot. Similar thing we can do with the bar Plot and boxplot and histogram. So let's do that also. So here I'm creating a valuable Bar Button, does scatter plot, GG plot function, I'm using Dataset students aesthetic X is equal to gender. Why it's called score. And I'll fill with gender. And then I'll use geom underscore bar, Plot hit Stat Summary and, and plot with the summary with the mean value. And then I'll use the scale, scale underscore, underscore manual. So here we have scale underscore color, and that's called manual for the bar Plot. Skill, underscore, underscore manual. And we'll pass the value added color. And then lab Swanson will be hedges will take. So let me run this. And now I'll print the bar Plot. See here now, the female is plotted. Boxplot, PMM in boxes bring and mailboxes. Mail blue. This way we can color aesthetics. Bach, bach bar Plot, sorry. Then boxplot. I will use the same data set aesthetic X is equal to gender, why it's called desk or female gender. And then geom bar, geom underscore boxplot to plot the points and then scale underscore, underscore manual and your values when we color and the left, X gender. Why does God love the chart is boxplot gender resist test scores. And let's run this. And then random boxplot. The boxplot. The boxplot is also having colors pink or female and blue for the meal. Okay. Next thing is we'll do for the histogram. So same GG plot function students Dataset aesthetic X is equal to H field. It's called the agenda and then geom underscore histogram. And he had will give the bandwidth one man with each call to one and color each color to white. And then scale. Scale underscore field and its command. We'll pass the Values As the colors. So this white will be the border color, and this will be done. Histogram colors based on the gender. Okay? And then last function as Israel histogram and distribution of students. This, and then print the histogram. The histogram is I haven't. Pink and blue color and dark border color is white. The adjoining lines are white. If you want to make it make it green. Let's run this program. The Nauta, more or lesser degree. Okay? So this color is for the borderline and this value to the colors we are picking from this column vector that we have created being female and grow formyl, you can give whatever color you want to give vent on your choice. Okay? So this way we can add the aesthetic to the plots in ggplot2. So I hope I have explained clearly. And you can walk on this. You can start playing around the radius function data. You can dig another dataset and you can start exploiting the things and start expanding, exploding the things and start experimenting things. The different color, different aesthetics. Okay, so see you inside the next lecture. 104. Fine tuning plot aesthetics: Hello and welcome back. So in the previous lecture, we have seen how we can add color aesthetics. We have seen how we can add colors to the plots, scatter plot, bar, Plot, boxplot and storing them. Now, what I try to fine tune the plots using and scale functions. So fine tuning of what is fine too. Findings really try to make it little more appealing, all those things okay, to fine tune plot, aesthetics and ggplot2, we can use aesthetic function, aes function to map variables to aesthetics and scale Function. Scale underscore started with, there are several functions under it. So we can use the skill functions to customize the appearance of radius plot elements. Simplicity, they can pull up aesthetics, can scale functions to fine tuning plot aesthetics. So last thing we'll load the library. Then we'll create the same sample dataset that we have created in the previous lectures. And then we'll do the fine tuning of a scatter plot. So same thing we'll do, age versus test score. The same thing. Here. We'll fine tune the lot using aesthetic and skill functions. The valuable scatter plot, the plot function will use will pass the dataset. Then aesthetic function when X is equal to H of Y is equal to test score and color made on the gel gender. And here we'll use another parameter called Save. And save. What Save we can associate will decide based on gender. And here, we'll use geom point function to plot the points. But here the point sides really specify. So earlier we have not specified any point size here we'll specify three. So geom point sides equal to three. This parameter will pass. And then we'll use the scale underscore, underscore manual. And he had values for the color will give C. Rector here blue and pink. And then for CEP also will use underscores. Underscored manuel here will give the values. It has 16, 17. So this will create safe, various tips you we can create, and then we'll use the lab function. And let me run this simply. When I run this, the scatter plot will be created. Let's bring the scatter plot. So now you can see here, now scatter plot had will create it, but see the theme of the points for male and females are indifferent. See the points are a little bigger because we haven't specified the size equal to three. If I make it tartan, what will happen? Let's see. The size will increase and it moves, print it again. Now the size of the points are bigger because we have a specified size equal to 13. And the color we have given here, blue and pink. So blue and pink colors, bluish Part, female and male. We have given pink. So here it is. Appoggiatura. We can now corrected by simply. Okay. So this is not accurate or actually so that you can calculate letter C. Now the points are bigger. So based on this side is equal to 30 and color is coming from here and see if it's coming from this heap. Whatever values we're passing here. If I make it 117, let's see. You can just play around with the numbers and see now this even one. So it is just a circle. And if I put a lyric was annealing process team support, I'll put 88. And let's see what secret it's going now. Notice coming like this. So based on these things, you can see for it, if I put 18, what would it be the same data also, you can see you let me copy this. Split. This please see, and now it is coming into this laminate tape and this is triangle. If I make it. This also at what will come in the same, say, if I make it 28, 28 is nothing like this. We can make it right. Okay. So based on the numbers on the safe will change. So this is the way we can fine tune the plots. Next thing we'll fine tune the bar Plot. And yet also did gender versa test score usually mean value of the summary. We'll plot that we have already plot here. The same dataset. Student aesthetic, gender, why test score and fill by gender. And then geom underscore Bar function. Geom underscore Bar function will Stat Summary and phon on sensitive to mean and scale underscore. Underscore manual will see blue and pink. And this is simple thing, will just be this and paste it here. Similarly, we can do the same thing with the boxplot. This for boxplot. Okay? So CEO, now, one dot line is coming in here so that we have non-binary gender fluid bigender. And here. Similarly, we can do this histogram. This is the way we can fine tune the plots using aesthetic and scale functions. Students like the next lecture 105. Modifying themes, labels, titles, and axes using theme Function: Hello and welcome back. In this lecture we're going to modifying themes, labels, titles, and axes using theme Function to modify themes, labels, titles, and axes. In GG plot two, we can use the theme Function along with various theme elements to customize the appearance of our plots. So let's start with an example and see how we can use the theme Function to modify themes, labels, titles, and axes. So very fast thing is we're going to load the library as is relatively plot to. The next thing is we're going to create this sample dataset that we have done already. So let me run this for now. Now, next thing is modifying themes, labels, titles, huge thing, Banza. So for us to recreate the barplot. So first we'll create the scatter plot between age and extra code that we have already done. But here we'll try to modify the theme levels and titles and axes using theme Function. So fasting is good if the variables scatter plot as usual, then we'll use the GG plot function. We'll pass the dataset, then we use the aesthetic function. And here, X equal to H, Y will be the test score Color mentioned on the agenda and say build on the agenda. So we have done already. Next thing is geom underscore point functions Sij equal to three. This will plot the points on the lot. Size will be sized, sides of the dot will be three. Then here we will use the functions scaled underscore color and DashCon manual. And here will give the values for colors blue and pink and safe for sample gives us this Vectors 16, 17, we have already seen in the previous slide, previous lecture how the shapes of the dots or points will change based on the values we pass onto here. Okay? Then the next thing is lab Swanson will use to give the title of the plot and the name for the x-axis and y-axis. And then we use the theme underscore minimal function. And then this is the main thing that we're going to learn in this lecture. That is theme Function. So inside the theme Function, what we will lose a lot, dot title, we will give. And here element underscore X, this will be, this function will be used to give the sides of the title. Okay? Take sides of the title. Like scatter plot with age was a Tesco. This title will come from the plots that text size we can change from here using the element underscore tax and size parameter we'll do here. And then we'll give the face is equal to bold here we can change the style of the font. Font style we can change by you didn't face, did you call two volt Italic whatever you want and act just will give 0.5 and exists dot title for x-axis and y-axis title, we will use the same function. Element underscore text sides will give whatever you want to hear I'm giving 12. And then face bowl. And then for access texts will give you the element underscored sex function and size will give ten. And then legend dot title, we'll use element. And that's called blank function. And loginData texts. We'll use the element under certain texts and sides will give ten and legend position will give on the bottom. So we'll give the bottom here. So let's run this scatterplot. Before that, let me copy this print scatter, scatter plot. And I'll put it, just see us on that scatter plot planted and the same time. So let me run this. See you now, we are getting the scatter plot here. And see this is our tech plot title that we have given in a bowl here. Like if I change this to 14 to 24, this will, the bigger, the texts will be bigger in size. So let me run this. Now it is omega, right? The same way for the axis title if I gave it to 22. If I run this. See now test score NAs or in bigger. Okay? So the way we can modify the titles, levels, and axes. Okay, So let me make it and 14, looking matter This each course scatter plot. Okay, next thing we'll do the same thing for the bar Plot. So for bar Plot also will use the same dataset. And we'll create a bar Plot variable here and then DD plot function dataset. X is equal to gender. Why it's called desk or gender. The same thing which we have done earlier when lifting is we are going to use this theme Function extra year plot dot title. Same thing we are using for the bar Plot as well. So I'm not going to take matchup the time explaining the same thing again and again. I'll just copy this. And I'll just best to tear an adult run this. Okay? Okay. So see here this is the bad plot. Similarly, we'll do the boxplot as well. So these are the things we have done already. We only extra thing we're doing here is theme. The same thing, Elements and discourse. I same thing. We are doing further boxplot as well. And I'll just go to this and paste it here. And run this for boxplot. To see the boxplot. The boxplot, everything has been changed accordingly. Okay, next thing is really doing what? Histogram is, Distribution obstruents here also, we had this theme Function here. Modifying the deck size of the size of the Plot take. So let me run this. So this is how we can modify that themes, labels, titles, and axes. Lot using in ggplot2. In each plot theme Function is used to modify the various theme Elements such as plot dot title, axes dot title existed or text legend dot title legend dot text, and ligand dot position. The theme or the element underscore text function is used to set the site, font-face and justification to the next element. We use a theme underscore minimal function to start with the minimalistic theme, where all the plots and then further customize the appearance with the theme Function. These examples demonstrate how to modify themes, labels, titles, and axes with the theme Function in ggplot2. Now, you can explore additional theme Elements and further customize the appearance of your plots to match your specific visualize and needs to every project will have their own Visualization need to be done that you can use this functions and you can explore further 106. Project 6 : Hello and welcome back. In this, we are going to do a simple project. And with that will walk on a real world dataset to create insight regionalization. So we're working with real-world dataset to create insightful licenses and important aspect of Data Analysis, Data Science, storytelling our projects. Let's go through an example using real-world dataset to create insightful visualizations using ggplot2. In this example, we will use the famous Iris dataset, which is a readily available with the R package by default, the IDT does it contain the information about different species of iris flower, including their sepal length, sepal width, petal length, and petal width, and the species of the flower. Okay, So this is the simple project that relu and try to create some visualizations, visualizations. So first thing, we lose that library, ggplot2. Then we'll load the iris dataset by writing data function, Data function passing down dataset name. So these two are the essential step to load the necessity library and load the Irish dataset and explore the structure of the dataset. We'll use the STR function and pass the dataset name. So let's see the structure of the dataset. So you can see here, there are one per employee objects or five variables. Five variables are sepal length, sepal width, petal length, petal width, and a species. Okay? And then Values are given here. Okay, so next thing is we'll see the summary of the Dataset, which will give us the statistics. The five columns, sepal length, sepal width, the length petal width, and a species. And their minimum values, crushed quartile, median, mean Values. Third, quartile and max values for each of the variable. Okay? So this way we can see analyze, okay, next thing is we reload. This is the statistical thing that we get by getting the summary of the dataset. Now, what I want to do, I want to create a scatterplot for this dataset. So for that, what I'll do, I'll create a scatter underscored plot variable here. And I'll use the GG plot function to create the plot, and I'll assign to this variable and then print it. So GG plot function, I'll pass the dataset, Iris dataset. Similarly, that like in the earlier lectures, we have used aesthetics function here also leaves the aesthetics function here also leaves the aesthetics Function. And he had XXS for the sepal length and y-axis output. Simple dot width and Color. Put built on the species and geom underscore point function I'll use to plot the points on the Plot and large functional, huge to keep the title for the x-axis, y-axis, and the title of the Plot. And then here I'll use the team underscored minimal function. And then I'll print those scatter plots. So let's run this and see how the blood coming up scatter plot. So let's just make it bigger. This is the scatter plot that we are getting after running this. So see here, sepal length on the x-axis, simple width on the y-axis, and Color based on the species. So there are two dosa, versicolor and virginica. So see, all the species are plotted here based on the sepal length and sepal width. So this is the same past scatter plot that we have created. Next thing is the Iris dataset has for numeric variables, sepal length, sepal width, petal length, petal width, and one categorical variable that is species. So this species is the categorical variable. Now let's create some insightful visualize suddenly you blocked off. So the first one is scatter plot, our data we have already created. So let me run this again. You get the same. Now we'll create a boxplot. So boxplot, same thing, boxplot GG plot to will you does it, then aesthetic function. And here X, X is, I'll put these pieces and y-axis I'll put the hotel put, I put the petal length. So species and their petal length and we'll fill with dark species. And then geom underscore boxplot. I'll use to draw the boxplot and then Laugh function called title and all. And then I'll print. So let me run this. See here this is the boxplot, petal length by species. So this each for setosa, this is for this boxplot is for versicolor, this is for watching Similarly, I want to create a histogram. And histogram I'll create based on the petal width distribution across species. So for this histogram, gram underscore plot variable DG, ggplot2 function does, it'll pass iris that we are using aesthetic X petal width. And I'll fill with this piece with their species and then histogram, histogram functional use and binwidth, I'll give 0.1 color, I'll give white colored. This is the suppression color between the bins. And I'll give 0.7 and labs functional use to give the titles and all. And then theme minimal I'm using here. So let's run this and see, yeah, this is the petal width are the x-axis and y-axis. We have the frequency. And this is the first set of size pieces. Greenish for the versicolor and virginica. Virginica. Blue for the virginica. Okay. So this way we can create this histogram. Next one is, I'll create a faceted scatter plot. And there I want to show the petal length versus petal width for each species. So for this, I'm using the x-axis, I'm using petal length, y-axis, I'm using the petal width and Color I'm giving better than the speed on the species and geom point size. I'm given three and all other things that same year, friendship to underscore wrapper function I'll give build on this species. Okay, So this preset underscore wrapper function will give you the professor did scatter plot. Let's run this and see the output. Now, we are seeing the facet scatter plot. Here, the set of size. This is possibly better land on the x-axis and petal width on the y-axis. This is for the setosa, this green for the versicolor, and blue for the virginica. So this is the faceted scatter plot. And here also you can see the blockage for setosa, versicolor and virginica. This is a clear categorization. Okay? So next thing, what we Under project is by utilizing that real-world data sets like Iris dataset, we can gain insight into the relationships and distribution of variables, identify the patterns, and make insightful interpretations. Okay, Now you are free to explore more Plots, type, and customize aesthetics that we have done earlier. Like you can modify the theme labels, titles, all those things that we have learned in the previous lecture that you apply on this and create your own project. And then experiment for the different datasets. Apart from this, to practice and develop your own DWT lesson skills. So this is the simple project based on the iris dataset to create a scatter plot histogram fester scatterplot, boxplot, and all those things. I hope you got the some idea about how you can walk on the real world dataset. You can walk on the real world dataset. Thank you 107. Handling date and time data in ggplot2: Hello and welcome back. In this lecture, we're going to learn about handling date and time data in ggplot2. So handling date and time when T2 plot to invoke, inserting that data is correctly recognized as date and time object store that GG plot can properly format the axis labels and create appropriate skills. What the mean? Mean that if you pass the data, date and time data in a correct format, then you plot two, will take care of vegetables, things by plotting and they didn't collect taxes. So April teach clinically recognized date and time objects. Gg plot can properly format the axis labels and create appropriate skills. In our date and time can be represented using various classes such as logics to city and audits, and LD functions. Okay, So we'll see project City and projects LTE later. For now, it is enough to know that data can be represented using various classes such as logic city and projects as the, okay, so let's go to a simple example using dataset with a date and time data to demonstrate how to handle date and time in ggplot2. So the first thing we need ggplot2 library and the library, how far these examples will load these two library. And then what I want to do, I want to create a sample Data Set which and time value. So here that'll create or Dataset which will have the date and the values corresponding to each day. So for this using set.seed 123, so that I'll get the same setup sample data each time. When I run the program. Then for Data, i'll, I'll create a valuable dates and I'll four days, i'll, I'll lose a sequence function to create the decks for me. So insights sequence functionality from add dark for six city, add dark four DX DT. And here I'll give date as 23 gen, first, first January 2023. So this is the origin of the date. So from this decrease and we'll start, okay, here, I'm using by one day. So one day will be the difference between each time date and lynda dot dot will be 30. 30. I wanted to create 30 days. In that 34, 30 days I want to create, okay. I'm used sequence Ronson and for Values using odd pnorm function, pnorm function. Here I'll leave that number of values I want to create. So I want to get 30 Values. What are these 30 days? And then I want to use the mean equal to 50 and standard deviation either ten, okay? And four days I've union sequence function and inside sequence function I'm using from is equal to Ag dot Cp. I'm giving the beginning date. And by one, by each one day I live. In titles will be created from 30 days, will be created from first-gen, goes on printed three. And then I'm creating a DataFrame, data.frame. Each call to date or date. I'm using these tests Values and for Values, I'm using this value. Okay? So let's run this and see what is happening. Now have run, let's bring to the dataframe df8, have rehab created. So see here. Now it is I'm two columns, date and Values. Data is starting from first-gen 2023 and it is going first gen to time per day three. And for each day, we have different, different values associated with that. So now we have simple data, sample dataset, date, and corresponding data. What was the value? It could be anything like cost of the particular item or something. So we have date and review column. Now we can try to plot them using ggplot2. So in this example, we created a dataset with two columns, data representing it in time data and value representing the some numerical values. Okay, so first thing, what do I want to create a simple line plot with date on x-axis for these alkylated valuable line underscore Plot and allude GG plot to, to create the line plot. So ggplot2 and dataset, I'll pass dF that we have created. Just know anesthetic X exists. I wanted to put the date and y-axis I want to put Values, so X is equal to date and why do value? And then I'll use the geom underscore line Human, that's called line function to plot these points on the graph. And then I last function to give the names like for X axis state y-axis value and datalab though, plot is line Plots value over time, and theme, minimal. So T-minus four millimoles, and then I'll print the length plot. So let's run this and see what is the output coming here. This is the line plot command. Line Plots value over time. So for January 1, day and Values As we've plotted here, for example, there are nine density like this, okay? So value over time with this, you can simply visualize it is producing Part two of plot that data time. Ggplot2 is the simple. On x-axis will Data y-axis Values and rest of the things when we're taking getaway ggplot2 in this broad GG plot to recognize that the date column contents, date time data, and automatically farmers access these levels accordingly. Okay. So labels here. Well creating this, we have not given any, like the format is 01012323 here, we're not given like this, but GG plot has recognized it as a date, time, and automatically given the labels that Jan to Gemini in Genesis team like that. Okay, So this has been done by automatically, by DG plot to know next thing we'll do, we'll try to plot the area plot with date and time on x-axis. So area plot, GG plot, the x-axis Alport date y-axis over ALU and filled with the value. By the geom underscore area function I'll use to create an area plot and labs exit, exit state y-axis value and title area Plot value over time theme minimal, and print, print the area plots. So let's run this and see. Now you can see here Arial Black value over time has come and here also dumb and here also the things that had been taken care. And this is the area Plot created by did you ggplot2. On the x-axis is just showing the dates January to January 9th, necessity men that are grantee and then 30. And for each date and Values over time with desperate and so on by the area covered. In this plot, we have your underscored area to create an area plot. And GG plot automatically polymers the x-axis levels with the date them information. So you can see here. Now next thing is we'll create a bar Plot, date on the x-axis. So aggregate data by my day, create a bar plot. So here what I want to do, I want to read a bar plot with the swing, the data for each day. For ductile do LLC create a DataFrame and our daily underscore DF and he has LDF, the pipe operator here. And I grouped by date, add dot date, and they'll pass the date and summarize by average value. That will be the mean of the value. So this way we will get the daily underscore DF, okay? So if I so see here this is the daily DF. Date and average value. Describe them like this, okay. That drone two more rows. Okay? So now I'll get it up bar Plot. So bar Plot, anti-D ggplot2 bar. This time I love knot past the DF and pass that into SQL DB that we have created just now. And for aesthetic functional pass, X is equal to date and Y is equal to average value. So date will come from this date and average value that will come from fear. And see your date. Type is date and average value W. Okay? So and then I'll use the geom underscore column function to create bar Plot. And on the latch function I'll give X is equal to date, Y is equal to average value and tighter bar Plot average value Part D and T Mileage minimal and theme exists dot X dot X element underscore tax angle 45 at just one. Rotate access to it levels for better visibility. Okay. So I'm rotating excesses levels by 45-degree for better visibility over D. Okay, and then print the bar Plot. So let me run this. So see here, this is how we have tilted. Sees Ajahn to Jenae has been tilted by 45-degree. So we haven't gotten by using the element underscore tax function and passing the angle is equal to 45. If I put angle each call to our 75. And if I try to run this, so this will be on the 75 regularly. If I for support Albert, 90, 30 degree, it will be tilted by 30 degree C. And 45-degree. That is looking better. Okay? Now we have the average value per day. So even though it is just plotting January 1, gender told me threes and fours like that for 30 days, average value Part D, we can see what we have done in this bar Plot. We aggregated the day by day and use geom underscore column to create a bar Plot. We also rotated the x-axis level by for better visibility that we have done here with the angle 45 digit jumbled. Demonstrate how to handle date-time data in ggplot2 by inserting the DateTime retires in appropriate format GG plot to automatically handle the excess leveling and scaling, making it easier to create insightful, virtually equal regulations over time. So we're only thing is we need to pass the correct DateTime value to ggplot2 under stop the thing GG plot to understand how to handle date and time in ggplot2 108. POSIXct and POSIXlt functions with example: Hello and welcome back. In this lecture we're going to learn about both city and projects LT functions, that is very common to see now, programming for handling the date and time data. So you not see the endpoint LTE classes Jewish to the present, date and time data. Both classes are part of our package. You wish for handling date and time Values. But they have differences in their internal representation and behavior. Obviously, if there are two objects classes, then there must be a differences in their behavior. So on. Let's understand how for the city and projects LTE works. So first we'll understand how poor city walks project City stands for forensics, calendar time. So CT calendar time, C4 calendar T for time projects is 4.6, 0.6 and represents date and time as the number of seconds since the beginning of the unique. That is January 1, 1,970. So project calendar time to start from January one-ninth and Under sovereignty. But you don't know unix system. Okay? So, and from that time, it will count the number of seconds since the beginning of the time from the beginning of the time match for the project City's 1971. Okay, So that is the way how it works. And it is the most common classes for handling date and time data in our neutered simplicity and efficiency, The preceding started talking. So it is really efficient while strong that it time Rita and Israeli simple also understand projects up with any Jump Box. So suppose I want to create or city object. So for that I'll, I'll give a name and date and time does cross city and and you add dot project City. And he had alkene date. And it'll give light to dark Grundy 3722. That means 20-second to lie to three. And here, Algol, hardly real bar 30 min, 45 s. And this is the date and time. And he and I will give the time, john. Thank you. And I can give anything that I wanted. Okay. So I'm doing it less precise, UPC and then I'll try to predict. Okay, So let's see here now it is 22nd, July 2023, Dynavox hurting when I 45 s asked for the UTC time, John. Okay. Now we have the type is a time drawn. Now I want to access the individual components operate and time that is year, month, day. Our minute hand, second hand that we can do very easily. If I want to. The ear probe the date, I can use format and then I'll pass the date. Time underscore city, that is the object of the day. And then I'll yield the percentage way and this will give me one year. It will fit here from this object. So see you artist. Okay. Similarly months he'd gone to face. You can use date-time underscore city, presenters them what they did, time ZT and you can reach present a D or odd person digest. Wasn't it? M for a minute and S4 second. M and S, odd minute and second son also been the capital interest. Okay, So let me run this and now bring to the all these components. So I'll huge base. Yes. He and underscore CPD per month, month and loss capacity for date and a scarcity. And for the odd man out a second, I'll use the same thing, okay? And then I'll try to print this. So when I print a year, this product to sell 23 months is seven squared. The second odd is 20 artist that didn't minutes and 45 s of projects City object. Next one is for LT toxin or LD Stanford for six local time and represent date and time data adds a list of components. So here it was stalling As a number of seconds mingling of the time. And here it will store it as a list. Okay? So projects LT will Stanford or local time and represent date and time data as a niche top components, year, month, day, odd minute, and second will be stored as a list in the local time zone. So by default it will go to the local type. It in more flexible than Project city as it allows direct access to new yield components of force that I'm John handling. Okay, so here's an example of politics. Lt. I'll create a date-time object, pia, add dot projects, LTE, MLK, Martin, 20-second, July 2023. Callbacks are different, 45 s, and I'll try to print it. So, you know, here that we have not specified the Tango. And when I, when I'm printing, I'm getting 20-second collides, ozone, Grundy three, well bar 30 mi 45 s with the Indian Standard Time. The time John, my local danger, that is Indian standard time. It's coming wash of fixed to this object toxic. So now it is common with the local timed you only window or Lias in Project City, we have specified the timeline, Martin projects LT. We need not to specify that time drawn here. It will automatically take the local, local time, john, because stands for projects local time. Okay. And if we want to access the monthly minute and second, we can simply do by Eugene date, time underscore LT, that object that we have created for this dollar, plus 1,900. So what it will do when you put in LTE stored as year since 1,904 month date time underscore LT dollar M1 month plus one projects LT, start the month with zero. So Data plus €1 for January at 11:47. Similarly default they will use the MD month, day. And for odd values are for a minute, minute mean and 4 s. So let's run this and accessing with real items and then I'll print it. Does come here total and Randy, three months, seven day 20 s or so. Same thing we are getting and projects LTL real one liter. The presentation is a difference yet projects local time and stored the component has unleashed where Project city it is for. Calendar time represent date and time, whereas number of seconds since the beginning of the Unix equal time starting from Jan 197. Okay, so this is stuff we can handle the data, date and time data or odd. And there are two ways, project City and projects empty. And we have understood both of them 109. Project 7 Data transformation and Summarization: Hello and welcome back. So in this, we're going to do another simple project in which we are going to apply Data transformation and Summarization techniques on real world data. So to apply Data transformation and Summarization Technique on real-world data, we will use the MPG dataset, which is also available in our by default. So this MPG dataset is readily available in our package. So we need not to download the dataset separately. It is already available in the R package. The MPG dataset contains information about various car models and their fuel efficiency. So let's use the GG plot when apply Data transformation and summarize some technique to gain insights from the MPG dataset, which is the dataset about the various Card models and they're feeling efficiency. So the first thing we'll load the ggplot2 library. Lord, the data set that is MPG dataset. So we'll leave the data function and pass that dataset name that is already available in the R package. So data and pass the dataset name. Okay, So this dataset will be loaded. And then to explore the structure of the dataset, we use the function called STR function and pass the dataset name. So when we run this, we get the structure of Dataset, this MPG dataset. So here you can see manufacturer models, displacement, years, number epsilon dot transmission, dr. city, highway, and classes. So all these variables are there. Okay? Next thing is you will see the summary of the dataset. So let's run the summary and then pass the dataset name. Yeah, So this will give you the better understanding of how, how old it is there, what kind of read either so manufacturer, model, displacement, Year, cylinder transmission drive and city highway, and the fill and then class. For everything it is given the minimum value. First quartile, median, third quartile max, all those statistical values from this dataset. Okay? So looking at this next thing is the MPG dataset has several variables including manufacturer, model displacement and displacement in liters. And then then number of cylinders and then transitions for some type. Dre is Dr. type and BYU city MPG and WY is highway MPG. So this is city average city Mileage, and this is high BMI less of the particular Car. Now let's apply that Data transformation and Summarization techniques using GG plot. So first thing what we do will use the group bar Plot. And in this, what we'll do, we'll do the compression of every city. Mpg Mileage based number up cylinder. So we'll use the MPG dataset and use the pipe operator. And here groupby cylinder. And then this grouping by cylinder. Then we pass it to the summarize function. So every city average underscore city is called the mean of the city average. And then we'll pass this to the GG plot function, aesthetic function. We'll use what x's as.factor and we'll factor reject cylinder and why will sake of recipe and we'll fill with the number of cylinders, IJ factors cylinder, okay? And then do jamboard will use to pass the starting equal to identity. And then labs number of cylinders on x-axis. X-axis will give number epsilon, the y-axis ever city, mile, BD and see titles will give you every city and VD by number of cylinders. And team will use the minimum. And then we'll print the group bar Plot. So let's run this and see, let me just maximize the space so that we can see the graph. Okay? Yeah, getting some you see smoking. Mali's, right though this, okay, So this is saying because we have nocturia dplyr library. So let me know that the plots Okay, If we run now, see you nowadays come correctly that either because regards the by function is coming from the dplyr package that you have not loaded. So naturally it was throwing that are so now you can see evidence the DMB D by number of cylinders. So number of cylinders on the x-axis. And every city is Mileage is on the y-axis, so number of cylinder date and the myelin is displayed here. But actually enter four and diversity Mileage things on here for in fact Mileage things on here. So this way we can analyze how a number of cylinders in a car model is affecting every city Mileage. Then we'll draw the boxplot and Color box director really are the same thing and beauty, aesthetic exit, dry, highway, MPG by type, type, type, type, and Y axis we'll behavior and favored the type type and how those things we know already done this boxplot highway MPG by Dr. type. So we're getting here. Next one is three. We'll put up scatter plot and that will source the city MPG versus engine displacement by transmission Types of what this will plot a scatter plot. So GG plot function, MPG dataset aesthetic, X displacement, y-axis will be city. And then color is transmission and geom point to plot the points. And then x-axis, y-axis, x-axis, them displacement y-axis, a damper you titles of this. And let's run this and see the output. The scatter plot by city MPG versus engine displacement by classmates and type. This is for automatic, automatic and manual. All those things we can see. Next is I'll even put the faceted histogram city MPG distribution for each type type. So here, same thing, XXS city and the derived type and geom underscore histogram in which will give to Title white, eye color white and theme minimal and fester. And here Reports the type type, and then we'll run and print the I'm time. It will come like this. So this is a festive histogram, city MPG distribution, right? Dry pipe. This is the city Mileage and this is the frequency. And this is why the dry pipe, this is that right? Okay. So what we have long in this small project, in these examples, we apply the data stone formation and summarizes some techniques, such as calculating the evidence city MPG by number of cylinders and using groupings to create insightful realize. And we also applied to summarize and using boxplot and medulla the Distribution Absolutely MPG for each type driver is interested in histogram. By applying that it does not form. Resenting somebody isn't technically you can uncover patterns, trends, and adolescents if in the data and making it easier to draw some meaningful insights from real-world dataset. From real-world dataset. Remember that ggplot2 offers various functions and flexibility to perform further determined lesson every lesson. So you are free to explore and customize the plasmid on your analysis need. So this is a simple project of plumbers and then somebody some techniques on real-world data that it MPG dataset. So that we have seen 110. Project 7 Part 2 Data Filtering and Color Scales: Hello and welcome back. In this lecture we're going to use conditional statements and Data Filtering in GD plot. Okay, so ggplot2, we can use conditional statements and our data first link to customize our plots based on this specific condition or subsets of data. Whatever requirement will be based on that, you can do that. This allows you to create a more dynamic and insightful visualizations. So in this lecture also, we are going to continue with the MPG dataset, which is the dataset able cards, models and their Mileage and different other things. Okay, So I'm preaching it doesn't to demonstrate how to use conditional statements and Data Filtering ggplot2. So first thing we wrote the ggplot2. Okay? And then we load that data set, that is MPG dataset. Did MPG dataset. And then we'll create our scatter plot, city MPG versus highway MPG with color made on the drive time. So yeah, that'll create our variable. That is scatter underscore plot. And here I'll use the digit ggplot2. And here dataset is MPD, does it aesthetics, fonts, and I'll pass CD and why highway and Color made under Dr. Greg. Okay. And then geom point functionally or to draw the point on the Plot. And then labs function to give the names projects existed, MPD, y-axis, highway, MPG, NBA title of the scatter plot. We'll with scatter plots to TMP versus highway MPG with the color red on the drape type. And then we'll theme will use the minimal, and we'll print the scatter plot. So let me on this. Now we're getting a scatter plot which is showing us the CT versus highway MPG based on the dry type. So these, these are the x-axis real having city MPG, y-axis is highway MPG. And the mileages sewing based on the dry pipe. So this color, colors are swaying the dry pipe. Okay, So this is the scatter plot that we have drawn. Now, what we do next in this basic scatterplot, all data points or see someone with different colors but under dry paper. Now, let's add some conditional statement and Data Filtering to customize the plot further. So now this plot is data points with the different colors buried under drive time. Now we'll use our conditional statements and Data Filtering to customize the plot. For the first thing, what we're going to do, we're going to create a scatter plot with conditionally colored points within the city MPG threshold. So here we'll define the threshold velocity Mileage. That is city underscore embryogenesis called threshold. This is the or whatever lemma ANA tests all value I'm giving us 20. So now scatter plot with conditionally colored point. When our city MPG touch all that we are defining here as 20. So GG plot, the data set aesthetic function. The same thing. Accessing DY is hi Rick and Color here is CD. And real giving the test or less, greater than grantee than those points only wanted to point D greater density and density. And that's going to predict as all that is currently source district greater than quantity. Okay? And then geom point function, and then we are reading scale underscore colon underscore manual function. And here we are giving the values for the colors red and blue and labels we are given below threshold and above threshold. So based on if it is greater than, relative to, it will be above threshold and less than 20, it will be below threshold. And then Laugh function we are using to give done named for the Plot and all. Okay, So let me run this. Okay, We're getting some letters. Here. We're getting some error. Problem while computing aesthetics. Add up a garden first layer called YLL objects city and sold. Not want August, sorry, I have not run this variable. So now we have. So let me again. So see here now we are getting this scatter plot with conditionally colored wind based on the City MPG. So these blue points are above threshold, that is currently. And the red points are below test on that this city MPG, less than 20. Great. So now the x-axis city MPG, y-axis is higher-energy. Blue points are representing the above threshold value that is currently city MPG grantees. So above threshold value, loop wipes and milo our swing as a, okay, So this is one condition we have applied. Next thing, what we'll do in this ligament that I explained a little further. In this plot, we use the aesthetic function to map the color aesthetics velocity greater than city MPG threshold value that is greater than Bronte, rigid logical condition that evolution true or false based on whether the city MPG is above or below that threshold is specified. The color, scale or desk or colonoscopy malfunction here to serve the custom colors to the points above and below pterosaur along with the custom levels. Next thing what we're going to do, we're going to use that Data Filtering will create a scatterplot with Data Filtering respect lifetime. The filter data for a specific Dr. Fred type. For example, F4, front-wheel drive. Okay, so here I need a variable and assign it as F, that will with a far front-wheel drive. So that I play front-wheel drive and freely God, scattered plot on this valuable alkylate. And I'll use the GG plot function and an advisor data, MPG. And here I'll give MPG and dipole-dipole defined as a dry pipe that is front-wheel drive. Here we'll passing the dataset to the GG plot function. I'm specifying that from this dataset that is embedded it as if I want to take dipole-dipole. They does it data which are having the dry pipe as a front-wheel drive to MPG. Dollar DRE is equal to drive type, that will be the front-wheel drive. And then aesthetics function will pass X is equal to CT. And why do I ran color based on the drive time and all other things. Geom underscore point from somewhere loose to Plot and lot and labs function. And then we will try to run this and see how the plots can see this either scatter plot, city MPG was this highway, MPG for front-wheel drive. So this is the scatter plot for the CD. And I lay my little car which are having front-wheel drive time. Okay, so now you can compare, okay. So in this plot to use the data file in Procreate stack scatter plot only for a specific type type that is front-wheel drive by substituting the MPG dataset, where we have done the Substituting here, Data is equal to MPG, MPG dollar, dr. V is equal to dry prep. So here we have taken the subset of the dataset from the MPG dataset. And that's a project subsidies based on the dry type that is front-wheel drive. So we have taken the data one for dodge vehicle damage vehicles, which are having front-wheel drive and then for dolt front-wheel drive car models, we have compared the city my ligand hybrid Mileage using the condition MPG dollar, dr. V is equal to dive type jumper. Demonstrate how to use conditional statement and Data Filtering in ggplot2 to create more customized and insightful visualizations. Next thing we will do will use the color, customizing Color Scales and legends. Customizing Color Scales and legends in ggplot2 allows you to improve the visual representations of data and make your plot more informative and visually appealing. You can customize colors, labels, breaks, and other aspects of the color, scale and legend to match your specific visualize and list. Let's continue using GG plot. How to customize Color Scales and legends and jittered plot. So we'll use the GG plot library here. So let's run this. We'll use the data Data, MPG datasets to load the data set that is MPG dataset. Then we'll create a scatterplot city MPG, highway. Mpg. Mpg means MPG. Okay. I hope you know, I explained earlier also with color based on type, type. Okay. So you're GG plot to MPG is that it doesn't excuse aesthetic from somebody project straightaway highway and Color and drive. And the same thing we have done earlier. So let's run this. This is the scatter plot we are getting Okay. So four-wheel drive friend from driven real drape for four-wheel drive is red one, blue one is rear-wheel drive, and green one is for front-wheel drive. Okay. Now, in this basic scatterplot, all data points are sworn with different colors based on the dry type. In GG plot to automatically assigns colors and create a legend. Now, custom color and legends will try to give so custom colors will create a variable. Customer underscore colors and create a vector. Here. Will create a vector and yellow will give what F will give. Blue for our will assign read and for four will assign doing. Queens waterfront will drive, it will be blue, rear-wheel drive, it will be red and poor will divert, will be green. And then we'll create the scatter plot with God. Let us Galen legend, so here called custom underscore, color underscores scatter and that's called plot. This scatter plot, we're going to create GG plot, your dataset same, and exit the highway. And here color based on the drive. And then geom point function will huge to plot the points. And then we'll use the Gayle underscore color and does come manuel to assign the color that we have created here. And we'll give Values is equal to custom colors this. And then we'll create the scatter plot. So let's run this. Now. Our scatter plot with the custom colors that we have defined is coming off here. So green is for the four-wheel drive, Bluish for the front-wheel drive and radius for the real, real drive. Next thing is, or what we have done on, let me summarize. In this plot, we use the colors scale underscore colon does commandment function to set the custom color for each type type. We define custom underscore colors as a named vector where the names that represent the drive Types there, forefront will drive RPA real-world ripe, and 444 wheel drive. And the values represent the corresponding colors. The scale and scale. Underscore color does come. Manual function allows us to map the custom colors to the color aesthetics in the plot. Next thing what we'll do, we'll try to add custom legend and labels. Custom legend levels entitled to outlawed. So scatter plot with the custom legend and level that we are going to create. So here GG plot function, MPD, aesthetic exit TY, CD. Why highway and elaborate on the type, type, geom point to plot the point, scale, underscore color, underscore manual to give the custom colors. So allergists call to see. Now that we have already done. Then here we'll add labels and he or C will create front-wheel drive, the elbows drive and affordable, ripe. So here we are given. F will be replaced by front-wheel, will be real and four will be 4-way. And then we'll give Dr. type. And then and then Laugh function will use to keep the names. Okay, so let me run this. See you nowadays, swing ensued up for the F, odd and 40 destroying us the direct labor as front-wheel will end for real. So this way we can customize the legends, okay? So in this plot we use the levels argument learned. A score is scale underscore color and rus per minute function to save the costume levels to, for the Landry also use the name argument to provide our custom title for the legend. So this is the name argument we have used to give the custom name to the legend. So here it is coming, right type. Okay. So I hope you understood how we can use the conditional statement and Data Filtering and how we can customize the color skills and legends 111. Creating interactive plots with plotly and ggplotly: Hello and welcome back. In this lecture who we are going to create interactive plots with plotly and ggplotly. So plotly and ggplotly libraries we will use to create interactive plots. Okay, So what is ggplotly? Ggplotly will enable us to ggplotly a function that will allow us to convert lot that we have created GG plot to two. We have named interactive manner. So any plot to have created with ggplot2 can be converted to interactive plots by using plotly and ggplotly. Ggplotly. Okay? So creating interactive plots with plotly and ggplotly in our allows you to enhance your visualizations with Interactive Planning. And many more features. Plotly is an R package that converts ggplotly to graphics into interactive web-based visualizations. Okay? So Plotly is another package that we are going to you. So you can install plot. Okay, so to install you can go to the tools and click on Install Packages. And then here you just search for Plotly, click, select that, and then click on Install, and it will be installed. I have already installed, so I'm not going to do it again. And to check this install different densities. Okay? So I'm canceling it. You can click on Install if it is not installed already. Okay. So we are going to UGG ggplot2 and plotly library in this, okay, packages in this. So what I'm going to do, what I'm going to do. I'm going to can you step-by-step guide to create interactive plots with plotly and ggplotly. Okay, So the very first temperate install and load the necessary libraries. As we already know, we have seen how we can install. So let me load these to leverage this first. So once these two libraries loaded, next step is good data GG plot, you ggplot2 plot. Okay, so let's start by creating a basic GG plot to plot using MPG dataset that mtcars dataset that we have already seen. Okay, so load the MPG dataset, but you didn't get a function. Load. And then we'll create a scatterplot, simple scatterplot to create a variable GG plot underscore scatter. And I will use the ggplotly. Wong Sun has use the data set MPG. And then we will use the aesthetic function, x-axis, sprint and y-axis and plot the highway mileage and then color red on the class of the Reagan geom point function we will use to plot the points and lab ponds and we'll use to give the name for the x-axis in them disbursement L and Y exists highway MPG and title of the scatter plot will be scatter plot, engine displacement versus highway MPG by radical class. And then theme yet we will be using theme minimal. And then we'll print the scatter plot. So let's print this simple scatter plot you didn't do to ggplot2. So see, you see here this is the simple dizzy plot, those scatter plots drawn in-between an engine displacement versus highway MPG. Why vehicle class? So radical classes here, two-seater compact, midsize, me, minivan, pick up, sub-compact, an SUV. So these are the different, different colors for each class. And here on x-axis we're throwing the displacement, okay? And then on the y-axis we're seeing the highway MPG. So this is the simple scatterplot. This is not interactive right? Now. I want to make this Chart, make this scatterplot interactive when I hover over it. So something, okay. So the Values, what each point and all those things I can click on June, I can manage all those things I want to add to our scatter, become more interactive so we can interact with the scatter plot. So to do that, we will use, we will convert this GG plot plot to Plotly interactive plots. And to do that, we will make the ggplotly Interactive using ggplotly function This function is very important to convert any GG plot, DG plot to function, sorry, plot into the interactive plots. So will you ggplotly function from the Plotly package, okay, and then convert digital plot to Plotly interactive plot. What I'll do, I'll create a variable here, simple interact to underscore plot. Just any name you can give here, and then unload the ggplotly function. Here we have UGG plot. Now, you'll ggplotly. And I'll just past the scatter plot, that data we have created using GG plot to. So unless passive to the ggplotly function. And then I'll just print this interactive plots. So just passing to this ggplotly, this simple scatter plot will be converted. It works, converted into an interactive plot. So let's see whether this is happening or not. Now you can see here this scatter plot In den displacement versus highway MPG by vagal. The plot is theme, but whenever I'm hovering over any point, it is showing the values corresponding to that point. Like for this point, displacement is 1.8 highway, MPG is a 36. And class of the list sub-compact. If I come here displacement 1.837 class compact. And if I come to the pink, these displacement 2.5 hybrid 27 and classes SUV for this. Okay, so see here, it is pretty cool that now the simple scatterplot, just bypassing that plot to the ggplotly funds and retirement, convert it to a beautiful interactive plots. Here. Here we can choose the C. When I'm clicking C, no point is being sonya Because I have this selected, unselected everything. So I'll select two-seater. So while it two-seater railroads will be sonia if I want to add compact also. So I'll come back, click on compact and compact. Deleted points will be reachable. Now, if I click on midsize size and are Missing Card Scheme minivan, We knew I ran can become way whichever you want to analyze, you can add, if you want to remove that, even just unselect and it will be done. So this is the interactive with that has been added by simply passing the scatter plot to the ggplotly. And it's, it's pretty simple and pretty cold here you can see your download the plot as PNG. This option has also come here. Then we have the Duma. Often we can click and we can do this. You see here this various options are coming here to ban, ban, ban like this. And you can analyze this is the band punctum. Then we have the box select. We can select a particular winds here and you can analyze those points which are collected will be highlighted. Next thing is we can Lasso Select. You can select like this in whatever way you want to select it and select if you want to select only a particular thing you can select here. So these interactive things has been added, thinks has been added just by passing the scatter plot color Plotly function. We can put that out. And we can put Jermaine. And then they said Texas. And then we have the subplot. Okay? So this way, okay, so now we're anestrus, but just by simply passing the scatter plot to ggplotly function, it will create a Interactive. Now the GG plot to scatter plot has been converted into an interactive plots with plotly. And you can interact with it using your mouse and Dutch bet. You can hover over the data point that we have already seen and view that is seen and mood or tone Tips and Jermaine and your mouth band Plot and many more things that we have already done. Now, we'll do some more cached myosin to this plot. So this again, okay, So this is the plot, interactive plot we have created. Now we'll do the additional customization. So now we can further customize Interactive Plots using Plotly function. For example, we can modify the Harvard tax at just the color and add more annotations to it. So let's do that. So we, what we are going to do, we're going to customize the Interactive plot. So far this will create a variable Interactive underscore plot, underscore custom. This is the customer interactive plots name you can give. And here, I'll use the Interactive to plot that we have here. And we use the layout sun. And inside the layout, what we can give, we can give the title customize. Interactive plot will be the title, x-axis title will give the engine displacement in bracket report. Yes. Sorry. L. And y-axis will put the highway MPG legend output list. And it will be the vehicle class and the hallway and mode will be the best. So let me run this and this will create the demise Interactive plot, and let's print that. So now you can see here the x-axis, it can predict crashed my Interactive plot. And here you can see the points being song. We can band like this. You can select box select. You mean your mouth. All those things we can go. In this example, we went divide the title axes and levels and legend title. Okay? So by using plotly and ggplotly Function, Plotly package and ggplotly funds on where you can easily create and drop coverage that is in front. Gg plot to Plots, making easier to explore and understand your data. So I hope you understood students in the next lecture. 112. Introduction to Plotly and Key Features: Hello and welcome back. In this lecture, we're going to learn about Plotly, and it's free. Just. So Plotly is an open-source library that allows users to create Interactive, high-quality, charged, and ritualized practice programming languages, not only in my Plotly is available in Python, R, and telescope. So Python and tell us Control so you can use plotly. And here. Plus we're going to learn about Plotly. Plotly offers a wide range up chart types from basic scatterplot to complex 3D essence and see how we can now regionalized and 3D Anna, how we can map broccoli. So let's look at the interrupts and up Plotly and Key Features. First thing is interacting tractor visualization. So Plotly enable us to continue it. Interactive lesson, nonferrous interactive capabilities. It enables users to interact. Let's chart and two, in and out fan hall or data points to see the details and toggle Data Series on and off. So these are the key features and that gives you a pointer interactivity with the plots and the like. Interacting with the charts. Show me your mouth. Banning charged, hovering over the data point to see the details problem, the data CDJ on and off. These interactions make it easy to explore and analyze data in an intuitive manner. So that is the best thing about plotly it's gets it makes the chart life in track with deal. So, and next thing is easy to use. Apa or CLI provides a simple any UCR use APA for creating what your license. With a few line of code. You can generate complex and Interactive Josh, making it a favorite among data scientists and analysts. So since it is pretty simple to use as much popular among Data Science Lesson tactics. So since plot laser level in our telescope and by Thomas, quite popular, multiple programming languages, that is another key feature of plotly. Plotly supports multiple programming languages, including Python and JavaScript. This allows to work with their report language and seamlessly switch between them. For different tasks. Chart Plotly supports a wide range of chart types such as line charts, scatterplots, bar Plot, histogram by Josh probably chart 3D Plots, choropleth, maps, and many other. It provides flexibility and versatility in ritualizing radius Types of radon. Then Plotly also provide web-based visualization. Plotly, web based, which means you can easily interactive charts and dashboard Online and embed them in web applications. Handy pores. So Plotly actually give you the option to integrate your Plotly charts and dashboards. Now, we embed that into the reports. That's more than web application with Plotly dash, you just can create interactive web applications and that's for Data. Click on the data Visualization. Making it powerful tool for building data-driven whoever Applications. There are few cost-wise substance actually, partly, partly, partly offers a wide range up Customer options to tailor the appearance of charged according to a specific requirement. You will just can customize colors, labels, font, styles, and Regression, and much more. Animation and ggplotly supports animations and smooth transitions. Wechat very useful for visualizing tinted in data over time. When exploring different states of greater export farmer ggplotly realizes it's can be exported to various formats such as PNG, JPEG, SVG, PDF on it, and even animated GIFs, making. You couldn't go up great presentations and documents. Integration with other library of also quite possible in partly Plotly can be easily integrated with other Data Analysis and Theory lesson libraries such as pandas in my tongue, RTT ggplot2 in R. This allows, you just took combine the strength of different tools. So these are the key features of Plotly interactive religious NADPH, APA. Multiple programming languages support each data type support web-based application. Support her best way. That sports and web applications. My animations and transitions. Plotly is widely used in data science, business intelligence, and scientific resource and various other domains. Trochlear, intuitively appealing and interact to replace red. Okay, so in the next lecture, we'll start doing the hands-on with the blocking. The next lecture. 113. Working with Plotly: Hello and welcome back. So in this lecture we're going to walk on plotly, will create Plots using Plotly. In the previous lecture, we have seen the introduction to Plotly are seen what are the features, which are the plots we can make using Plotly? So in this lecture we'll do the hands-on. So I have written some code for you and I'll explain and will die execute the code and then we'll see what are the things we can do with the Plotly. Some of the things like will create this 3D Plots using Plotly. So this thing is quite interesting and I tell you how we can plot these three 3D scatter plot. Okay, So these are the few things we're going to do in this lecture. Okay, so let's start this lecture. And things have taken from RStudio website rpubs.com and references from this. So let's get started. Plotly is a great tool to just to create beautiful interactive plots. This can be plotted using Plotly package as well as using TD ggplot2. So we can do with things we can only plot with the plotly, or we can also use the GG plot to along with Plotly. For this, we need to libraries Plotly and DD ggplot2. So I have already installed, so you just run it. And then here I'll use the Midwest dataset that is readily available with the RStudio are package. So let me show you how this looks. So let me put Control Enter. See, yeah, this is the dataset that we are going to huge Midwest, Ok. So there is a PID then country, state, area, population, total population, population density, population wide population, black, pop meridian, pop, ancient, other. But why black? So although each columns are there along with D, each column then add more variables and they're all in English variables. We're going to be the plots, okay? So this is the dataset. So like country, a state area, Population Total, all those variables are that okay? People said that. Okay. So now we reuse this data to plot. So first thing what I'll do, I'll use the Plotly function, plot underscore. And why Plotly function to create a plot for this LPA, pass this Dataset Midwest and x-axis, I'll huge part class, but class is a Dataset, Data column here, variable. But Clark College, okay? And Color and use the hidden state and Types box. I'll create a boxplot here. Plot percent college-educated, rushed it per cent college educated by the state. To plot this. So see here this is the boxplot because type we have given here as box here. When you click here, you can see that minimum first quartile, median, third quartile, upper fence. And you can see the max value, those things. So this way, we can quickly create boxplot, 4% college educated by step. How many people are getting educated through? College? State that this is the boxplot, okay? Now we'll try to put some basic Plots using Plotly. For this. What I want to do, what I want to create a basic plot directly using plotly or we can UGG plot two as well. So here, UGG ggplot2 as well. So here I'll use the Irish dataset that we have already seen. So if you want to see how this in this dataset You can put head iris. So let me first load this. So this is that I just dataset where you can see the sepal length, sepal width, petal length, petal width, and a species. These are the columns or variables have a level in this Irish dataset. Okay? So if you want to run this, you can see, yeah, okay. Next thing is I will use the plot underscore LY function again, and data set will be itis. So data is equal to Iris, Xe, x-axis and the sepal length and y-axis petal length. Okay? So let's draw this better plot that it's coming up like this. Okay? So on x-axis, simple dot length, and on y-axis, petal length, sepal length, and petal length. And these points have been drawn. So for this sepal length and petal length husband plotted here. Now, with ggplot2, the same plot can also be created in ggplot2 and served as an object. Then the plot can make, can be made interactive version ggplotly, Plotly function. Note that not all the features will work when using ggplotly. So sometimes have to do some trial and error. Okay? So here, the same scatter plot I'll try to plot using GG plot and plot that I'll create a figure to where the store in the figure to figure, to store in the figure two. So let's run this now. And I alluded ggplotly to plot this. Figure two. Okay, so let me run this. So see here, now, coming like this, okay? The same thing with the help of and ggplotly. We have plotted. By being able to Plot you ggplotly to. You can customize the Plot the same way that you can with ggplotly. Look at the common column names. So if you want to promote the column name for the Iris dataset, you can just simply run this. See the column names that we have already seen, right? There are five column points by species. So here you ggplotly, data, iris, aesthetics, fonts, and we'll use X, sepal length, petal length and Color is multispecies. So we will color the points by species. And then I'll use the geom underscore point function to plot these points. And then this figure to that we are getting from the TG Plot, I'll try to plot with the ggplotly. Okay, so simply print this and see here, now we have the colorful scatter plot and where this pink that is greenish for versicolor and virginica. This way we can for the color. So now you can also see that the species appears in the text box as you hover over them. Now it is showing you hover over them. Now it is sewing sepal length, petal length, and is preceded by Anika. Just like and ggplotly, you can customize the axis, title, and Color, a sector here also. So he has GG plot. Whatever we can do in GG plot that we do here. And then we'll pass that figured how valuable to the plot variable to the ggplotly. This object figure two to the ggplotly and it will plot, okay, so here the diaries X axis and then geom points here. Inside the geom underscore point function, real-world aesthetics and color each quarter is Prices and save also report based on the species. Okay. And then xlab sepal length, petal length and GG title will be sepal length, petal length. Okay? So let's run this. Now. You can see here. Now the points are also lots. And for this versicolor is coming in triangle, and for this guy, points are coming squared same. So this is because we have used separate species inside that human Netskope points function. This way we can customize the axis, title, and Color also. Now we'll create a basic bar chart. So to create basic bar chart, we'll use the data.frame. And here I'll create a sample dataset here. For this dataset, there will be a three columns next time And total bill. So male, female will be the sex for this Alcatel factor vector, and then time will be the lunch, dinner. None to be not okay. Levels will be lunch and dinner, and total will, will keep some random bills for here. So let me create this data, activate this DataFrame first. Now I'll create a bar graph. Time on x-axis Color Fill, group by sex and huge producing underscore dodge function here. So GG plot to will huge and data as you go to this one, that one object that we have, some training data that we have created. And then aesthetic function X axis, we'll put time y-axis, I'll put Crotonville, and I'll fill with the sex and geom underscore bar. I will use stacked and put identity and policing, the police and underscore Dutch function. And then I'll create another object, figure four, and I'll pop Boston bar graph G, ggplotly, ggplotly a, B, and then print the figure for C. You see here. Now, this is the simple bar chart. Okay, so he had lunch and dinner. Two things are there then this system for female and this is for male or female total bill per meal for lunch. And this is same thing for dinner. So if you want to analyze like at the lunch of female, total will abstract interface. Total. We love sustained rupees and for dinner, females that we love cysteine and MATLAB, rs17.17, 0.0. So this, we can analyze that at lunch, male and female, the expenses on lunches quite different. Male, spend more on lunch for dinner. Even male and female. This, we can conclude that for dinner they bought the expenses are almost same. Male and female day. Well spent same among almost same amount of money on dinner. But when coming to that lunch, females, they spend less on lunch. And will they spend more on lunch? This is what we can conclude based on this Bar Chart. Next thing, we'll use the basic SF. For this, we'll use the library itself. And for this we'll use the NCDs object, SF, colon, colon, underscore read System dot file function will yours and yet review SIP slash NC dot SHP and packet will SF and SHP and packet will SF and quite equal to true. And then we'll create an object figure three. And ggplotly, really huge and ggplotly. And we'll pass this NCO. Object to ggplotly plus geom underscore safe. Do you underscore SF fonts and review? And he had aesthetic forms and religions like this. And we'll go fill it by area. So let's plot this simple surface plot here. So now we're getting this surface area. You can see here on x-axis and y-axis. So at degree. And you can see etiquette at rest at different ways. So this way we can leverage to plot. Next thing is we can also put the maps for this. I'm your Zynga. That's it for February, us and traffic. Okay. So this data taken from this website directly readapt We are creating from this website. And this data is Part February 21, US Air Traffic dot CSV file. So we'll store that into this DataFrame. And then we'll use will try to upload this air traffic on the map. So far that value that you're styling. So I'll create an object G. G and scope is the US The scope of this map is us projects and these lists type projects and is list type. Albert's USA. Land through land will be our son. Land color will be to RGB and we'll use gray 95. And sub unit color will be greatly five country color will be great at five country with will be 0.5 and sub unit with blue 0.5. So this way we can use this. Okay? And then we'll use the plot underscore GO function to plot this on the map. So far these are past the dataset as dF, this one, the airport traffic notch CSV dataset we'll put here. Then latitude will put latitude and longitude will put luggage all the, these are the two things that will come from this knee up dataset. Does it really thick? Add markers and texts which will put airport, city, state, arrival. And then really rich color, symbol, square size. I hover in for texts. Okay, So let me run this tool and then forget color value in huge incoming flights. And for layout and use the title, the layout function to layout it. And then try to will be the most traffic USA are blocked. And then read on this together. Then really try to put something. So now it is coming as most traffic us add port Hauer for airport. So here, when we do see that glass yet Buck international arrivals are 90, then you get arrivals or 100 to this piece of friends that I will, 379. So you can hover over the map and you can find the Central Australia. Remington is for 26th arrivals for this eagle country regional 292648 in 1998 to Salt Lake City International Airport. So this way you can find the busiest airport. Flux us. Next thing is we can plot the rational actor creates. The next thing is we can plot the raster trash talk creates the colored heatmap with two variables acting as the X and Y coordinate. And third variable mapping onto the color. For this, we'll use the step two and Plotly. For this we'll use the magic function and we'll use them while cannot. Okay, so now GG plot to this dataset and aesthetic value1, value2, and geom Rushton, aesthetic fill Values can fill. Distiller will use this Function scale and Oscar feeling. Pellet will be spectral direction one. And this lag function will give x-axis west to east and y-axis north to south, title and sands. And then we'll try to plot this. So let's run this and see that after has been daughter, a disease elevation map for Mongo while. And this is created based on this. Okay. Now, next thing is we'll try to kidney duct 3D scatterplot. So 3D scatter plot for that, I'm going to do the empty cars. Okay? So empty cars, an empty car, M equal to zero and M equal to one. This is called automatic. Manual. And then here empty cars, edge factor, empty cards am, and then try to plot using plot underscore only empty cars dataset that is inbuilt into the are. Really use X bar, WT, Wi-Fi hotspot edge project for you sick Wi-Fi hotspot edge project for you, sick color based on the AM and Color. We'll define here as well. This is for color and the colors will give this talk color. And then for layout will use exit, tie await and y-axis cross horsepower and get axes will be 141 by four mile time. So let's draw this 3D plot. So CAN know this is the 3D plot, the scatter plot that we have done. So for this, you can see then our x-axis, y-axis. Y-axis is for gross horsepower. And the third exon, my time. So when you hover over the point, you can see 3.78 Y is one at and jedi. So this way we can plot the 3D Plots and so nice and so mode. So this is x-axis, y-axis, and that Card excesses there axes. So Plotly package in other larger to create highly customizable interactive plots, maps, because it's already ready to integrate with GG plot, the possibility and less. Okay, so you can do more exercises on this and you can practice and you can create much more interactive 3D Plots, maps and all. 3d Plots, maps and all, okay. So this is on a boat, Plotly 114. Creating 3D Plots in R: Hello and welcome back. In this lecture, we are going to create 3D plots in art. So I'll give you all a few different, different examples which we can create 3D plots. Okay, So first, I'll use the Plotly and for this user library properly. So you have not installed, you can install it. So first thing I loved the plotly library and then I'll create a sample data by using a non-function. So x, y, z, you know, 100, 100 and project exists and use squared plus y squared. Okay? So, but this will get that three. Does it later? Now, I want to plot these three points using a 3D scatter plot. Using Plotly. For that, I'll use a wavefunction broadly unclaimed plot underscore alway function called x, I get x squared, y axis x and y and then x is celebrated. Type will be the scattered 3D mode will be the markers and harmonicas and huge list size, I'll give three color, I'll get paid on the y-axis and color scale and you're done with this, okay? Then layout, I'll use the scene and for the list, I'll provide a list of which will contain the x-axis, y-axis, and jet axis. And for that, I'll give the Title X, Y ended. If you want, you can put existence x-axis, y-axis, and that x is okay. So these three titles I'll give and I'll try to, let me run this. So see here, this is the locked 3D plot. We got 3D scatter plot, we got 3D scatter plot. We are getting here, okay? So now you can see your x-axis, y-axis, and this is the jet axis, whatever way you want. So C and now x-axis, y-axis and that x is, these are the points that we have created using our num has been plotted in a 3D scatter plot. So x-axis, y-axis and that axis. So whatever we want to analyze, this way, we can create 3D scatter plot using Plotly. So let me revise again. I have created a sample data using our normal function, x axis point. I've created our norm of 100. Y-axis are no more 400 and dead x is x and y values. And this formula to get the points for 3D scatter plot I've use Plotly and the square wave function for X axis AB product X point Y axis y indexing. And then we have these things, layout and markers. And then when we run this, we get this 3D scatter plot. Next thing is but SP, SP function plus B bonds. And you can see in our programming, so syntaxes BER SP, this is a function name and we need to write that 3.4 x-axis, y-axis, and dead at this point, we need to write as an important parameter of this function as separate different parameters, x, y, and z with x and y are vectors defining the location along x and y-axis. And dead axis will be the height of the surface in the surface in the matrix jet. So this height will be the matrix yet, okay? So return value for this SP function will be the winged transformation of metrics for project in 3D coordinates x, y into the plane using homogeneous coordinates, that will be x, y, z, and t. So let's see the example of using this but SP function perspective. You can say sponsor. So whatever you want to, you can call, I call it SP, be sponsor. Okay, So here I want to draw our simple right circular cone. So to illustrate simple right circular cone, I create an object here called and for that function. And then pass the function, function at Boston that I'll use this square root function, x squared plus y squared. Okay? So this is the function for x values x and y are loose sequence of minus one to one, mentally retarded. And jet will be the outer x, y, and cold. So this will be the net axis and then brought the 3D surface, I'll use the parse function x, y, and these three points, I'll pass it to this. Okay, so let's How is it coming? See you? Now, this is the 3D right circular cone that we can create using this. So here endeavor code functions sequence is pre-generate the vector of equally spaced numbers and outer function to apply the function corn at every combination of x and y. So this is the right circular cone that we have created. Now. Now see one more example here. What I want to do, I want to add the titles and leveling access to the plot so far that this conference theme, then this also will be the same. Now, I plot the 3D surface and now we'll play are plotting the 3D surface while using the function and pass the x, y, z point here. And after that, I'll use managed CTO, perspective plot of a cone. Okay? Okay, So perspective plot of a cone. And then git lab, I'll give height for jet axis, I'll give a name as the height and Theta party ALX1, what is theta? And then phi is propylene and color is orange. And see which 0.4. Okay, so let's run this and see what output we are getting. We are getting this perspective plot of active plot of a cone. So here X Lab 11 dead lift can be used to the level, the level to level the three axis. And theta and phi are the viewing direction. Theta and phi other Windex and that we are seeing. So are we have you on Thursday and Friday, 1,245.15. Let's see what changes. See nowadays. Winning angle has been changed, right? You can make it. Let's see how it is tending. The viewing angle is changing, okay? So whatever angle you want to view that you can give in theta and phi, if I make it 4,560.45, let's see how it come and see this kind of like this, okay? This is T. Then it will come like this. Okay? Make a decision. Okay. Now, I'll use, I'll say to visualize the simple digital elevation model them. Okay, so far this will be two into volcano and genuine with ten into one column and row 10 m facing south to north. And why you will be done then into one into alcohol and cold. And it will take the jet and Danton be dispersing from east to west. Okay? Now, huge part is going to be degree and perspective function. I'll use X, Y, w1, 35 and fight all of these other angles. Color brown, skin falls, Theta one minus one. This is for setting and border and then false. So let's see how this coming. So see here this is just an Alpha digital elevation model. Okay? So this is how we can create 3D plots in R. So we have seen how we can do with the Plotly plot undescribed wavefunction. Then we have seen how we can use to function plus b functional perspective function, whatever you want, you can call it. This is how we can create TV 115. Creating Interactive Plots with Highcharts: Hello and welcome back. In this lecture, we're going to create Interactive Plots with Highcharts. So we have seen how we can do that with the other libraries. And now we're going to do the Highcharts. Back is here. And for that, we did to download the Highcharts. So you just go through the installed packages and such he had for high chart does how can so in dust and download it. Okay. So Highcharts library will be origin for this. Okay, so our programming language is widely used for statistics, data visualization and data analysis that we already know. Using Highcharts library. Highcharts library data is graphically represented in the software. Not only meaning, but interactive charts are also prepared. Okay, So let's see what kind of charts we can prepare. Interactive charts we can prepare with Highcharts, column chart, bar chart, pie chart, and scatter plot that. These are the four types of charts you can create with the Highcharts. So Highcharts in Texas, pretty simple. We'll use X Chart and then we'll pass the data object, then type, then at Cases, the points X and Y, and then color. So object to represent the data object type is the type of graph you want to create at it consists X and Y apprehend the Xs to represent the data. What is important and Color represent that plotting up colors, but you don't whatever value you provide. So chart will create the interactive chart, okay? And these four column chart by bar, bar chart, pie chart and scatter Plot we'll be getting it didn't. This is a pretty simple syntax of high chart. I chart, it's chart is the function. And we'll pass the data object type of the charter want to create and Cases, it will be the X and Y axis and Color, but whatever you want to give color. Okay, So first we'll create with the doc column chart, or column chart displays data with categories represented by a rectangle, sometimes called vertical bar chart as well. And categories are typically organized along the horizontal axis and values along the vertical axis that we know. So first thing is importing the library I charters, So Lamport. Then here what I'm going to do, I'm going to create a simple data, sample Data for our exercise. So here I'll create a vector which will contain the country name, America, India, Indonesia, Japan, Canada, China, and Brazil. And their GDP. So GDP vector will create and will give the GDP corresponding to country. And then we will give you the data, data.frame and country and delivery. So this will be the DataFrame. Okay? So next thing is plotting the column bar chart or column chart. For this. Store that into the P1 object and edge Chart functional use an object, all right, dot data, which will contain the country and GDP. And then type, I want to make a column chart. So I'll give column here. At Cases, I'll give X. X is I'll country and y-axis are called the GDP and Color. I want to put tax rate on you. If you want to make it something else you can give green as well. Okay, So let me run this and look at here. See how the hello Interactive, this column chart is. So let me just remove these and again, when I click see how it is appearing and disappearing in a very interactive manner. So this is how we can bar chart, sorry, column chart. Using Interactive column chart, you can hijack. Next we'll create a bar chart or column chart, display data with categories represented by the rectangle, sometimes called horizontal bar chart. Okay? Okay, so let's create a bar chart here. So same dataset I'm going to be with you. Then see the view data. If you want to see you data, you can see the country and GDP. Okay? Next thing is I want to plot a bar chart here. So I'll use the S chart object, data type Bar, edge cases, Country, X axis, country and y-axis GDP. And I hadn't read. Okay, so let's run this. Now. This is the bar chart that we are creating. So this is Interactive Bar Chart using Highcharts, okay, so see how it is appearing, okay? And when you see how it over it, you'll see the corresponding country and GDP, Okay? Next thing, each pie chart, pie chart or type of graph in which circle is divided into groups representing the proportion of category Values are divided among the various conferences in the circle. You sector the brands that category. So I'll use the same dataset, okay? And I'll try to create a pie chart. So here I'll give webpage equal to pi. All, everything else is same only type we are changing here. And the color, if you want to give some other color you can give, I will give you the pie chart. So see how Interactive Pie chart has come up here. So I'll address it again, see how it is appearing. And for each country, when you hover over, it will be highlighted. China, Japan, Indonesia, America. Like this. Yeah. So this is how we can create by Chart. Next, I wanted to get into scatter plot. Scatter plot, also known as scatter chart. You just got to the present values. What do different thing medic variable scatter plots are used to see, did listen to between the variables, how one variable is affected by another variable. Can we visualize easily? So I'll be using the same way, does it, and that really opinionated country and GDP. And he had used the S chart and diaper, I'll give scatter. That's it. Okay. Carla, suppose I wanted to give us yellow. Okay, so let's can you into scatter chart here and see how it is. You see the dots. So let me just run it again, CIA Boxer coming out here. So let me attain this to some other color. Make it pink. Pink. Pink is also not feasible that much and make it visible. Okay? So see, you know, this is the square root of block, okay? So this way we can use the high charter, high Highcharts library and we can give you the interactive charts 116. Project 8 Visualizing Airbnb Data in New York City: Hello and welcome back. So in this lecture we are going to do another small, very small project. And in this study, we'll do a case study of visualizing Airbnb data for New York City using GG plot. So Airbnb, hosting people on at your place. So we'll see how people are nesting and what is the cost or what is the r neighbourhood trends. So basically we are going to be within New York City data. So data will be provided here. So listing or CSP that I have downloaded from the Airbnb. So it will be containing the data for New York City, Airbnb data for New York City. So what is the objective of what this little project is to study is to visualize the game, visualize and gain insights from insights from the yellow ball a and B Airbnb data in New York City, huge entity will use the GG plot through and to try to visualize them, gain the insights from the real world data of Airbnb. Okay, we'll explore different aspect of data such as prior distribution. They were sector, okay? So dataset for this study or for this project, we use the listing dot CSV file from here when we did a sit for the New York City. The dataset contains information about various listings, including rice, neighborhood availability and other relevant details. So the first step is data preparation. For this will load the necessary library. So I'm going to use the deep layer. If you have not started the plan, you can go to the Tools installed packages, where I put the pliers and click on Install, it will be installed and then you can use the liability. Okay? So GG plot through and apply this to our library we're going to note for this project. And we'll see the structure of the dataset. And we'll take one term missing values and convert the DateTime if needed. Okay, So let me know the live necessary libraries. And next thing is listing large CSV file that we have on my local machine. So I wrote that. So I'll use that each dot csv function and pass the path of the CSV file with the filename. And I'm going to store that data into the a and B and B underscore data variable. So let's run this. Okay, So this data is having 48,008, 95, 95 objects, and 16 variables. Okay? So let me explore the structure of the data set. My using STR function and we pass this AirBnB and discord data. Let us it. So let me show you the structure for this. Let's see. So 48,008 95 objects and says two variables. Does this to my name and ID name. Religion for handling cost ID, host name, neighborhood, group, neighborhood. Let it out loud. The short-run price. Minimum now is the number of reviews. Larger, leucine-rich which got month calculated host listings, count, availability, research T5. So all these variables are there and they're datatype is also. So now we'll take all the missing values. So we can check the missing values by using, It's not an f function. And we pass that data set and the number of missing values and some monks and to get the sum of missing values, okay, proton number of missing values. So this then we print the number of missing values and dataset by printing this simple value. Okay? So there are ten pentatonic preparatory missing values out of 48, 95. Okay? Next thing is if the wire, if your dataset is not having good time, you can convert them. Hey, Airbnb data, last review, date, the date you didn't add back from some sense. I'll see your content is not there. Okay? Next thing is we want to visualize data or licensing. To do so, we use the GG plot to create insightful, visualize embedded on the alien. So first thing we will see rash distribution. So far here. I'll create a histogram for writers to personally that the d block and pasta. He had a B&B underscore data, that is the dataset that we have stored by accessing them Csv file. Okay, Then for aesthetics function x axis, our x-axis, we'll use the price. And then we'll use the geom underscore histogram to plot the histogram bin width will give 50. And Phil will feel that with the Skype loop and border color will be white. And labs, the x-axis will be pricing, pricing, diversity, and y-axis frequency and title will be the price distribution of Airbnb. Listing and team will use minimal. And then we'll print the price underscore program that we're creating here. Let me run this. See here. This is a graph. On x axis is price in US dollar and on y-axis is frequency. Listening. Okay? So this is fresh tissue vision offering Airbnb listings. Okay. Then we'll see the neighborhood. So for this, we'll create a bar plot, number of listings in each neighborhood. For this, we'll create a variable called neighborhood underscore. Underscore plot. The same data. Underscore data. We'll use the pipe operator to follow to the group by underscore group. This is the one variable in the dataset. And then summarize by count is equal to n. And then we leave the GG plot and x-axis will be order the neighborhood group, and we'll use the count minus column. That way, your access will be called and we'll fill it with geom bar from inside that will pass the identity. And last punks and x-axis neighborhood, y-axis, number of flesh tint, title, number up here be unrealistic in each neighborhood. Senior year, the minimum and team. Xis dot text element. You can do element under protect angle 45. So this will be the name on x-axis. I want propellant it by 45. And then we'll put the bar plot on this. On this. So this is how Manhattan and Brooklyn, Queens, the delta neighbourhood hand it is tilted by 45-degree. And this is the neighborhood and this is the number of testing for each neighborhood. New York City, you destroying the number of listings. Neighbor on group, Bronx, brooklyn, Manhattan, Queens, Staten Island. Okay. So this is how we can use a bar plot. How so in number up lifting weights. And then what we have understood by doing that, we can draw the following IR, the quietest response or the most listing below our target dollar and with some old class for that price. So that thing we can see from this histogram, see most of the 10,000 and some of the values are going up production that we can consider either Eau Claire. So most of them only. And then barplot number of listings in each group. The distribution I add across the parent and I borrowed, providing insight into the most popular areas for accommodation. So this way we can see it if you want. You can more regular license and you can just analyze the data and you can create more vigilant instance paragon, different, different specific good. Thank you. 117. Project 9 COVID 19 Data Analysis and Visualization: Hello and welcome back. So in this project, we are going to analyze the COVID-19 data. So COVID-19, one of the very bad thing that happened to mankind, humanity, or the deadly virus that was causing a lot of vets and people were suffering all over the world. So today we will analyze the COVID-19 data and try to do the exploratory data analysis on the COVID-19 data-set and can provide. When we do ADA, we get the valuable insights. So let's start by loading the data and then proceed with the analysis to find the top ten countries with the maximum number of COVID-19 cases. We'll do some more things also, but let's start with the minimal. So I have downloaded the dataset, dataset, country wise lattice dataset that CSV file that I'll provide to you. So if you want to go the bathroom and this app downloaded from the WHO website. For this, we need the library. So if we take not installed, you can go to the tools and installed packages and you can put the plan. Okay. And then you can download it and install. It couldn't be downloaded and installed on your hot. Okay, so next thing is what I want to do. So this part, you will have to replace whatever your thing. I pronounce that you've got your local directory path. Okay? So let me alone. Now, will this CSV file country wise latest data and store it in a COVID underscore data. Okay? So I'm going to use the dot CSV to read this CSV file. And then let's see the summary of this code, does it? So I put the summary. You can see here. Now we have country region, prompt, that record. Do you kisses new data, new to cover debts, hundred guesses, and then recovered or undirected cases that they colored confirmed last week. How many cases scheme and exon retention increased WHO region? Okay. So like this, there are these columns in this. It does it handle these columns in this dataset handed or minimum first quartile median values. So the next thing is, now I want to find the, if you want, you can do one more thing. You can check the structure of the dataset by using STR function. And now we have these many columns here, 197 objects and variables. So variable country, region confirm that. So these are the kind of, you can see columns and the data type also do it here. So let me find the top plane contains the maximum number of COVID cases. This will create a variable top underscore, underscore countries. And they'll use the dataset CSV file and store all this COVID underscore data. And so I'll use the underscore data and use the pipe operator. And I'll add in descending order by confirmed, confirmed cases and head. I want to see the top ten. So I'll put heptane. So it will source the top ten random unequal. We are then from there, they sent me another. So these are the countries with the maximum number of COVID cases. So WHO region these countries actually? So America. And it's our job In the record, we first load the required library and then read the COVID-19. Does it go coordinate squared, Theta. Then DataFrame, we then display the basic summary of the data set to get the overview of the data. Finally, we find the top pin contains the maximum number of COVID-19 cases and heck function pump the blood package been considered a failure, will contain the top ten countries with the highest number of confirmed COVID. And each goal will represent the country and each column will contain the virus at each country data. Now I want to do data visualization on this. So to visualize the top ten countries on carbon number cases, you use the pie chart, and we'll use the ggplot2 package for this. So the pie chart and do the GG plot to you, the top ten countries from here that we got from you. Okay? And then the aesthetic function x, y-axis will contract and pump filled with the country region. Geom bar, Yeoman Escobar function will use and status equal to identity will provide and cool art polar. We'll start with Jira labs. I'll give the title topper and countries, x-naught, y-naught and the contrary and team will be wired so that pie chart for this. So see here. Now we have the pie chart showing the top ten countries with the COVID-19 cosine maximum COVID-19 system. You can see most number of COVID cases and most number of COVID cases. Then United Kingdom. Then it's gradually first kingdom, then Brazil. And then we have a mean that is India. Then this one, then this blue, that is the CR. So from this we can come to know that US, Brazil, India, and Russia, where the top four countries, US, Brazil in the endosteum, the top poor countries, has the maximum number of COVID-19 cases and us being the top. Okay, so this code will create the byte charge. So in the proportion of the confirmed COVID-19 cases for each country. Now, we can add more dense texts and customize the colors of the pie chart to make it more interactive, informative, and visually appealing. Okay, so this code will create the pie chart. Now, we want to add more things. So I'll create a pie chart with Customize Colors levels. So please do the same thing. Here are the polar and that we have already used. And years after this team-wide function will do the same function again. And he hadn't put legend dot partisan bottom, more black into the bottom. And lot dot title will give sides yet we'll give it. Justice. Frank pipe, size is 16, and France will bold and texts. The element underscore text function and size will get legend. Text size will be 12. And sage for this title will be sites for title or 14. And would it be the board and box single-point force. Okay, so and then we'll use the pie chart. And for that, I'll use scale, underscore field and does manual. And here values will get manually. Are the many colors feel game, okay, so whatever color code you want to give you can get. And then we will create a pie chart with levels. So I'll take this pie chart and then I'll call X and aesthetic functionality, which I will come to a country dot region and Punk come from, right? When are you son? Stack. And then I'll run this whole thing together. And C, and now we are getting better pie chart with ligands and different colors. Okay? So now it's saying the legends. You said there has been because his kingdom gradually. And also here also you can see this board, okay, So this way we can create a custom bike and legend text. Now, I want to find the top ten countries dropped and country that has a recorded max number that find the top ten countries that have recorded maximum number added cases from the part that COVID-19 data-set, we can use a similar approach as before. We will sort the dataset based on the deck column and then select the top ten countries. Okay, So far this value. And then we'll use the COVID. I will read this dataset and top ten countries will use arranged by using that column. And then we'll take the top ten Okay, this one we have already. So CFTC, the top ten countries on the maximum number of bits. Okay? Next thing is I want to create a bar chart, but the top ten countries maximum number of deaths because you're the bar chart and geom bar, things have their angular five, dash one just on this. Now we have the bar chart up and countries for the maximum overnight and all those things are coming up. This is the pie chart. This is Sunda. Us bark. Okay. So this 1 bar chart and a pie chart. Well, the same. Okay? Now I want to calculate the percentage of recovery for each country. So part of that COVID-19 recovery, if you've covered graduate department 200. And I know we're getting recovered and pass into the recovery. Okay. Country data. So this is how a country and that covered percentage recovery. So these villi we have calculated the last one, okay. Now calculate the percentiles you can be for each country in top part that tell you that upfront countries and then create a pie chart. This is the bias chart, the percent recovery and adopted countries. Okay? So US recovery, Ivana is the max. Mexico. India is also having somewhat like comedy, and then Mexico. And then I want the top ten countries covered by chalk. This is the person to countering doctrine characters in the first sentence. By Chuck. Most active cases coming from which country that will find us is the country which has the most number of active cases. Top ten countries with the most active cases, so hard that he will die. Active cases confirmed minor debt minus the active cases come from minus that, minus recovered, will give us the most number of active cases and then we'll add that two pluses. Okay? So you can see the US, Brazil, India on the mock number vector. This is the percentage of active course doctrine countries and create a pie chart. This will show us the most number of active cases. Percentage of vector cosine doctrine countries here USE that will be 7.73% of it. Which contracts the maximum frequency of the guesses. Guesses will use frequency that because it's going to sum up. Then turning to see us as the frequency of that, which country experiences the maximum frequency of the cavity because this product some of recovered this. And so Brazil is having the most frequency of the court cases. Okay? So this way, if we want to analyze more you condition combination, your way of thinking and you can enlight the data. Okay. So this is all about COVID-19 data analysis 118. Project 10 Drawing Flowers using Mathematics in R: Hello and welcome back. So in this lecture we're going to talk about the project. And in this project we are going to huge mathematics and art programming to draw some flowers. So drying flowers using mathematics in our programming. That's what either goal for this project. We all know that when you look at the nature in everything that nature creates, you can find baton in that, even if you see a butterfly that has locked off, back on Dawn our string and looks over to false. So nature has lot of creativity. And it can be a lot of Baton under creators on the flowers, on the trees, even in humans. So we are, we are all humans have the same features like eyes, nose, mouth, here, hands, legs. But every single person looks different. Right? Apart from the twins. Even the Queen's have some different CSS. And that is the power of nature. So what we will do in this lecture, in this project, we'll try to draw some batons, which will further. So we'll try to draw a flower pattern. You do mathematics using our programming. So let's get started. So the reload to see here, the mandate will try to understand all those things, okay? The point is patterns in nature, that's too many examples of natural facts and batons that can be described in mathematical terms. Nice examples or say Bob snowflakes, the fractal geometry of school, broccoli, or how self similarity rules. In the growth of plots. Odd is a tool for doing serious analysis, but not everything in life is. As we know. Life is also funny and art can be used to have fun and to do beautiful things. We can do a lot of creative things also, Eugene, are, That's what the aim of this project does. Troy, we are doing this project, this activity, Newton, art and math in this class. So it's graphical power can be used to provide artistic images. Like we'll see how we can draw all those illustration we'll see when we draw, when we run the code, which is inspired by how plants are dead leaves. This fact, it's called phylo toxic and we'll solve as the basis for this project. Okay? So we will use the GG plot two packets. As usual for all the vigil existence in art, we mostly use GG plot to package. So here also we use GG plot to package. And apart from having fun, we'll learn many important features of it. And that will be useful not only to do art, but also to the present data and real life problems. So let's not delay anymore problems. Let's not delay anymore. Let's start by loading the library. So the first piece of code here, I'm using the AAD notebook. So the benefit of using our notebook is data. We can write a piece of code by just pressing Alt Control. Control, Alt, Control Alt I. When you put, you can write code inside this section, okay, So you can write the explanations and then you can write your code like we do in now. Google Colab. So that thing we can do in the adult, so you need to go to the File, New and dinner. We were creating our script now and then you can start tracking. So I encourage you to write the code by yourself by following the lecture, because that will give you the hands-on experience. If you just download the file and just run it, you will not get much benefit from the lecture. Okay? So I'll suggest start writing, okay? I'm not writing here. I've already written it. Because if I start typing all these syntaxes, it will take lot of time. And that will not be beneficial because it will take hours of time. If I start typing and explaining things. To save our time for the learning purpose, I want to do it in and I'll explain you. Okay? So first thing is we'll unload the GG plot library. And here we'll say the plot images to a nice taste so I can function and to brilliant dot plot, dot, width and height will be four. Okay? And then reload. So let's run this. So here you can see the two option, three options. One is modified chunk opsin, the next one is run all chunks above. Above these are all the terms will be R1, R2. You can see this option here that is a random current chunk. So we'll use this run junk for now. Okay, so let's run this. So now done, so GG plot library has been loaded. Next thing is the step number two for this project is watering up and drying points on a circle. So in this activity, what we will do, we'll try to draw the circle points. Draw the points around the circle, okay? So mythically will try to draw a circle here with the help of points. So there are many ways to represent data in GG plot two for simple scatter plot, to more complex ones, such as violin plots. The functions that start with geom underscore define how the plot is stone. So geom underscore defined, we'll be showing how the plot will be soon. In this notebook, we want only work with the geom underscore point, which warp plot points in two-dimensions. We just need a dataset with two variables. Let's call them x and y. So we need two variables, x and y for x axis and y axis. Now, I want to draw two points on a circle of radius, one at every x and y point. At every x and y point. B in the unit circle, it follows that x squared plus y squared is equal to one. We can get this using the super famous Pythagorean trigonometric identity, which is that sine squared Theta plus cos squared Theta is equal to one for any real number zero. So these are the basic math that we already know. Alright, so let me open this code. So here what I'm doing, I'm creating a sequence of numbers. Sequence of numbers. So for this, I'll use t as a variable to store these points. And I'll put, use the sequence function to generate. And I'll start with 0.2 into length will be, will be 52. Total point will be generated. Here. X for x point are x axis point. I'll use sine t. Sine of t. Actually this will be the angles, okay? So D is the angle that this sequence regenerate though. The angles that we use for finding our x and y x is sine Theta and cos Theta. So here sine t and then y or y, we'll use the cost, okay? Then you that a DataFrame, data.frame B will be the angle and then x, y x comma y, x will be sine of t and y will be the sine cos of t, sine of t cos up. Then with these two points, will try to make a scatter plot using GG plot to GG plot two. That is this one, this DataFrame, the DataFrame that we have created just now. And then I'll use the aesthetic function to the aesthetics on the plot. The points x comma y. Then I'll use the p plus geom point. So it will draw them blocked. Okay, so let's run this and see, see, and now we're getting plotted on a circle, okay, points plotted on a circle. Next thing is Make it harmonious. Okay, Let's unhide the code. Make it harmonious with Colin Angle. So blondes adding the leaves in this spiral. If you look at the, any plant leaves, it will be enough. Final form, I tried I spinal cord, which starts from the origin and moves away from its point. So when you look at a spiral, it will start with a particular point and it will go like this. Okay? So it will be going like this to start formulating point and then it will go like this. Okay? We're going to invigorate, okay? So it starts from the origin, moves away from this point, adjunct revolves around it. In the plot. Above, all our points are the same. Distance from the origin. Or simple way to arrange them in a spiral is to multiply x and y by a factor routine reaches each point. We could use the Azure AD that factor at it, could use Azure as that vector at it meets this condition. But we will do some thing more harmonious, will use the golden angle. Golden, what is the formula for golden angle? It is phi into three minus under root five. And this number is inspired by the golden ratio. One of the most famous numbers in the history of mathematics. Both the golden ratio and Gordon angle appeared in unexpected places in nature, apart from flower, petals and plant leaves, you will find them in. Seed heads, bind gold, sunflower seeds, sales, spiral galaxies, hurricanes, etcetera. Okay, so it's time to make it finalized. Okay, so let's look at the code. Here. We'll define the number of points you mingle. We'll define as a pie into three minus square root of five. And then we'll be, we'll be the one on wind into anger will multiply the angle into the 0.500 points and then exponent be the sine of t and y will be the cost of T. And then we'll use the DataFrame to create a DataFrame. And then we'll make the scatter plot using GG plot. And we'll add the geom point. Okay, Let's run this, the output. Now you can see how we have created a spiral using the each point. Next thing is, we'll remove everything unnecessary. Apart from data, applaud includes many other component that does. This plot contains a background friend with a gray color grid of horizontal and vertical lines in the background. Tricks along the axis, title on the x-axis, the extra along the axis and levels. So all these things we will be doing by running this code will define the DataFrame. I'm bypassing this side, the x and y to the DataFrame function and make the scatter plot points in his bridle. So the RDD plot. And we'll call this 100 A-frame. And then aesthetic values, x into boo, sorry, x into d, y into d. Okay? And then we'll assign this point. And then we'll use the team fonts on here and inside the theme functional group panel dot grid element fonts. And here and inside the console, panel dot grid elements underscore blank, xis dot txt element underscore blank, and title element underscore blank. They actually don't go grand cow can prep playground. Solid background will be the panel. That background element and underscore wrecked and feel really huge, white. So let's turn and see the output. This is the thing we're getting. If we can this white too. Let's see what is happening. Okay? Now the background is in red. So with this we can create background. And if I run this, the background would be the audience. Okay? So he had clean. So see now how this spiral points and looking like a pattern. Okay? Next thing a bit up, Makeup. Makeup will try to decorate it. Sij color and transparency are drying starts to look like a plant, but we cannot do much better by changing color transparency, also called alpha transparency, darker Alpha size of the image will become more appealing. So let's look at the code. Here. A new GG plot, Wong Sun and the data obtain x enthalpy and entropy. And then Dionne point from Central US and size will give a given color, dark green and theme and the theme, what we have in the previous block. So let's run this and see nowadays column, if I make it supports gray. And see how backgrounds and the sea winds suddenly green but the background becomes gray. Next thing is play with aesthetic. Than Dalian until now, all points of the theme of your insights, color, shape, and alpha. Sometimes you want to make the center point dependent variable in your dataset. Now we'll make size variable. We will also change the theme of points, although we won't be able to blow on it, they don't think investor reminding you of what Dan Dalian. So let me show you the core to the d block geom point in time with aesthetic function sides is equal to, will be the t alpha 0.5, say for me for eight and column I'm giving you a black background where I'm giving white. Let's run this. Now. This is coming like this. Okay? So if I change the CFE or support the sea, but the point where it can't be like this, okay? So whatever you want, you can put the safe here and it will be created in that tape, okay? In that tape, okay. Now we'll put all together what we have done till now and try to create that plot. So first-line is same. Then wind here, aesthetic size is equal to 4.5. And sip reality region 17, and the color green or blue nightgown document. Okay, Let's see. Deploy code. Clean. And see this is looking good. This dark manometer, magenta background. Next thing. What if we modify the angle? So let's see if we modify the angle, what will happen? Drainage patterns are very central to the angle between the points that are on the spinal. Small changes to the angle it can generate very difficult to religion say 1,704.5 and angle we are given two points, We are going periods. So let's now see how beautiful looking. From here, we have come to this realization. Next thing is altogether. Now. And now we'll create the imaginary flower techniques you have seen token allows you to create in finite number of pattern inspired by nature that the only limit is your pattern inspired by nature that the only limit is your imagination. But making art has also been a fun excuse to learn to do GG plot, right? We have learned so many things. And this is pretty interesting. I hope you all like the project. And this lecture. All the texts we have seen in a long year for plotting the real data also. So let's found this image on the left. Some red, it's unreadable of previous flower, is seen something very similar to the first two that, let's turn this, see how beautiful this pattern is looking. So from here, how we have come to the west, we have started with the circle wind, then we have made it little by using the golden angle. Then we have been given the background color, then we have bought lately color, size and transparency, and then the SIP. And then this, and then we have finally come to this and this beautiful flower pattern that we have drawn here. If I change the state to do that, to see what will happen. See this is triangular base. Teach one column. This predicts D come like some other pattern. Come late summer, the bathroom. Okay. I put a 17. It's coming like this. Okay. So I'll make it too. Then. The one is the most beautiful fat done that we have created. So you can also play with different, see, different Alpha angles, different size, different color combination, and try to create your own flower pattern. Flower using mathematics and hot. I hope you like this fun project. And you also are doing the project and upload your Florida in the Florida judge, in the project section of this class. And we all can see 119. Project 11 Analysing and Visualizing the Nobel Prize Winners using R: Hello and welcome back. In this lecture we are going to do and other Analysis and Visualizing essence. And this is also kind of project. And in this, what I'm going to do, I'm going to analyze the data off Nobel Prize Winners. So our Nobel Prize Winners every year. Somebody will get Nobel Prize in a particular category. Which category is related to Art Science, social work on those things, right? So we will try to analyze on a different bitters and we'll solve. We will also visualize the data. So then a few things I have Nika analyzed and I'll show you how you can do that. So most Nobel Prizes, and then we'll focus the Nobel Prize. These are the criteria that we'll discuss. Then. Now, we'll see how he's performing and how or how many Nobel Prizes USS I got dominate the Nobel Prize scenario. Then we'll visualize the USA dominance. Then we'll see the harm and females and males are getting Nobel Prize. And then we'll see who is the first woman to win the Nobel Prize. So all these things and many more are there that we will try to do that. Poor, we proceed. Let me take you through the dataset. So we're using this Nobel dot CSV file, which is having the ear in which the Nobel Prize has been given. The categories, different categories in which Nobel Prize has been given the price name, the Nobel Prize itself. Then Friday will be the Nobel Prize in chemistry, Nobel Prize in literature in physiology. All those things. Okay, So that is the name. The category will be chemistry. Rachel Madison Police predicts all those things. Okay, so there are different categories in which Nobel Prize they give, okay? And then the motivation, what is the motivation behind? Are giving the Nobel Prize to that particular portion and the price they're one-by-one or one-by-two hip, there are more than one people are getting the Nobel Prize in the same. Gashes is somewhere one-by-two. They will be two people. There. They were two people in the same category apiece. So the Nobel Prize will be divided half-half. Okay? So that's tray one-by-two. Then you somewhere you can see one by four. Okay, So this is the price there. So how many people are caught in the same category, same price. Then Laura tidy, everybody who received the Nobel Prize they will get, they will be assigned a lot of it. And then the lottery type, it will be individual or organization. It will be if it did organization getting all pie, it will be organisation. And if it is individual portion, it will be individual. So basically there are two kinds of people who get an individual and organization. Then full name of the portion of the organization, whereas getting the Nobel Prize than the Bar today price than the bar to date. Then the City of that particular portion, the birth country, and then the sex, male, female or whatever. And then the organization name. It, get Parson washed or sale is the organization name than it is. You put it out. Madison excel, done up nice and name and organize and City of majestic country. That date when that person expired and that city and country. So all these columns are there in this novel dot CSV file. We're going to analyze. Okay, so let's start. So let me tell you about fruit things about Nobel Prize. So the Nobel prizes, but have the world's most well-known scientific award, except for the honor, prestige, and substantial prize money. The recipient on Under 33 to 896 who established stock price. Every year. It is given to the scientists and scholars in the categories chemistry, literature, physics, Physiology or Medicine, Economics, and Peace. These are the category in which Nobel Prize Will be given and being given from the last few decades. The post Nobel Prize was handed over in 1,901. So Nobel Prize was started in 1,901. And at that time the Prize was really Eurocentric and male focus, but nowadays it is not biased in any way whatsoever. So this is the story behind the Nobel Prize. And now we're going to find out the Nobel Prize Foundation has dataset available for all Prize Winners. So that is the Nobel dataset available for all Prize Winners. So that is the Nobel notch CSP, 1901-2016. So first thing, we need to load the dataset. So dataset name is novel or CSP. So we'll use read underscore CSP and we'll read this and then we'll come the head will pass. These are dataset to the novel fonts head function. And we'll see how many, what, how that data looks like. For this, we're going to the tidy verse library, so we'll note that library also. When we run this, we can get the clips up dataset. So see you at all. We have already seen in the CSV file here, category price, and then motivation column. Then the price here, Laura tidy Laura type, fullName, birthright, but date, but City, what countries? Sex organization. So all these informations are in there. So after loading, we can access this information through our as well. Next thing, what we do will find who gets done all pray. Just looking at the first couple of reinforced Prize Winners on Nobel laureates, as the also called, we already see a celebrity will have Conrad Rontgen who discovered x-rays. And actually we see all died off. All of the Winners From in nitrogen where guys that came from Europe, since it was started in Europe. So all the amino sweat, which from the, from the US, but that was back in 1,901. Looking at all Winners, the dataset 1,901 to 1966 and rich countries the most commonly represented. That's what we're going to terminate. The country of the winner as a small country is not applicable for all set price on us. Okay? So counting the number of Nobel Prizes handed off 1901-2016 will be quite easy. We'll use the data set Nobel and use the pipe operator and we'll count will use the count function. Okay? And then counting the number of prices, one by the male and female recipients. So we'll use novel and then count based on those six will give us the sex count for male and female separately it will give then counting the number of prizes owned by different nationalities are here will account over the helper bought country. And then we'll add the injured by the descending order. And we'll see the first term T. Okay, so let's run this. See the country. United States on to people nine, United Kingdom, 85, France, Germany, such T1, France for P13, that 29. And this country not a specified 26, Japan Grundy for Canada at, in the lens it usually 17. And then we can see the male and female ratio here. So one lift 49 female till now, till 2,016.8. 39 may have won the Nobel Prize and 26, gender is unknown. Okay? Next thing is, you can see an total number of 911 people got the Nobel Prize. In that it 36, the male and 49 or the female, and 2067 or disclosed their sex. Okay. Us at the pricing, perhaps the most common now alert. You do 1,901 to two times 15 was a man born in United States of America. But in all, in one laughters where European bread did the USA starts. So before when we started the Nobel Prize, all the winners were from Europe, but slowly, you say America took over. They dominated the Nobel Prize charts. So let's see. So here we'll calculate the proportion of US One Winners per decade. So far that will create this prop underscore, underscore with us. And we'll use the dataset Nobel, even the pipe operator and will mutate USA born winner is equal to birth country. And here we'll birth control pride as a United States of America. And they will use er divided by ten into hundred. And we'll use the floor shrunk cell. So attending 200 and we'll use the floor functional group by decade will use to get the, then summarize. Proportion equal to mean US bond will not. And a data which equal to true. So let's run this. See your in 19091900, proportion of us say winning is 0.07 in 19 and then it has increased 2.078. 1920, 0.70, 741,932.25% Data and 40.32, 92.299, 290.14 plus dentists. And in total in front 4321 EGN, you teach 1030. Now. And then we'll use the GG plot to plot the proportion who were saving us. And for this, we'll use the diaphragm and pass it to the DD Plot and aesthetic for aesthetic reviewed, x-axis will not die decade and y-axis will denote the proportion between women. And we lose the geom underscore nine line function to draw the points. Humans could point to draw the points. Scale underscore via continuous levels is equal to Scales. Column, column percentage limit X1, X1 zero, less than this. And see here, now you can see a particle. How it is going. Next is, what is the gender of typical Nobel Prize? Next is, what is the gender of typical Nobel Prize winner? For this, what we'll do, we'll calculate the proportion of female laureates per decade product. We use a prop on descoping and underscore winner from variable here and Nobel dataset. And we'll use the mutate function. Female underscore winner. You call two sexes equal to female. And take it floor will be divided by ten into ten. And we'll close the pipe operator and then we'll go up by decade and category and summarize. By passing the proportion is equal to mean premier winner and the data from two. And then we'll plot this proportion Freeman lauric predicted for this will pass this. D, D drop underscore. Underscore will not do the easy plot from some aesthetic will use X, X squared Y. X is four. Number of winners and Color will be the category based on the category. So let's run this and see. So now you can see here on the x-axis, the cats have been plotted and on the y-axis, chemistry, economics, or the category has been rocked it. And now we'll see how many other repeat or Winners so far that, let me show you that. Here we'll use the novel on Data Frame. And it does it and we'll use count equal to fullname and filter. If N is greater than one, even number of occurrences more than one, then we'll count, count. Understand this. There are at least six people who got Nobel Prize to more than two or two or two or more than two times. So okay. Maddie query time and Lena's gone pollen two times. Committee international law, international community of Red Cross, or three times. Now how old are you when you get the Nobel Prize and the lift up repeat Winners contents in some illustrations, name, we again meet Marty query Madame Curie, who got the Nobel Prize in physics for discovering radiation and chemistry for isolating medium and polonium. John got twice in the fridge export transistors and superconductivity. Frederick Sanger. Sanger got it twice in the chemistry. Linus Pauling got it first in chemistry and lettering piece for each work in promoting this garment, this man and this current moment. And we also learn that organization that also Prices are subordinate cross and you still haven't gotten it price. So let me show you. Use the lubricate. Lubridate. Sorry, not lubricate is Lubridate package here. And here we're calculating the E had the Nobel Prize Winners. So novel and then mutate is equal to minus era. But and then we'll plot the parsing this DataFrame and aesthetic will use Export As and wife for years. And with the hand drop geom point and geometers, small even plot this lesson we're getting. Next thing is a difference between the price categories. So plot ever saw that a lot. We see that people who used to be around four to five day where they sit upright. Nowadays the average is clear to 65, but there is large spread in the theaters in the most appropriate plus someone very young. And you also see that in then City appoints is much high nowadays than in the early 90s nowadays, many high nowadays than in the early 90s nowadays, many more prices are set, so there are many more Winners. We also see that the disruption in our price around second wash, Second World War 1939 to 1943. The Nobel novel underscore age and aesthetic X, Y. And let's turn this. See how different category, how the ages affecting chemistry, economics, law. They try child may receive funds, fees and predicts, medicine, peace and predicts or less than and youngest winner. So we try to find the hole was the oldest and while the youngest. So let here we have used the office as a category by category abdominal price. So you can see the for chemistry and this is for economics, for literature, for medicine and physics. Then are less than Angular's to either plot with lots of cyclins are going on. We see the Boltzmann nursing, chemistry, and medicine have gotten older over time. Trend is strongest while predicts diabetes is yours to be 50. And now the literature and economics or more so Table. And we also see that economics in newer category, but based on a project ten veteran nurse are getting younger. The obese category, we also this category we also were not around 2010, that seemed exerts its nearly young. This begs the question at around two and then that same, except smelly young, this begs the question, Who are the or less than youngest people ever Nobel Prize. So for this, we'll use the novel underscore age variable yet on top underscore N will give one and then will it. And then for top underscore N1, descending way. Okay, so let's run this. See you. In the piece category 2014. Nobel Prize was given to Nobel Prize was given to you. So Malala. Malala is two. J and C was just 17 years old when he was given to see was just seven. Suppression of children and young people for the right of all children to education. And CSIA, the price with them, one. See how Data Partition July 1997 and the age was 17. So C is the youngest one, and here is or list1 in total and seven economics. The savings bank Prize in Economics signs total seven for having laid the foundation upon mechanism, which in theory, and he said the three other group to other people. And her name was Leonardo harvest and he was the all Nobel Prize for now. And at the age of 90 he got Dahmer Nobel Prize and he was from America. And Malala Yousafzai was from Pakistan. This way we haven't liked or less than from Pakistan. So this way we haven't liked or less than youngest Nobel Prize winner. You can offer Nobel Prize winner. You can find some more points and you can analyze that. I hope you got to understand how we can visualize and later. Later. Thank you. 120. Project 12 Finding Password Strength using R: Hello and welcome back. In this lecture, we're going to do another project in which we are going to find the Password Strength using odd. So as we know, quite popular programming language project and this is an odd. Sometimes we need to take the Password Strength also. In that case, you can use this exercise. So let's see. So in this project, we will go through the roles of NIST's special publication 863 be reached. What kicks the verifier secondary party who is responsible for storing and right drank Password lot known as body fat here. It perform to make sure you just don't pick a bad password. We'll go through the passwords of users. What are fixed a little company and use our to flag that you just with bad password. But as being able to do this already means the fixed signal company rolls off 800. I consistently. But if I add cell store memorized, sacred in form, that is ten to Offline attacks. Memorize the Christelle be started and had using Table one way key derivation function. This is never save you just password in plain texts, always encrypt the passwords. Keeping this in mind for the next time we are willing Password Management System. Let's load that data. The list of Password and the fixed or limited database. Both content, real Password link from real website. This Password have not been filtered in any way and not in CloudWatch data exfiltrate data Work Europe. And so this is just for wanting. Okay, so let's load the CSV file here. We're going to use the tidy verse library here. So let me just, okay. So we'll first, we'll load the library, the library, and then we need to read the CSV file where all the you just data has been stored. We are using you just start CSV file. Yeah. And we're storing you just valuable. Counting how many you just rehab got from now we'll use the Endrew function and pass that you just. And then we will print the first pre-print. Let's see the data. So this is the year ID, username and password. So this is the thing file database which is containing the user ID, username, and password. Okay? So there are 982 Raj, okay, unedited two rows or the S. Okay? So now we will check the password should not be. To solve. That is one rule. So if we take a look at the first preprint users, we already steady see some bad password. This password is bad, this password is bad. This password is also back. Almost all the Password in these roles are bad, right? But let's not get ahead of ourselves. Can start flagging Password manually. What is the first thing we sort check according to the NIST's special publication, 800 C3b, when you first cell requests cry. Suzanne, memorize secret to be at least eight characters in length. Okay, So the password of users should not be to sort. So let's start by checking that. So what we'll do, we'll check the length of the Password. First thing. So for that, what I'll do, I'll create a huge length, one variable, and here I'll use the STR underscore length. Check the password length here. I'll check the length part that you just go to the uterus and Dataset and the password and check the what is the length of the Password. Okay. Okay. So and that Alice told in the, you just lend and then I'll create another variable you just to sort, to sort. He put it to sort when it is to start when the you just length this length. The length which we calculated from the Password field. If that is less than eight, we will flag this as to sort, okay? And then we'll print the sum up to source. How many are there? Will sum it up and print. Okay, so let's run this. So see you, there are 376 Password in this database, this Dataset, which is having to sort less than eight characters and see you do sort false. This is more than eight characters, 3368. This is also, this is true. So this is seven characters, six characters. So those Password WeChat less than eight characters, they are flagged deal as a tool to sort for L2. Okay, so now we have pounder way to flag the password we can having less than eight characters. Next check is common passwords. People huge support this. We are going to use the 10 million password list Okay, So we'll check with this list whether the password is from this list or not. Okay. So already this sample rule, flat couple of offenders among the first 12 next day special publication, it an SSD three V's the role that when you fire sell, compare the prospective secrets against the list that contents of Values known to be commonly used, expected or compromised password upfront from the previous breach corpuses. Okay. So these are the part deja, the common Password people use, the leaked. So dictionary words repeated over seconds. We'll characters can access specific words, such as name of the service, username, derivative. Okay, all these, think of it. For common password. We will read the lines of this file, that is 10 million password list dot TXT file. And then we'll see the common password list. Okay, so just in that list. So these are the common Password like 123456 or somebody could Password is equal to Password I 1234530. Okay, So all these superman, all Jada and Jennifer, Jolly robot. So these are the common password that people use. So we sold are wide. They store passwords should not be the common Password. How we check that? So we need to flag all Password in our user database that are among the top, the intelligent. Already you used Password are the common password, okay. For that, I'll use, I'll create a variable called dollar common passwords. So I'll create a field called common password in the user's database. And this will be if the password inside this common password, password is inside this common password. Okay? If you jet Password is inside the common password, it will flag that as a common password and then we'll see how many common passwords are. So let's see. So see your 129 passwords in our database or the common passwords that people have used. See here. These are the common Password March Tom Murphy. Okay, so now we have found common password. Now next thing we'll see how we can take that for a password or not, because passwords are not very common password, right? So far that we will create a variable worse and read lines from the Google 10,000 English texts. So it is worth that Google premium vodka corpus that Google has given us, okay, this file, and then we'll see if people are using common words in their password. Okay? So we'll use the STR underscore to lower function, to lower it. And then we'll check inside this towards whether it is there or not. And then we'll summary tab and find how many people are using common words. Password. So CR1 37 passwords are having common words into them. Okay. Next thing is Password would not be repeated to the Predicting. So how to check that? So first we'll split the worst. We'll use the STR underscore split and we'll split the password, okay, and then picking out the maximum operative repeated characters plot each Password. So I'll create max underscore repeats. And I'll use apply function here and I'll pass this list Password here. And then function, you'd split underscore Password and max is equal to RLE, split Password. And then we'll find the length. And if too many repeat, oops, I'll store in that too many repeats. If it is more than, for, more than or equal to four, equal to More, More than or equal to four. Okay, then let's plug it. So see here. Max repeat to max, repeat one, max degree 31. Okay, so now what we'll do, I'll put all these together. And we went all the basic tastes per bad password. Now we've got bad password. Now we'll see, we'll put everything together. I'll create a valuable bad password and I'll check if it is to sort our common password or common word, or too many repeats are too many repeats. Many reprise I have added two times. Okay? And if, if either of this condition is true, flag that password has a bad password and then we'll see how many bad password we are getting. So let's see, You see, now we're seeing this bad password and here few password has been flagged as true. Bad password, true. So this co, this is the normal, this is the common word, Commonwealth. So these are flagged as a flag Data bad password. This way we can write a code to find the bad password or the Password Strength using odd 121. Introduction to Machine Learning: Hello and welcome back. In this lecture, we are going to learn about machine learning. Now, we have seen how we can use our programming for data visualization and data analysis. Now, from now onwards, we'll learn about machine learning. We'll start with the basics of machine learning. We'll see how machine learning works. What are the types of machine learning, what are the applications of machine learning? Okay, Then we'll see what are the process of doing machine learning, How we apply algorithms, how we create, model, how we evaluate. Okay, let's get started with a introduction to machine learning, unlocking the power of data. Let's start with the machine learning definition. Using data driven world, machines are playing an ever expanding role in making sense of the vast amount of information at our disposal. As you know, in this digital era, we are getting data, turns of data every second, Be it in the aviation industry, be it in social media everywhere, in the businesses, in the e commerce, in the learning, in the pharmacy, in the medical field, everywhere. Every second we're getting a lot of data. Those data are not the normal data that you used to store in Adim tables, but they are the big data. They are not in a very static format. They're in a very dynamic format. Will be images, they will be images. All those things, these dynamic data data are very difficult with a traditional way. Machine learning is a subset of artificial intelligence that has emerged as a transitive force, empowering computers to learn and adapt from data, to perform tasks without explicit programming. Now, since we have a lot of data at our dispersal, if we are not able to get information from the data, it is of no use if you're getting a thousands of feedback from the customer or consumer or our end user. And if you're not working on that, if you don't know how to work on that, then it is of no use, right? We'll not be improving our processes, our products, our services with the emergence of artificial intelligent data science, machine learning, data analysis. What we can do, we can learn from the data, we can explore the data, and we can get the relative good information from the data. And we can adapt according to the data. We can analyze, we can drive decision from the data. In that case, earlier we used to do it by ourselves, but due to machine learning and artificial intelligence, now we can empower the computers to learn from the data and adopt to perform tasks without expedited programming. We even no need to write the program. It does pre retail and we can train the models. And it will give start working for us less understand us less understand machine learning in a little more detail. At its core, machine learning is about creating algorithm that allows computers to learn from data and make decisions or predictions based on tech knowledge. Unlike traditional programming, where human explicitly instruct computers on what to do, machine learning enable systems to improve their performance and autonomously through experience. This is achieved through the following key components. The data algorithms, the model training, evaluation and testing, and then the deployment. These are the things we do in machine learning. First we get the data, then on that data we apply algorithms, then we train and then we evaluate and test the model. And then if model is given the required and results, then we deploy on okay production on what is data. Data is the lifeblood of machine learning. If there is no data, there is no machine learning, there is no AI. Because machine learning, AI data science, everything is all dependent on the data. Lots of data. The data there, it will be beneficial more, It will be efficient machine learning, artificial intelligence, deep learning. All these things are all dependent on the data. Data is the blood of machine learning. If it is not there, machine learning will not be walking, algorithm will not be walking. Algorithms learn patterns and relationships from historical or real time data, which serves as training ground for those systems, algorithms that we learn and apply. They learn from the algorithm pattern and relationship from the data that we use to train machine learning models. So that when the new data comes, they will predict based on the historical data that we have trained the system on. The quality and quantity of data are critical factors in the success of machine learning model. The data which we are getting should be meaningful. If data is not meaningful, data is not enhanced, data is very clear, then machine learning models will not be very vague. First, we have to work on the data. Data should be very clean, precise that machine learning models work effectively. Algorithm algorithm, machine learning algorithms are mathematical engines that process data. Algorithms are actually a mathematics behind. They work on the data and process the data, identify the patterns in the data, and then based on those patterns they do the predictions or decisions. These algorithms can be categorized into various types, including supervised, unsupervised learning, reinforcement learning, each suited for different tasks. Algorithms and like the way they learn from the data, way they learn from the data, they read the data are different types that we will learn while we will be seeing the different types of learning. Okay. What is the model training During the training, Machine learning model is exposed to data and it learns to recognize patterns. Every data will have some pattern. If you look at the learning websites, if they collect the data, they will be coming to know that the user is coming on the website. They're clicking something then based on if they're buying a bread they're buying, but they're buying sugar, they're buying the E, the machine learning algorithms or model. When it reached the data, it reached the data, it will see that pattern. If a person buys a bread, the person is also buying the butter. When buying the bread and butter is also when he's buying the tea, also buying the milk sugar. This is the pattern recognized by the model. And then model adjust its parameter to minimize the error or deviation from the expected outcome. Evaluation and testing. Evaluation and testing. After training, the model is evaluated. When the model is trained, the model will be evaluated using the separate data set. Suppose we have a dataset, we will train our model on that particular dataset. And we'll say that recognize the pattern, like Brad, when person buys Brad, he buys the. But we have to predict what user can buy next. What will be behavior, what product we can sew him. We train on one dataset and then we taste and evaluate the model on another dataset so that we can come to know its performance and generalization ability. Fine tuning and optimization are often required to improve accuracy. After that, we fine tune the model and optimize the model and algorithms to improve the accuracy of the machine learning model deployment. Once model is the reliable, it can be deployed in real world applications to make predictions, automate decisions, and provide recommendations. You would have come across all these things like on Amazon flip card way. When you see when you buy a product, the system will automatically start recommending you new products based on your previous behavior or previous product that you have bought. It will recommend you new when you watch Netflix. If you watch a comedy movie, next time when you go to Netflix. Start recommending you the new comedy movies, right? It has recognized your pattern of watching it on machine learning models, behind the Netflix, Amazon, all these things, they start recommending you the new products, movies, all those things, Okay? What are the applications of machine learning? Machine learning has found applications in a wide range of industries and domains, revolutionizing the way we approach various tasks. Here are some notable jump. We use machine learning in healthcare. Machine learning aids, diagnosis planning, drug discovery, and personalized medicine by analyzing medical data, images, and genomic sequences. In finance, it powers fraud detection system algorithm, algorithmic trading, credit scoring, risk assessment, and help financial institutions make data driven decisions in E commerce. Recommender systems that I have already discussed use machine learning to suggest product, personalized sopping experience, and optimize pricing strategy. Autonomous vehicles, machine learning is integral to self driving cars. Enabling them to perceive their inform, moment, and make decisions and navigate safely through one example. Cars natural language processing, NLP and LP applications include language language translation, sentiment analysis, chat boards, and voice relation, making human computer interaction more seamless and L of create use nowadays. Manufacturing, predictive maintenance and quality control are enhanced through machine learning. Reducing downtime and defects in production processes. Environmental science, machine learning is used for climate modeling, predicting environment trends, and analyzing ecological data. Significance of machine learning machine learning extend beyond its explications. It has potential to accelerate decision making, enhance efficiency, uncover insights, personalize your experience and innovation. Machine learning foster innovation by opening new possibilities in fields of such robotics, virtual reality, and augmented reality, we can uncover the new insights from the data using machine learning. What we conclude with that that machine learning represents a put technological advancement, That is reshaping industries, fostering innovation, and improving decision making processes at data continues to grow in volume and complexity. Machine learning will play an increasingly central role in extracting value and intelligence from this data rich world. As we delve deeper into this field, we will explore its various techniques, algorithms, and applications in greater detail, unlocking the full potential of machine learning. The end of this lecture, we'll see more of machine learning in the coming lectures. Thank you. 122. The Role of Machine Learning: Hello and welcome back. In this lecture we are going to learn about role of machine learning in data science and artificial intelligence, transforming data into insight. We have already learned about the basics of data science and machine learning, but we'll just little understand how we can visualize the data using machine learning and artificial intelligence. And what is the significance of data in these fields. Data science and artificial intelligence, two of the most transformative fields in the modern technological landscape. Within these domains, machine learning plays fatal role in extracting accessible insights, automating processes, and enabling data driving decision making. In this lecture, we will explore the intricate, intricate relationship between machine learning, data science, and AI, and how they collectively contribute to the evolution of technology. Data science is the interdisciplinary field that combines various techniques and processes to extract knowledge sites from structured and unstructured data. Let me give you a little clarity about structured, unstructured data. Structured data is the data which is very structured in a way that we have the predefined things for storing the data, like we do in the icicle database or traditional relational database. Things where we have defined things like there will be name, there will be a, there will be address. There will be the product he bought. The date, it will be defined and in a tabular form. Okay? And it will be a data will be defined. It will be a text or image or whatever it is, right? But when we look at this thing, unstructured data. Unstructured data is very common these days because of the emergence of the social media, the Internet, e commerce, aviation industry. Everywhere, we get a lot of unstructured data. Unstructured data data is not defined. Data is coming in a various ways. Like it, sometimes it comes in a chat form. In the chat, we are sharing the text, we are sharing the images, we are sharing the images. We are selling the files, we are sharing the videos, we are sharing the animations. It could be anything. Data is not defined. Data is structured, unstructured in nature. With the traditional things, it's very difficult the information from the data, store the data, or get the meaningful insights from the data part. With the big data and the data science and AI machine learning, it is very easy to analyze, store, and get insights from those unstructured data that we get from social media, from e commerce, websites, from our business. That is pretty easy nowadays, extract knowledge and insight from the unstructured data that we do with the data science. It encompasses a wide range of activities, including data collection, how we collect the data, data pre processing, how we preprocess the data before working on it, then how we do the analysis on the data. That is data analysis part that we will see and data visualization can visualize the data. The primary goal of data science is to turn raw data into accessible insights and inform strategic business decisions. Every business which has a lot of historical data, if it doesn't learn from historical data, like we are running a clothing business, e commerce website where we sell the cloth, we don't know what we have. The data that some of the board, the products are not selling, some of the products are highest selling. If we're not getting those information from the previous data, then we will not able to recommend the new product, new designs, to the customers. With the help of historical data, we take the decisions and we can make our strategic strategies both robust and more accesible. Okay, the crucial role of machine learning in data science. Machine learning is the subset of AI that focuses on developing algorithms and models that allow computers to learn and make predictions or decisions without being explicitly programmed. Here is how machine learning enhances data science, data analysis, and prediction. Machine learning algorithms can analyze large datasets to discover hidden pattern trends and core relations between the data. This ability is instrumental in making predictions, whether in sales forecasting customer churn predictions, or even the digits diagnosis In medical fields, automated data preprocessing machine learning can automate the data pre processing tasks such as missing values, imputation, outlier detection, feature scaling, saving data scientist significance, time and effort. Feature engineering can assist in feature selection and engineering helping identify the most relevant variables for productive modeling, classification and clustering. Machine learning models like dissenter support vector machines, clustering algorithms are used extensively in data science for tasks such as customer segmentation and image classification recommendation system we have discussed already movie recommender system, product recommender system. These recommendation systems can be very easily improved with the machine learning. Collaborative filtering and content based recommendation systems are driven by machine learning, providing personage recommendations in e commerce and content platforms such as Netflix, natural language, natural language language, natural language processing, NLP techniques. A subset of machine learning enables sentiment analysis, text classification, and chat board development. Enhancing data driving insights from textual data, data visualization. Machine learning can assist generating interactive and informative data visualization, making complex insights more accessible. The synergy of AI and machine learning. While data science primarily focuses on extracting insights from data, AI extend the capabilities of machine learning to perform tasks and typically require human intelligence. Machine learning is driving force behind many AI applications, enabling machine learning to mimic human cognitive functions. Here is how machine learning bridges the gap between data science and AI powered automation. Machine learning algorithms automate various stat, various tasks from image recognition in autonomous vehicles to language translation in chat boards. Making AI applications more efficient and accessible with the help of machine learning in the terms of the supposed autonomous vehicles, when we can train the model, like when it sees the signal, we can feed the signal image, whether it's red or green or yellow. Based on the sign of the signal, the automated car can stop when it sees the red signal. It can move when it is seeing the green signal. All those things we can train with the help of AI personalization. Machine learning enables AI systems to provide personalized experience such as personalized content, content recommendations, tailored marketing campaign and campaigns, and even healthcare treatment plans. Predictive analysis, I driven productive analysis powered by machine learning. Health organization anticipate trains customer behavior, equipment failure, allowing for proactive decision making. We can do the predictive analysis and we can guess on how our trains are going, what customer behavior is they are, and how customers will like the new product. Enhanced decision making, machine learning algorithms assist AI system in making informed decision making by analyzing vast datasets in real time, reducing human is error, natural language processing and understanding and LP models. A subset of machine learning enables AI system to understand and respond to human language driving applications like virtual cat pots. In conclusion, we can say that machine learning is the linchpin that connect data sets and I, transforming raw data into actionable insights. And enabling artificial intelligence system to perform tasks previously thought to be dream of human intelligence. It means that machine learning will be the crucial part of gaining the human intelligence into the machines earlier, the things which were just for the only human can do. Now it is possible with the machine learning, data science and I, our machines can do the work that human can do with the very precise thing. As technology continues to advance, the symbiotic relationship between data science, machine learning and AI will continue to save our digital landscape and drive innovation across various industries. Standing, the role of machine learning is essential for anyone interested in dynamic and ever evolving world of data and artificial intelligence. That is why the relationship between these data science, machine learning, and AI must be understood to gain knowledge in this field. I hope you understood what is the synergy of AI and learning, what is the role of machine learning and data science. We'll meet in the next lecture, and we'll discuss some more things about machine learning. Thank you. 123. Machine Learning Types: Hello and welcome back. In this lecture, we are going to learn machine learning types, types of machine learning. We'll look at it as a very comprehensive overview. Let's get started now. We have the basic understanding of what is machine learning, how it is related to the artificial intelligence and data science, the overall things. Right, let's revise a bit more one more time. Machine learning, a subset of artificial intelligence has transformed the way computers can process information and make decisions. What we understood, right? It's a part of artificial intelligence. If you look at the bigger picture, data science is the big picture. Under that comes the artificial intelligence and under that comes the machine learning. Deep learning, all those things. Okay, Scene learning is a subset of artificial intelligence. It has transformed the computers can perform a human that can predict the results like human and faster than human. That is the transformation machine learning has brought to the table. It's very easy to make decisions for businesses by processing their data. Uh, machine learning algorithm making models and then evaluating the model. And then predicting, and sometimes they predict 100% of a correct information. Okay, machine learning algorithms allow system to learn from the data. Data is the key. In the last few lectures, we have understood how data is the lifeblood of the machine learning, artificial intelligence, and overall ecosystem of the data science. Everything depends on the data. Everything starts with the data data algorithm. Machine learning algorithms allows systems, or computers or machines to learn from data and improve their performance over time. Without explicit programming, You need to program every time. You just need to train the model. And it will start learning from the data and it will improve based on the learning process it has gone through. There are several types of machine learning, each suited to specific task and applications. In this lecture, we'll explore the main categories of machine learning and its characteristics. Okay, let's get started. There are basically seven types of machine learning. The first one is supervised learning, then unsupervised learning, then comes the semi supervised learning, then reinforcement deep learning, then comes the self supervised learning, and then comes the transfer learning. So these are the seven types of machine learning that we're going to learn. First thing, we'll start with the supervised learning. Supervised learning is a learning that we can relate with the class blue room learning where a teacher supervised our learning. And he teaches us with the images, with the video, with the audio, with the various educational aids right, that teach supervised learning. Supervised learning is one of the most common types of machine learning. It involves training model on a labeled dataset where each data point is paired with the correct output target. The model learns to map input data to the correct output by finding patterns and relations. Key characteristics of supervised learning include. Before we see the key characteristics, let me tell you what does it mean? Supervised learning is the most common types of machine learning. In this what we do, suppose we want to train our system to recognize the human faces. In that case, what we will do well, uh, feed the data to the machine learning algorithm or model with images like human. If it is a human phase, we'll tell that this is a human. If it is something else, we'll put this is animal phase, this is a cartoon phase, okay? We will feed so many images and each image will be tag with a name. It will be labeled, okay, If it is a human face, all the human faces will be labeled, will be labeled with human. All the cats, dogs, animals, elephant faces will be tagged according to their specification. Okay, now when we feed the data through the machine learning model, it, it will see, okay, this is human face, this is human face, this is human pace. It will recognize the pattern of a human faces and based on the patterns that it sees in the human faces, it will learn that if this face comes, it is a human face. It also sees the pattern of other images which are not having similar pattern to the To the human faces, it will recognize, okay, this pattern belongs to a cat. This pattern belongs to the elephant. This pattern belong to the monkeys. Like this, it will start learning after the training, when we feed a new image that is not part of that dataset without a label. It will run the patterns and it will match the pattern that it has learned. Whichever pattern matches, it will say that, okay, this is the human face. Or if it is a cat, it will say it is a cat face. Based on the previous training data, recognize the pattern of a new image and it will give you the result, whether it is a human face or not. This is how the supervised learning works. It has key characteristics, that is classification and regression. These are the two very important terms that we should remember and we should be aware of. The first one is classification. Classification in classification task, the model predicts discrete label or categories. For example, classifying e mails as spam or not spam. When you look at the, you can see mail. Whenever an e mail comes, mail will automatically classified as a spam. It will put that into the spam folder. How does it do? It has some pattern feed trained the artificial intelligence or machine learning models for mail that. If the E mail comes from this keyword domains, it will be classified as a spam. If it is not in the contact list of the receiver, it will be marked as a spam. There are many E mails that has been sent from that particular e mail ID in a bulk to the unknown people. Then it will be classified as a spam. This is how the classification task works. Okay, It will be having a specific categories are labeled spam or North spam spam or North spam. Whereas the region, the other characteristics region in region task. The model predicts continuous values. For example, predicting house prices based on the features like square foot and location. Based on the square foot and location, it will predict the house price. When we feed the data, we train, the data will feed for this particular area. This is the premium location rate will be high based on that. Whenever you put a new location with the new dimensions of the house square foot and all it will predict that, okay, this house should have this price. This is the region where the continuous numerical data comes, where we use the regression, where the classification data comes categories, then we use the classification. These are the two. Consider. The next type of machine learning is unsupervised learning. In the unsupervised learning, it will deal with the unlabeled data. Unsupervised learning deals with unlabeled data, where algorithms aims to find pattern structures, groupings, within the data. It is used for tasks such as clustering and dimensionality reduction. Each characteristics of unsupervised learning include clustering, clustering algorithm, what it does, it group the similar data point together based on the features. For example, grouping customer with similar purchase behavior for targeted marketing clustering means suppose you have a learning website. Suppose you have the e learning website where most of the customers are buying the courses and marketing, few of the customers are buying from the technology domain courses. Then what clustering will do, it will cluster all the customers who have bought the marketing related courses to a particular group. It will cluster them together and the technology it will make another cluster, we will have two clusters of customers. One is interested in marketing courses and another interesting technology courses. This is the way, based on that, when the new customer comes with some behavior, it will classify, it will cluster them into the particular group. Then comes the dimensionality reduction. These algorithms reduce the number of features in the data while preserving essential information. Principal component analysis. Pca is a common technique for dimensionality reduction. Suppose you have so many features in our dataset and that will unnecessary data point. It will reduce the number of features available in the dataset and it will focus on the main keywords and features and it will do the learning, okay? This is called Principal Component Analysis, which is the main feature of your dataset. Which is the main feature that is affecting the target, affecting the result that we will find in the Principal Component Analysis, PCA. The next is semi supervised learning. Semi supervised learning combines elements of both supervised and unsupervised learning. It leverages the dataset with a small amount of labeled dataset and a large amount of unlabeled data. This approach is often used when obtaining labeled data is costly and time consuming. Okay, then come the reinforcement learning. Reinforcement learning is about training agents to make sequences of decisions in an environment. To maximize, in an environment, to maximize a cumulative reward. It is commonly used in applications like game playing, robotics, and autonomous systems. Key characteristics of reinforcements, reinforcement learning include agent and environment. The agent interacts with an environment and learns by receiving feedback in the form of rewards or punishment policy. This policy is the strategy or set of rules, that the agent follows to make decisions. The goal is to learn an optional policy that maximizes the long term rewards. Deep learning. Deep Learning is a subset of basal learning that focuses on neural networks with many layers deep neural networks. It has gained significant attention and popularity in recent years due to its exceptional performance in tasks such as image and speech recognition. Key characteristics of deep learning includes neural net, artificial neural networks with multiple layers of interconnected nodes or neurons. Deep convolutional network neural network CNN's. These are the commonly used for image recognition task recurrent neural network RNNs. These are used for sequencial data tasks such as natural language processing and time Sage analysis. We'll see in detail when we see deep learning, self supervised learning. Self supervised learning is a technique where a model learns from data without human provided labels. Instead, it generates labels from the data itself. Supervised learning, it will recognize a pattern. It will label the data as human phase or Caha. All those things it will label by itself, often by creating surrogate task. It has shown promise in various natural language processing and computer processing and computer sion task. Then comes the transfer learning. Transfer learning involves training a model on a task and then using the knowledge gain to improve performance of a related task. It's practical approach when you have limited data for a specific task. Now, machine learning encompasses a diverse range of types and techniques, each with its strength and applications. Understanding the data types of type is essential for choosing the right approach when tackling various real world problems. As machine learning continues to advance its impact on industries such as healthcare, finance, autonomous systems is expected to grow, making it fascinating and dynamic field to explore and master. There is a lot of opportunities in the market for machine learning, engineers, data scientist, AI engineers, data analysts. All those things go to those things and we can learn much faster. You said the next lecture lecture. 124. Machine Learning Workflow: And welcome back. In this lecture, we will learn about machine learning workflow, how we deal with the data, how we start from the data, and how we move to the data insights. Let's get started. Machine learning workflow is a powerful tool to visualize data. Actually, hello and welcome back. In this lecture, we will be learning about machine learning workflow. Machine learning workflow is all about getting the data, processing the data, applying the machine learning models, and getting the insights from the data. That is what machine learning workflow means in sought. In this lecture, we will learn about workflow of machine learning in detail. Let's start in, machine learning is powerful tool. Machine learning is powerful tool that has revolutionized how we handle data and make predictions and decisions that this is what we have learned in previous few lectures. However, harnessing its potential requires a structured approach known as machine learning workflow. In this article or in this lecture, we will explore the key stages of this workflow, highlighting the essential steps involved in the turning raw data into variable valuable insights. The first step is data collection. Data collection is the foundation of machine learning or data science because everything depends on the data journey. Begins with data collection. The step involves gathering the relevant data from various sources such as databases, API's or sensors. These are the data points where we collect the data. A data significantly impact the success of your machine learning project. Making data collection a critical and often time time consuming tasks. As you know that most of the data scientist and data analyst, machine learning engineer, most of the time around 60% of time they work on the data, 40% of the time they work on the code and creating models, applying algorithms, evaluating the model, all those things, okay? But 60% task is all about working on data and making it workable. Then comes up to the data collection. The next step is data preprocessing. Data preprocessing means refining the raw data and making it workable. Raw data is seldom ready for machine loading because if you have collected the data from the sources, there might be many problems in the data. There will be problem in the data, There will be a missing values, there will be duplicate values, there will be the wrong values, there will be missing values. So many impurities will be there in the raw data. It will never be ready. If you work on those data, you'll not get the required result due to information which are lacking and which are wrongly filled in the data. Data processing involves cleaning, transforming, and structuring the data to make it suitable for analysis. Key tasks in this phase include handling missing values, removing outliers, and encoding the categorical variables. The next steps comes the exploratory data analysis. In sort, we call it EDA. This is a very critical stage because in EDA we understand the data. Before diving into the modeling, it is essential to understand your data, understand your data thoroughly. Understanding of the data is very important. Da involves visualizing data, calculating descriptive statistics, and identifying patterns and correlations. This step provides insights that guide feature selection and model building. The next step comes, and it is called feature engineering. This means that crafting the informative features means if some of the things are there and you can base it on those information, you can create a new feature. Feature engineering is the for the feature engineering is the process of selecting, creating, and transforming features, that is, input variables that the machine learning model will use for predictions. It is basically working on the input data. Skilled feature engineering can significantly improve model performance the more improved model you will be getting. The next type is model selection. Choosing the right algorithm. Selecting the appropriate machine learning algorithm depends on the problem type. It is classification or regression. First thing you have to decide what kind of problem you have in your hand. And then you can decide the algorithm. You have to first think of whether it's a classic replication problem or a regression problem based on your data and the characteristics of the data. The common algorithms include decision tree support vector machines, and neural networks. All these algorithms you can decide to use based on your data and based on your classification of the problem. Okay, the next comes after the model selection. The next step is model training. Learning from the data in this space. The selected model is trained using the portion of the data. Suppose you have the thousands rows of the data. What you do data, what do you take? 60, 40% of the data for training and remaining 60% you'll keep it for the testing, the portion of the data we take and we train our model on those data. Then the remaining 60 or 40% data which we have kept for the testing on that data, we will be testing the model. The model learns to recognize pattern relationships and friends present in the training dataset. Then training involves adjusting the models parameter to minimize errors or deviation from the expected outcomes. Based on the training that we give to the selected model, we will fine tune the parameter to minimize the errors or deviation from the expected outcome that is very critical. Then we move to the model evaluation. We'll see what performance or what result we are getting, what we're getting, and then we will evaluate our model. After training, it's crucial to evaluate the model's performance. This is done using separate dataset, that is the validation or the test dataset, to access how well the model generalizes the two unseen data. Common evaluation metrics include accuracy, reason, recall, and F one score. These are the evaluation metrics, accuracy, precision recall, and the F score. The next step is hyperparameter tuning, or optimizing the model parameters. Machine learning models often have hyperparameters that are not learned during training but must be set manually. Hyperparameter tuning involves searching for the best combination of hyperparameters to optimize the performance of the model. Then the final step comes, that is model deployment to the production, production, or the real world. Once the model is deemed reliable and accurate, after testing, it can be deployed in a real world environment to make predictions or automate decisions. This phase may be involved integrating the model into existing systems or applications. Like suppose you have the banking system in place where you decide which candidate to give the loan and which to reject. And you have developed a machine learning model that can give you the decision like fit or not fit for the loan. Your model is giving up to 97, 98, 99% of accuracy. While testing, you can deploy into the production environment and you can put the user data into that. And based on the user data analysis that the training it has gone through while training the model, it can analyze the data, financial criteria, all those things. And it can decide whether we can issue a loan or not. These are the things we can do in the model deployment thing. Then after the deploying the model, the work is not 0, we need to monitor and maintain the model. Maintenance is also very important for long term success because it may be giving the false negative or false positive. All those things we have to closely monitor that our system model is our algorithms are working fine and giving accurate results. Machine learning models require ongoing monitoring and maintenance, data distribution changes over time Because sometimes the model, when we have worked on the model, we have created the model, the dataset or the criteria of support in the banking sector. Time to time, you get the guidelines from the government. Based on those guidelines, the criteria may change. So we have to keep monitoring, uh, and we have to keep looking at the data and the outcomes. Data distributions change over time. Model performance can degrade because the criteria have been changed. It may give the wrong outputs, so we have to monitor it. Regular updates and retraining. When something changes, we have to retrain the model on the new data so that it can work on the new changed criterias. Regular updates and retraining may be necessary to ensure continued accuracy of the model. So now we have understood how the workflow work. Machine learning workflow is a systematic approach to solve real world problems by leveraging the power of data. It transforms raw data into actionable insights and intelligent systems capable of making predictions and decisions. Understanding and mastering this workflow is essential for data scientists, analysts, engineers looking to harness the full potential of Maine to harness the full potential of machine learning in today's data driven world. As technology continues to advance, the machine learning workflow will play an increasing central role in various industries driving innovation and informed decision making. I hope we understood how the machine learning workflow work from data collection to data processing, to exploratory data analysis, to feature engineering, to model selection, to model training, to model evaluation, to hyper parameter tuning, to model deployment, to the monitoring and maintenance. All these steps are very crucial for the successful implementation of a machine learning model. I hope you understood, we'll meet in the next lecture. Thank you. 125. GIGO Principle: Welcome back. In this lecture we will understand the very important concept and principle that is much applicable in machine learning and that is very old concept of garbage in garbage out gig is a principle that is being used in the IQ industry and software for many a time. It is also applicable in machine learning. This will ensure that the data quality is good and that will become the power of data quality. Let's get started. Garbage in, garbage out, go In the realm of machine learning, a fundamental principle holds true, as we all know that garbage, garbage out, is a very fundamental concept. That is, being for many years, garbage, garbage out. This concise phrase encapsulates a profound concept. The quality of input profoundly influences the quality of output produced by machine learning models. What does it mean? It means that the data, the data you put into the machine learning models that result, if you input the quality of cleaned precise data, you'll get the precise result. If you put the data with garbage in, missing values, wrong information, other things, wrong missing values thing, then you will not get the required good output. The outcomes that you will get, the breaks you will get will be very, very much wrong. Because it all depends on data thing you put in the system that result on. You'll get in an age where data reigns. Supreme understanding and appreciating Gig principle is very much essential essence of Gio. Simple, profound idea at its core. The Gig Principle is a reminder that no matter how sophisticated algorithms, how powerful the computing infrastructure or how skilled the data scientists, the value of machine learning output is linked to the quality of data fed into the system. What does it mean? It means that whatever the algorithm, strong algorithm, how much you code, how you make it sophisticated, powerful the computing infrastructure nowadays, and how much qualified data scientists, machine learning engineer, AI engineers. If the data is not good, all these are use because if data is not good, your skilled data scientist, your infrastructure and your algorithms are not going to work properly. Because it all depends on the data. The base should be strong, base should be very much good, then you can grow u. If seed is not good, good quality. A plant cannot grow in the same way. The data is the seed for all machine learning activity. A good data, you'll get the good result. You'll get the good idea out of it. You'll get the good outcomes. You will get the predictions out of it. Data should be good, then algorithm will work properly, the comfort infrastructure will support, and the data scientist will be able to get you the best outcomes and the predictions based on which you can make good business decisions. Quality of data is very much important. Sense of key. Imagine a self driving car navigating through the street. The city on the vehicle relies on sensors and camera to perceive it. Surrounding, as we all know that self driving car driver less cars basically work on the things. How does I work? I work but on the camera, camera, camera eyes of the human for the self driving car. And sensors are the sensory organs for the self driving cars. The sensors, what it received, the signals and camera, what does it sees and what it translate and combined together. And it will create a environment for the cell driving card to move forward. If it sees some red signal, it should stop. If it sees some obstacles upfront, it should stop. Right? That's what the signal and sensors and the cameras do, right? If the A faulty. Suppose you are putting the faulty sensors or the camera ***ses are dirty, or the camera quality is not good. If there is a red signal and the camera quality or the camera ***ses are dirty and it is not visual properly, it will not stop, it will crash into the signal and it may cause a deadly accident, right? So, the faulty sensors and the camera ***ses, if they're dirty, the data it will collect through the sensors and the camera, it will be inaccurate or incomplete, inconsistent, and on that it will take bad decisions. Consequently, the car's decision making process, such as detecting pedestrians, recognizing the traffic signs, become compromised. In this scenario, even the most advanced algorithms cannot solve the situation. It's a classic example of key the garbage in garbage out. If you put garbage, you'll get the garbage. If you put the right sensors in place, you will not put the good cameras. If the system is not getting good input through the camera and sensors, the algorithms and machine learning models will not able to predict anything good. And it will cause self driving car, it will cause accident. Right Industries. Now let's understand how if the garbage data is freed in, into the system, what it will impact on the industries where the AI and machine learning applications will be used in the, in case of health care, in medical diagnosis, the accuracy and integrity of patient data are paramount. That most important thing, mislabeled or incomplete medical history can lead to incorrect diagnosis and treatment recommendation. Case of finance, financial institutions rely on data to make investment decisions and ****** risk. Poor quality of data can lead to financial losses and misinformed strategies. In case of marketing, marketing campaigns hinge to data driven insights to target audience. Effectively, inaccurate customer profiles or outdated information can result in ineffective campaigns and invested resources. You need to get the correct customer profiles and the data into the customer profile. The customer profile data should be very, very much recent so that it can be easily targeted if a person so an interest in buying TV and that interest was captured six months back. And if you are calling that customer, okay, you want to buy, buy a TV TV. Then he may say that I was looking for a TV six months back, now I have already bought a TV. Your marketing campaign will go in, you're to call, the customer will go in. And if you target a customer who has just started searching for the TV and you got the data very recent in a 23 days, he still might be searching for the right TV. And when you call that customer and you assist him in deciding to buy a new TV, he might end up buying a TV from you. That is the way that is the impact of garbage in garbage out. Data should be recent and it should not be outdated, and it should be accurate in terms of e commerce recommendations of commerce. Platforms based on user behavior are only as good as data that fits them. Low quality data can lead to poor product recommendation recommendations and lost cells. In case of natural language processing, NLP applications like chat boards or sentiment analysis, the quality of training data significantly impact the models ability to understand and respond to human language. Mitigating go the role of data quality to mitigate the effect of Gig and ensure the integrity of machine learning learning outcomes and organization must prioritize the data quality and how they can prioritize by following the steps like data collection, rigorous data collection process must be in place, including data validation and cleaning procedures. While collecting the data, you should not collect the data just like that. There should be a process of collecting the data. After the collection of the data, there should be a validation of the data and the cleaning procedures also should be followed. The data preprocessing data collection data should be processed through various techniques such as handling missing values and finding the outliers. Very critical in our data processing. Then the data governance, establishing the data governance framework to maintain data quality throughout its life cycle is very important. Data validation is another very important step to mitigate the gig that is regularly validate and verify accuracy of the data and consistency of the data because that is very important. Like we have discussed about the lost TV sale, that data was not verified, that it has been 67 months that customer has shown the interest. That should be very important. We have to validate your data, how recent it is, how out dated it is, how it is, how consistent all these things should be checked. Then the feedback loops implement mechanism to continually monitor and improve data quality, ongoing relevance of gig as we move forward in the area of big data and machine learning. Like we have already moved in a lot further and now we have to put a lot of emphasis on Go principle. Gig principle remains as relevant as the quality of data will continue to be the linchpin of ensuring the success and reliability of I systems in a world where data is abundant, but not always pristine. Embracing the Guigo principle is essential for harnessing the full potential of machine learning while avoiding costly and consequential errors. Taking care of the gig, you have to be very careful. And if you take care of the gig, you'll avoid costly and const and consequential errors. In conclusion, the Guigo principle serves as a reminder of symbiotic relationship between data quality and efficacy of machine learning systems. By prioritizing data quality organizations can the value of their machine learning investments make informed decisions and stay ahead in the data and landscape. In the world of machine learning, quality data is not just an asset. It's the foundation upon which intelligence systems are built. That is a very important thing to understand. Quality data is not just an asset. Quality data is required. The quality data is not just an set or other requirement. It's the foundation on which we create all these AI systems, machine learning models, deep learning models, recommender systems, all those things that we are going to work on that we're going to use. Everything depends on the data quality. Data is the foundation and the very essential thing for the data science ecosystem. I hope you've got to know how we can mitigate the gig and how we can get the data quality data for our machine learning models, data for our machine learning models. Thank you inside the next lecture. 126. Supervised Learning Algorithms: Hello and welcome back. In this lecture, we are going to learn about supervised learning algorithm. It will be the path to the intelligent predictions through machine learning. We do intelligent predictions, right? Let's get started. We have the basic understanding of supervised learning. But before moving to the algorithms, let's understand what is the fundamental concept of supervised learning. Supervised learning is a fundamental concept in the world of machine learning. It's the rem, where computers learn from labeled data to make predictions. Or it is the starting point of predictions, right? Very basic thing is supervised learning. That is, we will provide the machine learning models with the labeled data and those data machine learning algorithms through like what kind of label the data has. Then based on that, when the new data comes, it will analyze the pattern learned in, while training and then it will take the decision. Okay? And it underpins numerous applications in our daily life. In this lecture, we will have a closer look at supervised learning algorithms. Exploring what they are and how they work, and some of the common examples of supervised learning. What is supervised learning? Supervised learning is the type of machine learning that we have already covered in the previous lecture. Where the algorithm is trained on our labeled data set. In other words, the input data is paired with the correct output or target, the algorgate. The algorithm learns to map input data to the correct output, effectively learning from the examples. This learning process allows the algorithm to make predictions or decisions when presented with a new unseen data. The mechanics of supervised learning. Supervised learning can be broken down into few key components. The first one being data as the data is the foundation of all these data science, AI, machine learning ecosystem data. The label data set serves as a training ground for algorithms. It includes both input features, for example, attributes variables, all those things. And corresponding target values, for example, labels and categories. Okay, data is the starting point. Then comes the model selection, A specific supervised learning algorithm based on the nature of your problem. Common algorithms include decision three, linear region, logistic regression, support vector machines, and neural networks. All these are the machine learning algorithms that we apply on a model model. We will select based on the nature of the problem we are going to solve. Training. During the training phase, the algorithm analyzes the input data, learns to recognize the pattern that is available in the data, the relationship. Also, it will analyze or decision boundaries. And what the boundaries are defined for the decision taking that lead to the correct output. It iteratively adjusts its internal parameter to minimize prediction errors. Then comes the testing and evaluation. After training, the model is evaluated on a separate dataset that is not used during training. So that's why we divide the entire dataset into two part before we proceed to the training that is training dataset and test dataset on training dataset, we test the model on the test dataset. We test it to ****** its ability to generalize to new unseen data. Various performance metrics like accuracy, recision, and recall are used for evaluation model after training. Testing and evaluation is one of the very, very important step in supervised learning. Now, what are the common types of supervised learning algorithms that we use during the supervised learning? When we select learning, when we select data, we select the model, We select the algorithm. What are those algorithms that we use? There are a wide range of supervised learning algorithms, each suited to different types of problems. Each algorithm will work on each and every problem, every problem. We will need a specific approach to solve a problem in daily life. Also, you cannot solve all the problem in the same approach. In the same way in the machine learning, while using the supervised learning, we cannot solve all the problem through one algorithm problem. Each specific problem, we'll need a specific way to tackle it, right? For that matter, there are many algorithms and based on our requirement, we can choose best suited for our problem. And sometimes we might not be sure which algorithm will work. In those case, data scientists would apply to three algorithms. And they will analyze the result and predictions. And whichever is giving the more correct predictions, they will move with that algorithm at trial is the most common way to finalize which type of algorithm we should use in our model. Here are some notable examples, like linear regression, one of the very popular supervised learning algorithm. Then logistic regression, decision tree is very important. Random forest support vector VMs are very important neural networks. And the K nearest neighbors. These are the very popular machine learning, supervised machine learning algorithm that we use in problem solving. Using supervised learning, what is regulation used for predicting continuous values that we have discussed earlier as well? Right? There are two types of values. Continuous values, it means the numerical values and the categorical values, where we have the clearly defined the categories of the data. Linear regulation is used for predicting continuous values. Here. Now we are very clear that whenever we have the continuous values, we need to predict some prices of a stock, price of a house of things where the numbers are involved. We may use the linear regulation. We can think of linear regulation as one of the problem solving algorithm for machine learning using supervised learning. Linear regulation used for predicting continuous values such as house prices, based on features like square foot location. It assumes a linear relationship between the input feature and the target feature. In case of the house price prediction, it is pretty linear relation. Whenever you are going to a premium location, your prices will be high. When we go to the least budget locations, rate will be high. If you're going for a square foot of the construction area, then your house price will be going up square foot, less price. It is linear relationship between the input feature and the target input feature. In this case, in house a prediction it's square foot of the location of the house, right? Whether it's a tire, one city to city. All these are the input feature and input feature. Our prices will go up and down. This is one approach that we can use, the linear regulation for continuous values. The logistic, logistic regationrimarilyedinaryssifbre, problems where we have a very clear specification that there is a yes or no situation where you can say yes or no or you can classify in terms of E mail spam or not spam. There are very specific classification that we want to do. Either we want to categorize that E mail coming from an unknown person as a spam or not spam. This is called binary classification, the two kind of classification that we do, spam or not spam. Binary is very clear that we need to put in to put in one of the categories, okay? In this case, it's a spam or not spam, okay? In case of loan approval, loan approved or not approved, approved or rejected. In the loan application problem, it is binary Classification will be loan approved, loan rejected, approved and reject. That will be the binary classification. Logistic regulation models the probability of an input belong to a specific class. Class is spam. Spam, okay, Decision. The decision. The versatile algorithm. These are very versatile used for both classification and regrationasketil. Linear region and logistic region we are. So that linear region, linear region we can use for the continuous values logistic regression, we can use for the categorical or classification problems. But sometimes linear region and logistic approaches are not educate enough for the problem. We need a better approach. In that case, decision Rich comes into the picture. They are very versatile algorithm and decent can be used for both classification and region tasks. Decision create a tree like structure for decisions and outcomes based on the input feature. Based on input feature, you'll get the tree like structure. Based on that you can predict the output. Random forest, random forest Gimble running method that combines multiple decision tree to improve accuracy and reduce vector machines, support vector machines. Svms are very effective for classification task. Sbm finds the optimal hyper lane that best separates data points belonging to different classes. In this case, we will be classifying the, uh, entire data set into different classes. And SBM will find optimal hyperplane optimal line that will separate the data points belonging to different different classes. Neural network, deep learning, neural networks particularly feed forward neural networks and conventional neural network. Cnns are powerful tools for tasks like image and speech recogationf're going to solve a problem of image and speech recognization. We have to deal with that neural networks in deep neural network. Also we have the two approaches. Two approaches feed forward neural network and convolutional neural network. That is N. Then comes the K nearest neighbors used for classification and regression. Both problems can be tackled by non makes predictions based on the majority class or mean value of the K nearest data point in the training set. It predicts the values based on the majority class or the mean value of the K nearest data point in the training set. Some of the real world examples, applications of the supervised learning in case of medical diagnosis, predicting G outcome based on patient data like cancer detection is one of the example of the medical diagnosis that we using supervised learning. Based on the patient data, we predict that whether it cancer or not, natural language processing, NLP is used for natural language processing. We use sentiment analysis, text classification, and machine translation. This is also a example of supervised learning. Image recognition, identify objects, faces, and anomalies in images and videos, that is, image recognition. Another example of supervised learning, credit scoring, assessing the credit worthiness of individuals based on the financial and personal data. That is also an example of a supervised learning that is being used in the financial sector a lot by accessing the personal financial data and personal data to define his or her credit score based on the credit score, they decide that the person is eligible for home loan, personal loan, or a business loan or not. Then recommendation systems recommending product movies are contained based on the user preference. That is the mostly used supervised learning nowadays. In everywhere we see the recommendation system be the e commerce websites, Amazon, ebay, flip all those. They recommend products based on the recommend system and that internally they use the supervised learning. Then the movies, Netflix, Hot Star, all those things they are using recommendation systems. Then the content based on the fence, like Facebook ads, Youtube, Youtube videos based on your references. These are the real world applications of supervised learning. Now what we can quote from this lecture is that supervised learning algorithms are work hours of machine learning, enabling computing learning, enabling computers to learn from labeled data and make predictions with remarkable accuracy. The application span industries, span industries and domains, making them an essential tool for extracting insights, automating tasks, making data driven decisions in our data rich world. As machine learning continues to advance, supervised learning algorithms will remain a cornerstone of innovation and problem solving in the fields of artificial intelligence and data science. I hope you got to know the machine learning super wide learn algorithms that we use. And we'll see in the next lecture se, in the next. 127. Linear Regression: Hello and welcome back. In this lecture, we are going to learn about linear regression, which is the fundamental of predictive modeling. Let's get started. What is linear regression? Before that, we'll just understand the linear regression is a cornerstone of statistical analysis and predictive modeling, playing a pivotal role in understanding relationship between the variables and making forecasts. In this lecture, we will turn into the world of linear regression. Exploring its principle applications and how it empowers data scientists to lock valuable insights from the data. Linear region is very important for the data scientist or even the machine learning engineers, because with the linear region we can understand and we can predict, we can sign a better productive model. Okay, let's start again understanding linear region. As it says, linear region is statical method used to model the relationship between a dependent variable that is called x, the dependent variable which is often denoted by y, and one more independent variable, typically denoted x. X. Y will be the dependent variable and x will be the independent variable. Okay, Linear arion is the statistical method to define the relationship between the dependent variable x and independent variable Y. What does it mean when we change the x? How the Y is changing that relationship? We try to find this modeling techniques aims to find the best fitting linear regulation that describe the relate variables. With the linear regulation, we try to find the best fit linear equation which can define the relationship between x and y when x is changing how Y is changing that relationship. You try to define by this equation linear equation regulation equation is denoted by y. We want to find the y, how it is changing based on the x. The linear regulation equation is beta 0 plus beta on e into x plus error. Okay, we'll see what are the beta 0 and beta one and what are the epsilon, this is actually error. Now, this equation will give us the y value when we put X beta zero and beta one plus error. Okay? Here is the independent, Y is the dependent, independent Y is the dependent variable or the target variable, Okay? The value which we want to predict based on the X, okay, is the dependent variable or target variable that we call it, right? And then beta zero is the intercept representing the value of Y when X is zero, will in the next lecture, in the next slide, beta one is the slope indicating how much y changes for one unit change of x and epsilon represents the error term. This is the error term accounting for the variability in y that is not explained by linear regulation with x. See here, this is the linear region, this is the regration line. This is the X that is the independent variable on the X, X, this is on the y. X is changing how Y is changing. That is what we want to predict here, That is what we want to define here. This is dependent variable, this is X independent variable on this equation, beta zero plus beta one into x. See here, the beta zero is the independent intercept. This is the intercept where it is intercepting the y Xs. That's what is the beta zero. If you consider this point here, it will be the beta zero, this will be the beta zero. And beta one will be the slope. What is the slope? For the x axis? Okay? How much x, y changes when x1x is changing. When x is changing, how much Y is changing? That is what? Beta one. Okay? Here, this is the actual point and this is the degration line here. Our prediction is this one. For x is called to one, we are getting y here. But actual point in our dataset that lies here, this distance between this predicted value and the actual value is called error term or error, and denoted by this epsilon when we add beta zero plus beta one into x plus. A term, we, we get the regration point, okay? You got it right. This is the regration line and this screen distance is called epsilon, okay? And the rate points are the data point that we have, these data points, we try to find the best fit line, that is the regration line to define the relations between the x and y, okay? Applications of linear region. Linear regression has a wide array of application across various domains. Like in economics, economists use linear regression to analyze the impact of variables like inflation rates, interest rates, and consumer spending on economic indicators such as GDP finance. In finance, linear regression help model stock prices, predict asset returns, and asset risk factors affecting investing. Investment portfolios in medical, medical resources employ in linear regression to predict the patient's outcome based on the factors like age, genetics, and treatment protocol. In marketing, marketers use linear regression to assess the effectiveness of advertising campaigns, analyze customer behave, sales, social sciences. Social scientists apply linear regulation to study phenomena like crime rates, education outcome, and protic trend. In engineering, engineers use linear regulation to model physical properties such as the relationship between the temperature and material strength. Building and evaluating linear regulation model. Building a linear regulation model involves the following steps. Data collection, the first thing is data collection gathered the dataset that includes independent variable y and one more independent variable X, independent variable X and y, we need this is the data collection. Then the data exploration, explore the data, use descriptive statistics and visualizations to gain insight into the relationship between the variables. Then comes the model bending. Use statistical software. Use statistical software or programming language like Python or R to create linear regreationdel software, estimate the value of beta zero and beta one that best fit the data. Model evaluation is the next step where access the model's performance using matrices. Metrics like a coefficient of determination, R squared mean squared error, or Root mean squared error, RMSC. These metrics quantify how well the model fits the data. Interpretation, interpret the coefficient beta zero and beta one To understand the strength and direction of the relationship between the variables prediction, use the trend model to make predictions for the new data points. What are the challenges for this linear region? While linear region is a versatile and powerful tool, it has limitations. It assumes a linear while may not always hold true in real world scenarios. Additionally, it may not capture complex or non relationship. In conclusion, linear regulation is a fundamental techniques in the world of statics and predictive modeling. Its simplicity, interpretability, and wide range of applications make it an essential tool for extracting valuable insights from data. And making informed decisions across diverse field of fields and industries. While it serves the, serves as a stepping stone to move forward, move forward to the advanced modeling techniques. Mastering linear regulation is a fundamental skill for any data scientist or analyst. We understood the basics of linear regression and what are the applications for the linear region, how we build a linear regression model model and what are the challenges has. In the next lecture, we'll try to do the region using our programming, see inside the next lecture. 128. Performing Linear Regression in R: Hello and welcome back. In the previous lecture, we have learned about linear regreation in detail. We have understood the concept in theory. Now we'll do a simple example of linear regression and we'll see how we can implement the linear region using R programming. R provides a wide range of packages and functions for linear regreation analysis. Here I'm giving you a guide for performing step by step linear region using R programming. The very first step should be the data preparation. As data is the base for all the data signs, machine learning, data visualizations, deep learning, whatever related to the data science. The first step is data preparation. Data is very important. For this, we are going to use CSV file that will be containing a very simple dataset. Okay? First step is prepare your tape, is prepare your data, load your data into R. Then it should be very much formatted, it should be having the dependent and independent variables. Okay, here we are using data, CSP that I have loaded. By using re file in R, we all load. That is a function which enables us to read the data, okay, From a CSV file I'm creating a variable here, data. And I'm using the CSV file to read the data from this file, and I'm storing the result in to the data. Okay, Let's run this. Okay? I'll provide the data CSV file to you. You can do download it from the class and you can use it, okay? You need to place the data CSP into your project directory so that it will read it correctly and give the full path to that directory. And it will read, okay. Otherwise, it will give the wrong path. It will throw you the error. Okay, file, not a, just all those errors you will get. Next thing is we'll see what is inside this data. We will use the head function head and we'll pass this object data. And then when we run this, we can see what is there in our dataset is a pretty simple dataset which is having x and y. X is the independent variable and Y is the dependent variable based on x. Y is changing here. Okay, for each x, we'll plot y and we'll try to find a linear region. Regress line will depict, which will represent this relationship between x and y. It will best fit for all these points, points, okay? Based on that, if you give some other x value which is not there in the, it will predict the y value, Okay? Next thing is model building. For this, we are going to use the LM function. L function for linear model. Okay? Lm means linear model. Lm function is there in which we are going to use for the linear regression or creating the linear model. Okay? It is written as L M LM function we used to build a linear regression model. In this example, we will predict the dependent variable y based on the independent variable x. Okay, to build a linear regression model, we'll create a model object here. And then we'll use the L M function and we'll say that X, Y is dependent on x data data will give as data, data object we have created here. Okay? Data is equal to data and Y is dependent on x. Now let's run this. It will create our model. Now I want to see the summary of our model. Okay? For that we can use the summary function and pass this model object here. Let's run this. This is the summary for our model. Here you can see the residual mean, median and the first quartile. Third quartile and maximum values, okay? Maximum value is 24.469 Okay. Then we can see the coefficient estimated standard error and value R. Okay? All the interception, all you can see, these are the things. We'll see how you just look at the things and we'll see how we can determine things based on this met first quartile, third quartile and maximum values letter. Okay, next step is model evaluation. Now you can evaluate the model's performance by examining the summary statics including coefficient value R squared values. Next step is predictions. We'll use the predict function to make a predictions. With your model, you can specify new x value which is not present in the dataset for which we want to. Predict the y value here. We'll create a, a new object here, new underscore data here. I'll use the data frame here, x. I'll provide us 102030 x values. I'm giving here three values. I'm giving 1,020.30 For these three values, I want to predict for ten, what will be the y? For 20, what will, and for 30. Then I'll use the predictions. Okay, I'll use the prediction. I'll store the new values in the predictions. And I'll use the predict model predict function two, the y values based on the x value. For this, I'll use our model that we have created here. Okay, In the alum function we have created this model. I'll use that model to predict the values. I'll pass the model to predict function. And new data is equal to new underscore data. The new underscore data that we are passing X vector. Okay? This vector will pass through this new data. Now this will give us the new predicted values. Okay, now we have the predictions. Let's print the predictions here. Let's do this. Let's you see here now we can see the prediction value 22.8 for 102,035.5 and 48.5 30. Let me put this into the chunk so that we can see the result right here. See here. Now we are getting the three values for y x. When x is ten, we are getting the y value, 22.78 When you look at the data here, there is no ten, right? No 2030. But you see a 38. For 38, it is 56. 30 it is at correct, right? For 20 we're getting 35.65 For ten we're getting 22. Now we'll visualize this by plotting it. We'll use the plot function and we the x and y and we'll give the heading as a linear region and Xlab and Ylab Alta. Try to plot the linear region line by using Able Function. For Able function to plot Alec regress line, we use the model that we have created and we'll specify the color as the regrlineilllottdlt'seeeow'etlegrelinere, our linear regression models. Now when you can predict any value of x here for 30, this will be the value of y for 35. This will be the intercept. When you put a line from here to here and where it is crossing, that will be the predicted value, 50 for all those things. Some of the values are far away from here, from this red line to what will be the distance for this point. That is called error, right? Epsilon that we have seen in our theory part, right? The distance between the point and the regression line, epsilon or error, okay? This is how we can use the linear region to predict value. Now if you take any x values and go through here, you'll find the y value on this regrese, the predicted values. Okay, Now linear region is a powerful tool for data analysis and predicting, predictive modeling. With R, you can easily perform linear, assess the relationship between the variables and make predictions based on your model. In this we have seen the basic information introduction to linear region, but there is much more to explore including handling Monte Colinearity model diagnostics, advanced regression techniques. Linear regression is just one facet of the rich landscape of statistical analysis and machine learning that we can do much more of these things. We'll see a few more examples also for linear regression. I hope you got to know how we can create the linear region model and how we can predict the values. How we can plot a regression line here for predicting the values see inside the next lecture. 129. Predict the height of a person using linear regression: Hello and welcome back. In the previous lecture, we have seen how we can create a linear dig model, okay, to predict Y values based on the X. Okay, Here X was the independent variable and Y was the dependent variable. Now we'll move one step further, a little step towards the son. In this exercise, what we will do, we will try to predict height of person using linear deg. Using our programming of course. Okay, very first thing is when you work on a project, you'll get the real dataset. Here. What I'm doing, I'm not using the real world data. Instead I'm generating a synthetic dataset. Most of the time we get the real time data. But for practice and all, every time, we cannot work on a very large data set, right? In that case, what we do, we create our own dataset, okay, with the random values, and we try to create models and work on it. Okay, later we implement the same thing using the real world data. For this example, I'm going to use synthetic data set with age values ranging 18-65 We will have the age values for the persons from 18 years to 65 years old. Okay? And then what we'll do, we'll create corresponding height values using a linear relationship. Okay, first we'll create a age values, 18-65 For each value we'll try to create a corresponding height support 18 years, person will have the height, 19 years, site like that. Okay. We'll create random corresponding height for the person of a particular age using linear relationship. Then we add some random noise. Every data will never be the perfect, right, some noise in the data. We'll have some irregularity in the data. We'll add some random noise to simulate with the real world dataset. Because in the real world dataset, we'll never have the correct values, we'll have the noise into the dataset. We must have the ambiguity in our dataset. Okay, we'll create that ambiguity using the random noise to simulate the real world data. We create a dataframe called data from the H and height variables. Then we will perform linear grationusingLmfunction, where we predict height H from the data frame. We summarize the region model using summary model. We have done all these in the previous exercise as the same thing we will do here but in a different perspective, right? The summary, we'll use the summary to summarize our regration model. And we'll pass this model which we have created during the Lm function and we'll see the cops and statistics. We predict height for a new H value. Once we create a model, what we'll do, we'll pass a new age value, supposed 30 years or 35 years old, and we'll try to predict the height for age. For that we'll use the predict function. Finally, we'll print and visualize the linear regression, okay? Predict the height and visualize that also regression line. Okay, let's get started with the first thing we will do will generate a synthetic dataset. For that, I'm using set 123. What it will do, it will set the set for the reproducibility, okay? Each time it will produce the same setup data. Okay? Here will take from the range 18 to 65. Here the he will alum 18 to 65. Then for height I will 15150 will make it a basic line minimum. People will have 150 okay. Height, 150 centimeter. Then we'll have the plus 0.5 into H here. We'll use the norm function here. We'll give the length of H, then we'll use the mean zero and standard deviation five. This will generate the synthetic height for our each age. Okay, here we are, passing the H here. Okay, let's run this. Now we have created the synthetic dataset. Now what I'll do, I'll create a dataframe. I'll store into the data data frame is equal to h, and height is equal to height. This height, okay, is equal to h is equal to this. Okay, let's run this as well. Now we'll see the dataset, what we have created. Run this, see here now for 18. The random height. We have generated this for 1920 years. This, okay. Now we have the dataset in our hands. Next thing what we'll do, we'll create a linear regation model for that. I'll use a variable here model and I'll use the LM function to create our linear ation model. Here I'll give height, height as the value which you want to predict are dependent variable and as a independent variable. And data is equal to data. Let's create this model by running this piece of code. Now that is created, let me remove this. Okay. That I have done earlier. Okay, now our model is ready. Now our model is ready. Now, let's see the summary of our model. Here you can see all those values coefficients and standard era value value, all those things, okay? Our squared values. Okay, What I'll do now, we have the model ready with us now. I want to predict the age for a 30 year old person. For that, I'll create a variable new age. And I'll use the data frame dataframe function here. I'll pass a 30 and then predicted height. I want to predict the height for this 30 year old person. I'll use the predict function. On what basis I'll predict. I'll use the model to predict the height. I'll pass this model that we have created here using LM function. Okay? And new data is equal to new age here. I'll pass the new age as 30. I pass this variable which is containing the value. Okay, let's run this. Now what I'll do, I'll try to print the predicted height. We got the predicted height. If you see here, the predicted height is okay. Let me run. This predicted height for a 30 year old person is 165.29 This way we have predicted the 30 year old person's height. If you have the real time data, you can use that data here. Okay? Right. And then you can predict, now let me put some other value like I want to put as 49 years old person data. Now it is 165. Let's see what will be the value for 49, for 49 year old person, the height will be 174.84 8,174.47 Okay. So that is the predicted height. This way we can predict the height. Now let's draw the regulation line here. I have written a piece of code where if you have not installed the GG plot two, you can write this code. What this will do if it is GG plot is required, it will install the package and use the library, okay? And if it's already there, then it's okay, it will use that. Okay. Then here we'll use the G GG plot function. I'll pass the data. X will be the H and y, x will be the height and go point, gomo method LM, color blue for the regressor line and all these headings, you know, right? The minimal that we have seen in the GG plot section. New is data frame equal to 30 and predicted height. I'll use the model. Okay, then we print it. Okay, let's run this. Let's see here. Here also we are getting the height of 30, 30 years old person. And see here we are getting the regression line. Also here, regression predict height based on age. This is the X axis age and this is height for each age you will get a point on the regression line. That will be the predicted height for support 52 years. It will come here, this will be the height, 52 years old, person 62 years. It will come here like that, okay? Will come here like that, okay? This way we can predict here if you want, You can give any and it will predict. I hope you understood the linear regression and see you inside the next leg. 130. Logistic Regression: Hello and welcome back. In this lecture we are going to learn about logistic region. We'll understand what is the mathematics behind this mathematical behind the classification using logistic region. Okay, so let's get started. Logistic regulation is a powerful statistical method used for binary classification tasks such as spam detection. Like we can see the e mail, it uses this classification of e mail, like when you get the e mail, it will classify it as authentic or spam. So all the spam e mails will go to the spam folders, right? So that is one example of logistic regulation or a binary classification. Medical diagnosis, whether it's a or any specific digits or that kind of classification. Also we can do then the bandwidth classification is also the risk assessment while giving the loan or credit card. The banks and financial institutions, they do the credit risk assessment fit, fit, or approve or reject thing, okay, to find whether the lender will be able to repay the money or not. Despite its name, it is not a regression in the traditional sense, but a classification algorithm. Although the name is logistic regression, actually, it is not a regression in traditional sense, but it's a classification algorithm. In this lecture, we'll explore the fundamental of logistic regression, mathematics, underpinnings, and real world applications where all we can use logistic regression, understanding the binary classification. In binary classification, we deal with problems where the goal is to assign observations to one or two classes or categories. For example, classifying e mails as either spam or not spam, predicting whether whether a customer will churn or stay the telecom company determining whether a medical test result is positive or negative. Logistics region help us solve such problems by modeling probability that an observation belongs to a particular class. The things like spam or not spam. Medical test, blood test is a particular disease is positive or not like in the case of covid, it was covid positive or covid negative. That binary classification and logistic regression help us to help us to solve such problems by modeling the probability that an observation belongs to a particular class. Logistic region function is the heart of the logistic region. And the key to logistic regression lies in the logistic function, also known as sigmoid function. This sigmoid function is very important in logistic region, which maps any input to a value 0-1 0.1 in 0-1 it will map, Okay? The logistic region function is defined as p of y equal to one equal to one by one plus e to the power minus beta zero plus beta one into x. You remember this Beta zero plus beta one into x. We use that into the linear equation, but here the formula has a change. It is one, y plus e to the power minus beta zero plus beta one into x p of y. Qual one is the probability that observation belongs to plus one. Here It means that this will give the probability of the observation that it belongs to class on e. This formula will give the probability beta zero is intercept that we already seen in the linear regression. Beta one into x represents the linear combination of input features. Logistic function sept curve starts near zero and gradually increases and symptoically approaches to one. It will go like this sine curve right? Then training a logistic regression model. To train logistic recreation model, we use a dataset with labeled examples. For each example, we calculate the probity of belonging to class one using logistic function. Then we defined a loss function, that is a cross entropy loss. To measure the model's performance, the goal is to find the value of beta zero and beta one that minimizes the loss function. This optimization is typically performed using numerical methods or optimization algorithms like gradient descent. This is another very important term. We will see applications of logistic region. Logistic regression is widely used in various domains like we have already discussed in the medical diagnostic predicting a disease outcome support, for example, diabetes. Based on patient data like test results or medical history, we feed the data and it will give us whether it's a patency diabetes, diabetic or not. Marketing and customer analysis identify potential charnes targeting advertising and customer segmentation. All these things we can use logistic regulation in finance. We have already discussed assessing credit risk, fraud detection and loan defaults, natural language processing, NLP, text classification, for example, a sentiment analysis spam detection is a common application of NLP using logistic region image analysis in an image processing. Logistic regulation can be used for binary classification of task. If you give support images of human and animal based on the certain Ca, it will define whether this picture belong to the human category or animal category. Like that. Interpreting logistic cofficiens, interpreting logistic logistic recreation coefficients, efficient beta zero and beta one in logistic relation have meaningful interpretation. For example, in a credit risk assessment model, beta one might represent the impact of income on the likelihood of default. A positive beta one indicates that higher income reduces the risk to default. What is the conclusion? The logistic cation is volatile and interpretable tool for binary classification problems. We should always remember that this is for the binary classification problem. It's a mathematical foundation and able to estimate probabilities, make it an essential component of predicting modeling in data science and machine learning. While it is fundamental algorithm, its applications are far reaching, contributing to better decision making across industries and domains. As we continue to escape our data driven solutions. Logistic regression remains a valuable asset for tackling real world classification challenges. This is all about a theory, part of logistic relation. In the next lecture, we'll do a simple hands on using R and we'll try to implement a logistic regression. Sees the next lecture. 131. Customer Churn Prediction using Logistic Regression: Hello and welcome. In this lecture we are going to write our program logistic regression. And we'll try to predict whether a customer will churn out or not. So let's see, this is our while the project is predicting customer san. Okay, so here what we do, we create a sample dataset with two variables and on where John is a binary outcome that is zero or one representing whether a customer churned out or not. Okay, so that is what we are trying to do. Okay? We use GLM, GL, GL is general ration model. Okay? We use GLM here, GLM function to fit a logistic ration model, where model on a function of age, the family argument is set to binomial. To specify logistic regression. The argument set to logic indicates logistic link function. Okay, we display a summary of logistic regression model using summary function and we'll pass the logistic model to this summary function to see coefficient values and other statistics. We make predictions for new data. And we'll use data 38.62 and see whether the two a customer with the 38.62 what is the probability by using predict function with the argument type as a response. Okay, this returns predicted probability of John for these two customers. And then we print, okay, let's see here. I'm creating a sample data for band color classification here as well. We are using set seed 123 and data and I'm using data frame and I'm creating these two and churn vectors here. In age I'm giving the H here, I'm giving the John value 01 for the age groups. Okay? Churn is binary outcome as we already discussed. And then let's run this. Okay? Now we have created sample dataset. Okay? Now lex is creating a logistic model for that. We'll use the GLM function and in the GLM function, on what basis will create this thing logistic creation based on the H, okay? Here, data will be the data and family will be the binomial linkage equal to logic. This means that we're going to the logistic regression. Okay, let's run this now. Our logistic regression model is ready. Now what we'll do, we'll pass this logistic model to summary function to see the summary of this regression model. Here you can see the Copic values, intercept errors, jet value, and R values. Okay, here you can see this person parameter for ban, taken to be one internal deviance and residual deviance C values. All these things, things you can see. Next thing I want to now our logistic model is ready. I want to predict whether the customer will churn out or not. I'll pass the two H 38.62 and I'll see the probability for data. I'll create a new data. I'll pass the two vectors to H to, I'll store in the new data. Then I'll pass this new data to the predict function. I'll pass the two things, logistic model we have created and the new data that is here, the pass and type a response. Then we run this. Now we have the predicted values in this predicted props so that we will try to print it. Let's run this. See the probability of churning out the customer H 38 is coming at 0.077 weather, whereas the 6022 year old customer, it is around 97% Here we can easily say that the customer with H 62 will be churning out as 100% 97% is almost 98% 97.75% here, the probability of sending out the customer with 62 is almost 98% Customer with 38 is less less. Okay. Here we come to note that with the two data, the probability is very less 38. If I change to 18 and suppose I'll change to 52. Now I'll run it again and we'll see now the probability for 18 is even reduced. For the 0.004 6% here it is 76% What we conclude from here, the more the probability of customer churning out will be there for 28.72 here. Even the 72 years old, we're getting almost 99% probability that customer with 72 will be churning out for 28. It is almost like zero to get the probability, more probability for less aged customers. The more the age, the more will be the probity of customer churning out. This is how we can use logistic region to predict whether a customer will churn out or not. I hope you understood how to use the GLM function for Los for logistic region see inside the next lecture. 132. KNN algorithm: Hello and welcome back. In this lecture, we are going to learn about k nearest neighbors in sort. We call it the simple effective algorithm for classification and regulation both. Okay, To we have seen region and then we have seen the logistic regulation tag for classification. Now we'll see the non which can be used for both classification and region, both problems. Okay, Let's get started. Nearest neighbor or simple effective algorithm for classification and region problems. In the world of machine learning and data science, the nearest navas or in algorithm stand as one of the simplest, remarkably powerful techniques for solving classification and regression problems. Both when you have the problems which can be of classification and regression, both then we can use in it's an instance based non parametric algorithm that relies on proximity to make predictions. It's all about the proximity. We'll see how it works. In this lecture, we will explore the fundamentals of Canon and how it works, its variations, and the real world applications. As the name suggests, at its core is straightforward algorithm. It makes predictions based on the majority class for classification. And the mean for regression basically takes care of two things. For classification problems, predictions are based on the majority class for the regression problems, it mean as the base, okay, of the k nearest data points of a given query. The k in the k NN represents the number of nearest neighbors considered when making predictions In this nearest neighbor, k represents the number of nearest neighbors considered when making predictions for classification, these are the steps we follow. The first one is data representation. Start with labeled dataset where each data point has a class label, then distance metric. Choose a distance metric, for example, including distance or man pattern distance to measure the similarity between data points in the prediction. The next is prediction. To classify a new data point, calculate the distance between the data point and all other points in the dataset. Select the K nearest neighbors based on the smallest distance majority vote. Determine the class of a new data point by taking majority vote among the classes, its nearest neighbors K For regression data representation. Again begin with the label dataset, but this time labels are continuous values, that is numeric, okay? Distance metrics use the same distance metrics in symmetric, as we have seen in the classification. That is the Ec distance or Manhattan distance, to measure the similarity between the data points. Okay, Then comes the prediction. To predict a new data point value. Calculate the distance between that point and all other points in the dataset. Select the K nearest neighbors based on the smallest distance, then mean value to complete the mean average of the labels of the K nearest neighbors. So all the nearest neighbors will be taken and then mean, those will be calculated. Then this means that this mean value, whatever the mean value will get, becomes the prediction for the new data point. Okay? Variations of, There are three variations of non. The first one is weighted Canon, second one is distance metric, and the third one is Fit Scaling in the weighted Canon assign different weights to the neighbor based on the distance. The distance and closure neighbors may have greater influence on the prediction. Then the second one is distance metrics. Experiment with the various distance metrics to tailor the algorithm to specify data types, data types or domains. Feature scaling normalize or standardize the features to ensure that no single feature dominates the distance calculation. Real world applications of non finds, applications across a wide range of domains, like recommendation systems like Netflix or the Facebook, or even the Amazon and Flip cards. All these e commerce websites, the Ott platforms in the recommendation system, recommending products, movies are contained based on usual behavior or preferences. There also, we can use the image and speech recognition. Then the medical diagnosis. Anomaly detection, detecting the anomalies in the financial transaction network traffic or industrial processes. The natural language processing NLP problems can also be solved. This canon classifying text documents, sentiment analysis, and language translation. Then another real example is environmental monitoring, predicting air quality, weather forecasting, and pollution monitoring. All these problems, we can use the cannon. Okay, the challenges. While Canon is a style algorithm, it has some limitations. It can be computationally expensive, especially with the large data set, as it requires calculating distance of all data points, okay? That will be quite costly to compute, right? Because for each data point we need to calculate the course of dimensionality. As the number of dimensions feature encourages, the effectiveness of non can dim due to the course of dimensionality. Choosing k, selecting appropriate value for K is crucial and can impact the model's performance. Okay, handling imbalanced data on may struggle with the imbalanced dataset where one class significantly outnumbers the other. Okay? All these are the challenges for non Canon is a valuable addition to the machine learning tool box due to its simplicity and versatile. But it has some limitations that we have already understood. It is particularly useful when dealing with a small to moderately sized data set. Small and medium sized dataset can be very, if you have the dataset which is small or moderately medium or not that big, then non can be really helpful for recreation and classification task. By understanding the principle behind on and its various and data, scientists and machine learning practitioners can leverage its power to make accurate predictions and classify data effectively in various applications. This is all about non theory. We'll try to do the hands on as well using our programming. See inside next lecture. 133. Implementing kNN: And welcome back. In the previous lecture, we have seen about algorithm. In this lecture, we are going to implement the non using our programming K nearest neighbor algorithm for classification. Okay? The steps, I'll tell you what are the steps you're going to follow in this example. First thing we load the class package which provides the in function for classification. We use the popular Irish dataset that we have already used in of our lectures in this class. In the beginning, you must be aware of the Iris dataset which is available in R and contains measurement of iris flowers along with their species labels. We split the dataset into training and testing sets, with 70% of the data used for training the dataset. We will split into training and testing sets, where 70% of data will use for the training and 30% of data will be for the testing purpose. Usually in real world example, we take 80% of data for training and 20% for testing, but 70 30 is also okay. Depending on the requirement of your project or your data, you can decide whether you want to take 60% for training, 70% for training, for 80% of training. So that you can decide based on your requirement and the business needs. Here we define the number of neighbors, that is k, which determines how many nearest neighbors to consider when making predictions. Deciding the Uber number of neighbors is very important while doing the nonce determines how many nearest neighbors to consider because it will impact the predictions. Right? Then we perform the nine classification using nine function where we specify the training data, testing, data, class levels, and the value of K. For this example, we'll take k five, nearest neighbor considered will be the five. After that, we evaluate the accuracy of the model by comparing the predicted species to actually species in the testing set. Then we create a confusion metric to assess the model's performance further. Okay. When you run this code, you will get accuracy of the canine classifier and confusion matrix that shows how well the model classified the different species of iris flower in the test set. Now let's get started here. The first thing is if the class is not installed, it will install and then it will. Okay? Let's run this and get the data load. The data data function will use data and we'll pass the dataset name, that is iris. Let's run this as well. Then we will be splitting the dataset into training and test sets. That first thing we'll need to do is set set 123. Okay, that is a pretty essential steps. Then We create a variable or object sample underscore indices. Here we'll take the sample one to enroll iris. And size will take 0.7 That means 70% of the Iris dataset for training train data will be the iris of sample indices. These indices will go into the iris and 70% of the data will become the training data data will be the iris of minus the sample indices. Okay? That's why here we're using the minus sign. Okay? Remaining will be 30% Let's run this as the number of neighbors, that is k here I'm defining equal to five. Okay, let's run this as well. Now we'll perform the nine classification. Here I'll create a variable or object predicted underscore species. Here I'll use the nine function here. Train is to train underscore data one to four. Test to test underscore data, one to four class will be the train underscore data species, okay? Based on the species we are going to classify. Okay? Then qual two k, this will become five. Okay? Now let's run this. And now we will evaluate the accuracy of the model by using some function. Predicted species is equal to test data dollar species divided by N up test data. Okay, let's see, then we will print it accuracy in 200. Okay, See here now we are getting the accuracy for this model is 97.78 That is pretty good. Now we'll print the confusion matrix by using a table function actually equal to test data of species P. Okay, let's create the confusion matrix here, see here. Now the confusion matrix you can see here for Setosa 14 almost, it has been categorized the species correctly. Only one has wrongly classified. Otherwise, everything is classified correctly. This is the way we can implement the non. 134. Decision Tree and Random Forests: Hello and welcome back. In this lecture, we are going to learn about decision trees and random forest. These two are best friends, they work together, and they are just awesome for machine learning. Their applications are used and they are very useful and handy if you're working on a machine learning, especially in the supervised learning. Okay, let's start learning about these two powerful do machine learning. First we'll understand what is dysentery and random forest in the ring of machine learning. Dysentery and random forest are two formidable tools that have revolted data decision making. These algorithms are widely used for classification and regression. Task we have understood what is task right? Classification means where we need to classify in two or more segments, like true or false cred, application, accepted or rejected. It's a threat spam things, right? Regression task. We do it for the continuous, for numerical things, right? These algorithms are widely used for classification and regression task, making them in dispensable for various applications. In this article, we will, in this lecture, we will explore the working principles and real world applications of dissented forest. We'll explore what is the concept behind the degrees and random forest. What are the working principles behind, behind the decision trees and random forest? And what are the real world applications of decision trees and random forest? So let's start with the decision trees. Decision tree is a tree like structure as you can see here, okay? The parent node, then the child node, and then the child node, okay? It will grow like this, right? A decision tree is a tree like structure that is used for decision support. Okay. Here, a simple example. If you are going to buy something, whether you have a confusion, you want to buy, buy, buy. Now decision is two things, by or not by email, spam, spam like that, okay? It breaks down a complex decision making process into sequence of sim. Each node in the tree represents a decisions or a taste on an attribute, each branch represents an outcome, every branch is an outcome. Okay? Here these are the outcomes, right? The decision test on attribute, Each branch represents an outcome and each leaf node represent a class level or a decision, Okay? These are the nodes. These are the leaves, okay? How Desiree buildings, trees are constructed using top down recursive approach called recursive partitioning like you can see here is a type top down approach, right? It's a recursive would keep on repeating itself, right? Decision trees are constructed, are constructed top down recursive approach called recursive partitioning. We are recursively partitioning each node for the nodes. Right algorithm select the best attribute at each step to split the data into subsets. On each steps, the decision three algorithm will select the best attribute at each step To split the data, it will split the data based on the attribute into data homogeneous as possible with respect to the target variable. Keeping the target variable in mind, it will split the data into a subset. It will also make sure that the subset is homogeneous, okay. It will help in achieving the target variable. Attribute selection. Various criteria such as gini, impurity, entropy, and information gain are used to determine the best attribute for splitting. The goal is to minimize impurity or maximize information gain. Third is pruning. Decision tree can become overly complex and wound to overfitting. Pruning involves removing branches that do not contribute significantly to improve accuracy, leading to a simpler or more generalizable branch. Pruning is simple thing, that if you keep on growing a tree, it can grow in any direction. But if you want to keep a tree in a very good safe, we need to cut down the extra branches in the same way in decision trees. Also, if we keep it growing, it will become overly complex and prone to overfitting. Then that's where comes the pruning. Pruning means removing the branches that do not contribute significantly. Some branches may not be useful. Later we'll prune those branches, which will lead to the simpler and more generalizable model applications of dysentery dissenter. In many domains you can find the applications of different scienter in many domains like medicine, diagnosing the disease based on the patient symptoms and medical history in the financial section, credit card scoring, fraud detection, investment making a example of dissenter. In all these domains, we can use the dysentery marketing, customer segmentation, targeting and churn prediction can be done using dysentery. Manufacturing, quality control and production process optimization can be, can be done through series, environmental signs, species classification, and environment monitoring can be done with the help of dysentery. These are the applications of decision Tree now covers the random forest. Random forest is like gimble. While decision trees are powerful, they can be sensitive to small changes in the data. Random forest addresses this limitation by combining multiple decision tree to create a robust and accurate ensemble model. See here, this is one decision tree, but random forest combination of many decision trees. Right? What it does, the decision tree is to this is very small something. It will drastically change its outcomes, process the nodes, and all. Everything will change. If you make some change in dataset, everything will change. To overcome this limitation, random forest addresses this limitation by combining multiple decision trees to create a robust and accurate jumble model. It will create similar decision trees and make a robust random forest thing. There will be many trees, and that's why it is called random forest because many group of trees will make it forest, right. How random forest work? Bootstrap sampling. Random forest starts by creating multiple bootstrap samples, random samples for replacement from the training data. It will start creating the bootstrap samples from the training dataset. Random features selection for each tree in the forest. A random subset of features is considered at each node when deciding the best tree. Building multiple decision trees are built independently with its own bootstrap sample and random feature selection. Then the voting and averaging. For classification tasks, random forest use majority voting among the trees for regrationaskyuseveragingeere classification. It uses the voting for the regreationag averaging to make predictions. What are the advantages of random forest? It will reduce the overfitting, that is the limitation with the sentry. The random forest are less prone to overfitting compared to the individual disses. Improved generalizations and robustness to data feature importance. Random forests provide insights into feature importance aiding in feature selection. Applications of random forest. Random forests are widely used in many domains like image analysis, finance, ecology, marketing and healthcare. In analysis, object recognition, image classification, and feature extraction can be done in finance. Again, the predicting the stock prices, credit risk fraud detection can be done. Ecology, we like species distribution modeling and biodiversity assessments can be done in marketing, customer prediction and recommendation system in health care predicting patient outcome and G diagnosis can be done. Okay, What is the conclusion? These two dynamic sentry and the random forest are dynamic in the world of machine learning. While Gentry provides interabilitysitrandomest enhanced accuracy and robustness, understanding trees algorithm and their application equips data scientists and machine learning practitioners with powerful tools for solving real world problems across diverse domains. As data driven decision making continues to save power world, the versatility and effectiveness of decision trees and random forest ensure their enduring relevance in the field of machine learning. These two are very, very important machine learning algorithm decente and random forest. I hope we covered the base of these two see lecture. 135. Support Vector Machines Algorithm: Hello and welcome back. In this lecture we are going to learn about support vector machine. Okay, Let me okay, ignore this one. Okay? We are going to learn about support vector machine algorithm. Support vector machine, or SPM. In, we call it, is the one of the most popular supervised learning algorithm which is used for classification as well as regression problems. However, primarily it is used for classification problems in machine learning, it can be used for both regression and classification, but primarily we use it for classification only, okay? So most of the time you'll be using for classification problems, SPM can be used for classification. Most of the time, 99% of time will be using one or 2% of 5% Cases will be there where you can use it further, will be there where you can use it further. Regression problems as well, but that is very rare in real world, okay? The goal of SBM algorithm is to create the best line decision boundary that can segregate dimensional space into classes so that we can easily put the new data point in the correct category in the future. This best decision boundary is called hyper lane. It is like creating a hyper le, creating a line. Suppose you are plotting the data points on x and y u space and you are drawing, we have seen in the linear regression, right? We were drawing a line which was dividing the data points in the same way SBM. Also, it will try to find a line which can divide the dataset is okay. That is the goal of the SPM. Diction boundary is called hyper line. Spm chooses the extreme data points or vector that help in creating the hyper line. Extreme cases are called support vectors. Sbm. Support Vector machine algorithm chooses the extreme data points or vector that help in creating the hypo. These extreme cases are called support vector. It will first find the extreme points or vectors and these vectors are called support vectors. For creating the hyperline algorithm is term support vector machine. Okay, finding the support vectors, that's why this algorithm is known as support vector machine. Consider the below diagram in which there are two different categories that are classified using descent boundary or hype See here. Now we have the X one and X two. And here you can see there, this is the maximum margin line. This line is the maximum margin line and this is the maximum margin hyperline positive hyperlink. These are the support vectors. These are the support vectors. These points which we have found. These are the support vectors. Okay? This side, this is negative. Hypolositiveypolineximumrgin will be the hyper lane, maximum margin, hyperlinear, see here. Now, this hyper lane is dividing these two set points, right? Accurate. These points are known as support vectors. Okay, First we find the support vector and with the support vectors we'll try to find the hyperlink. Bm can be understood with the example that we have used in canine classifier. Suppose we see a strange cat that also has some features of dog. Suppose we have a picture of a cat which has some cystic characteristics of the dog. Also, if we want to want a model that can accurately identify whether it's a cat dog, such a model can be created by using a algorithm. We will first train our model with lots of images of cats and dogs so that it can learn about different features of cats and dogs. And then we test it with this strange creature, which is similarity with both cat and dog, but it's actually a cat. As support vector creates a distinct boundary between these two data, cat and dog choose extreme cases, support vectors. It will see the extreme case of cat and dog on the basis of support vectors. It will classify it as a cat. Consider the below diagram. See this past label data. These are the fast able data like this is dog and this is a cat. Here we are creating a model and giving it training that this creature is dog and this creature is a cat. It is reading all the features of these two images and our model is getting training now. Our prediction model will predict that it's a cat. How it will do. Suppose we are giving a image which is very much similar to the cat and both the dog. It has some features of dog. Also what it will do, it will see the extreme features of a cat. It will classify features of a cat. It will classify, it will find the hyper lane it based on the features, extreme features. It will decide whether it's a cat or dog. The extreme it will match with the cat. It will categorize that as if it is not matching with the cat, extreme features. Then it will categorize that as dog with the PM, Even with this strange cat, which is very much similarity with the dog B. With the help of the support vectors and the hyperled accurately that it's a cat, not a dog. Let's understand example, support vector machines, like applications of support vector machine, SBM can be used for image classification. Like we have seen just now, SBM have been used for image classification. Tasks such as handwriting recognition and object detection, like sentiment analysis, spam detection, and document categorization. Bio, poetics, PMs are employed for protein fold recognition, gene classification, and G diagnosis. In finance sector, PMs are used for credit coding, stock market prediction, and for detection in health care segment. They have been used in the DG diagnosis and prognosis based on the medical data, natural language processing and LP SPM play a role in named entity recognition and text categorization. Let's understand what are the types of SPM. Spm can be two types. The first one is linear regression. The nonlinear SPM can be categorized in two types, linear SPM, I said regression, it's SBM, linear SPM, and non linear SPM. Linear SBM is used for linearly separable data. Which means if a data set can be classified into two classes by using single straight line, then such a data is term linearly Separable Data Classifier classifier used here is called linear SPM classifier. If the data is linearly class fight, we are able to classify the data with linearly separable hyper lane, a straight line, then it is termed rather linearly separable data. And the classifier used here will be called a linear SBM classifier. Linear non linear SVM is used for non linearly separated data, which means if the dataset cannot be classified by using a straight line, then such a data is termed as non linear data. Classifier used in this case is known as non linear SPM. We'll see how these two differs. Okay, hyper lane and support vectors. In SPM algorithm, we know what is hyper line which we draw hyper lane. There can be multiple lines or decision boundaries to segregate classes in n dimensional space, but we need to find out the best decision boundary that help to classify the data point. This best boundary is known as the hyper lane of SPM. The dimensions of the hyperplane depend on the features present in the dataset. Which means if there are two features as shown in the image, then hyperline will be straight line. If there are three features, then hyperline will be the two dimensional plane. Okay? We always create a hyperline has a maximum margin, which means that maximum distance between the data points. Okay? Okay. Support vector. The data points are vector data, the closest to the hyperline, which affect the position of the hyperlinea, which are very much close to the hyperline, are going to affect the hyaline position of the hypo line very effectively. These data points are vectors are called support vectors, since these vectors support the hyperleupportvector. Okay, how does SPM work? First we'll see the linear SVM. The working of SPM algorithm can be understood by using an example. Suppose we have a dataset which has two tacks, green and blue. Here we have taken these two color brown and lighter brown. Okay? The data set has two features, X one and t. Want to class classify the pair x the corn in either green or blue. Suppose this is green and blue, okay? The colors, I have written a little wrong here, grey and lighter gray, okay? Okay, suppose this is blue and green, okay? Now, in two D space, just using a straight line, we can easily separate these two classes, right? We can draw a line here. We can draw a line like this. And this two green and blue, we can separate, but there can be multiple lines that can separate each classes. Consider them. We can draw this image, red one, this is also the two. I can draw a line like this. Also I can draw another line also here which can classify this. I can draw another line here also that classify, there can be multiple lines that can separate these classes. Hence, the Pm algorithm helps to find the best line, or did this best boundary is called hyper line here. Now we can have multiple lines, like the aim is to find the best, best line for making. Okay? This region is called hyperled support vectors. These are the blue and green points are the support vectors. Okay? The distance between vectors and the hyperlinelled margin, see here the support, this distance is called margin. Okay? This is called distance between the support vector and the hypo lane is called margin. As to maximize this margin, hypolimrginlledptimhyplaneeowf, try to put a line here, the margin will be less. If you put a line here, the margin will be less here. If we consider this line, the margin from here and from here, it will be maximum. From every point, the margin from the support vectors to the plane will be the maximum. This is the optimal hyperline finding, right, these points distance between the vectors and the hyperline margin. And the goal of SPM is to maximize this margin. The hyper line with maximum margin is called optimal optimal hyper line. This is the optimal hypo llene that we're finding. Now come non linear SPM. If the data is linearly arranged, then we can separate it by using a straight line. For non linear data, we cannot draw a single straight line. Consider the same, the blue points are also here. Also here also, and greener here. Now, we cannot draw a straight line which classify blue and green points into a separate region, right? Some points will be in each of the regions. Straight line classification is not possible here, right? In this case, to separate these data points, we need to add one more dimension. For linear data, we have used two dimensions, x and y. For the non linear data, we will add a third dimension, will be calculated by using this formula, egal to x squared plus q. We will add another dimension here, because in the two dimension, this is not possible to classify, to separate, to draw a line with separate green and blue points, we will add another dimension that will be the jet dimension. And that jet points will be considered by this formula. Jet call to x squared plus y squared. Okay? So for every x and y, there will be a jet point, and that will be calculated by x squared plus y squared. Okay? By adding the third dimension, the sample space will become like this image here. Now x y and jet blue here and green are now. We can see here, we can see that there are two regions right now. Sbm will divide the data set into classes. In the set into classes in the following way. This will be the best hyper lane we can draw, and this will separate the data points, blip lane on this side of the hyper lane, since we are in the space, hence it is looking like a plane parallel to the x axis, right? If we convert it into space with Gal to one, then it will become like this. Okay? If we consider this one is in the space, if we make cal to one, then it will become like this, okay? Hence, we get a circumference of radius one. In the case of non linear data, this way we can find the best hyper lane in a circular thing, right? Circumference of a circumference of radius one. See inside the circle green and outside the circuit blue this way, this circular hyperline we can get for this data. Okay, I hope we understood how SPM works. See inside the next lecture. 136. Understanding Regression Analysis: Hello, and welcome back. So I'm Son, and I'm back with the new update for this class. So in this series of lectures that I'm going to upload now, we will cover the regression analysis in detail. So first thing we'll start from the very basic and these series of lectures will come in coming weeks, and you may see updates very regularly in this class on skill sare. So let's get started. So the first thing is an overview of regression analysis. So what is regression analysis. Regression analysis is a statical technique used to understand the relationship between a dependent variable and one more one or more independent variable. So it's simply analysis of relationship between the dependent and one or more independent variables. So you can understand in such a way that getting a heart attack for a patient is dependent on variable multiple variables here, heart attack is one dependent variable. That is dependent on the one or more independent variables like your age, your lifestyle, your rating habit, and any other things like you have diabetic or you have any genetic problems history of the heart attack in your family. There are multiple independent variables that are going to that are going to affect the chances of getting the heart attack for a paciente. Here, heart attack, getting heart attack is dependent variables and all other health symptoms or your lifestyles or the existing diseases are the independent variables. So regression analysis is a statistical technique used to understand the relationship between these variables, dependent and independent. I hope I made it very clear. Now, you have the understanding how we do it. So it is widely employed in various fields such as economics, finance, social sciences and health care. So healthcare, example, I've already, uh given you. In economics, also in finance, also, you can see if you're investing in some stocks and all, so your profit and loss will be the dependent variables and the market sentiments and all other things, how the news and all affecting the stock prices will be the one or more independent variables. Healthcare to model and predict outcomes based on the input variable. In this we will dwell into the basics of regression analysis and types, jumpsons and applications. Now we'll understand in a mathematical way. What is regression analysis? So at its core, regression analysis aims to estimate the relationship between dependent variables, often denoted y, and one or more independent variables, denoted x one, x two and so on. Okay. The relationship is typically represented by an equation of the form. It's a linear equation, where y is equal to Beta zero plus Beta one into x one plus Beta two into x t plus Beta three into x three, and so on till Beta n into x plus epsilon. Okay. Beta zero is interceptor, where x one, x and y will intercept and Beta one, Beta two and B are the coefficient representing the efforts of the independent variables, how independent variables are going to affect. This is error term, representing the difference between the object and predicted values. So what we are observing and what we are predicting. What is the predicted value and what is the object value? The difference will be the, is that we call the error term. Error term is what we have predicted and what exact output came. So suppose we are predicting that we have a profit of six 50, but we obser that we got the profit of 500. So six 50 -501 50 will be the error term, In the simple if I explain. Okay? So type of regression analysis, simple linear regression that we all know involves a single independent variable and a linear relationship with the dependent variable. So if you look at it will be very simple, graph x, y, and there will be a line going through it, right? And then comes the multiple linear regression deals with multiple independent variables and a linear relationship with the dependent variable. Then comes the polynomial regression fits a curve to the data by introducing polynomial terms of the independent variable. And then last one is the logistic regression very popular used for binary classification problems where the dependent variable is categorical. We'll see how we can achieve all this and how we can understand and how we can use to decode all the logistic ponmial multiple and simple regression. Now, adjmptons of regression analysis. For regression analysis to provide accurate result, certain ajumptons must be made. The first one is linearity. Linearity. The simple relationship between the dependent and independent variables should be linear. So the first exemption is the relationship between the dependent and independent variables should be linear. Then the second As is independence. Observations should be independent of each other. The next one is Homo Sdsticity, the variance of the error term should be constant across all levels of the independent variables. Then comes the normality. The error term should follow a normal distribution. So the error term that we have seen here E should follow the normal distribution. Then comes the multicolinearity. The independent variables should not be highly correlated with each other. So what I'll do in the next lecture, I'll explain what is normality and multicolinearity and homocd in a little bit so that you can understand in a better way. For now, you just need to follow the lecture. So no mult colinearity means, the independent variables would not be highly crelated with each other. Applications of regression analysis. So like I said, economics for costing demand and analyzing the price elasticity, Then in finance, we can use it for predicting the stock prices and evaluating the risk factors. In social sciences, we can use it for studying the impact of interventions, analyzing the survey data. Survey data for the social welfare, that government run all those things we can do with the regress analysis. And then in healthcare, we can predict the patient outcomes assessing the risk factor for a particular dG like we have discussed about the heart. Digs, whatever digs you can predict based on the data. So now comes the conclusion for this lecture, regression analysis is a powerful statistical technique for modeling relationships between variables and making predictions. By understanding its fundamentals, ajumptions and types, analysts like business analysts and data scientists, and resources can effectively utilize regression analysis to gain insights from the data and make informed decisions, in diverse field. So once you have the grasp of the concept of regression analysis, you can use it for any domain to gain insights from the data and to make informed decisions. So this is very powerful tool that we are going to learn. So the next lecture, we'll learn some more things about the regression analysis. So S inside the next lecture. 137. Understanding Linear Regression Model: Hello, and welcome back. So in this lecture, we are going to learn about linear regression model. So linear regression is one of the fundamental technique, used in statistics and machine learning. And this is used to model the relationship between the dependent variable and one or more independent variables. We have learned this in the previous lecture while understanding the regression analysis. So linear regression is one of the regression analysis model. In this, we will delve into the mechanics of linear regression. It's exemption and how to interpret the model parameters and practical consideration. Let's get started. Understanding the linear regression. Linar regression aims to fit a linear relationship between independent variable x and dependent variable, y. The relationship is typically represented by the equation, y is equal to beta zero plus a one into x plus e, where y is dependent variable, x is independent variable. Beta zero is the intercem this is the value of y when x is zero. And Beta one is the slope coefficient, where the change in y for a unit change in x, then is the eratom that we already know representing the discrepancy between objective value and the predicted values. Now it's clear. If I can draw Okay. There is no drawing tool here. Let me check if I can draw here. Okay. Let me open a pin here. So if I can tell you, this is x axis, and this will be the y axis. This is, this is y, and linear regression is finding a regression line like this, where x is equal to y is equal to. This line will be y is equal to, Let me. We'll put it B zero plus B one, x plus. So you can understand like this, y is equal to Beta zero plus p1x plus. Here, where we will understand here, Beta zero is the intercept term where y, the value of y when x is equal to zero. So when you go here, x is zero, and when you intercept from here like this. Here x is zero and y, from here to here, The site B zero or B t zero. Whatever you can. This will be the b zero, and beta one will be the change in y for a unit of change in x slope. So if you see here, as you move from here to here, and then from here, so this will be the slope will be this slope will be the B one. Is the error tom, will be the error term. Like suppose we predict some value here, and we object something here, so the difference will be the value. This is the regression line. So let me tell you this line is the regression line. This line is the regress line. Okay. This is the straight line equation, y is equal to x plus c with that also you can understand. I hope this is clear. The next ajmpton for redia regression for linear regression to provide reliable tal, several aumpton must hold. First one is linearity, independence, sadisticit normality, no perfect ty colinearity. Linearity, we have already understood where between x and y should be linear like this. For when we plot x and y, we should get a straight line. So that is linearity. Independence, the observations should be independent of each other and homo sadicity is the variance of the error term should be constant across all levels of x normality. The error terms should follow a normal distribution, and no perfect mult conearity the independent variables should not be perfectly co related with each other, okay Then interpreting the model parameters. Intercept Beta zero here, the intercept beta zero, beta zero, represent y when x is equal to zero, the meaningful only if it falls within the range of the data. Slope Beta one, the slope that we have observed here, Beta one, indicates the change in y for a unit of change in x. So can see as we move if we can see this x, then this will be the. Let me change the color. For this x, For this x, this will be the y. That is what it is saying, or change in y, for a unit change in x, a positive slope suggests the positive rec negative slope indicate the negative releancefficient of determination r squared measured for the proportion of a variance in the dependent variable that is predictable from the independent variable. Higher R squared value indicates better fit for the model. Practical consideration model evaluation access the goodness goodness of fit using matrices like mean squared MSC, root mean squared RMS and R square value. Outliers and influential point and identifies and handle the outliers that may disproportionately influence the regression result. Colinarity check for the mult coolity coolinarity among independent variables and consider addressing if it Pregent. Cross validation, validate the model's performance using techniques like K fold cross validation to ensure general legibility. Okay. Now, conclusion, the linear regression model provides simple a powerful framework for understanding and modeling the relationship between the variables by adhering to its assumptions, interpreting model parameter effectively and considering practical consideration, analysts and resources can leverage linear regression to gain insights from the data and make informed decisions in various fields. So this is what we should know about linear regression. We'll do the practical also with the core in our programming, but that would be the latter. For now, we'll understand the basics of various regression analysis models like linear regression, then we'll see the multiple regression as well. Se inside the next lecture. 138. Understanding Homescedasticity: Hello, and welcome back. So in this lecture, we are going to learn about mo Cdesticdy. So homo ststicity also known as homo genity of variance is an ajumpton of linear regression that refers to the condition where the variance of the error or residuals is constant across all levels of the independent variables. In simpler terms, it means that the sprad of residuals should remain the same regardless of the value of the independent variables. So what is homodsticy It is the homogeneity of the variance, and it is an Ajmson that we take in linear regression, and it refers to the condition where the errors, that residuals that we gate will remain constant across all the levels of the independent variables. So whatever value you choose for the independent variables, the variance of the errors will or residuals will be constant. I hope you got to know this, right? So what is the importance of homodatity? Homo sdsticity is an important ajumpton in linear regression because violating this ajumption can lead to the biased parameter estimation. So if we just ignore this homocdat, what will happen? We will get the estimation, but that will be the biased on sub parameters, and it will take you to the it will give you the incorrect standard errors and misleading hypothesis test, when heterod the project of the homo sadicity is present, the model may place too much emphasis on certain observations with large residuals and leading to insufficient and unreliable estimates. So when If we ignore this homodatity will move towards the heterodatity, which is opposed to the homocdaty when this herosdsty will present in our model, what we'll have? What we will do, we will put more emphasis on the observation with the large residuals, and it will lead to the efficient estimates are unreliable. Estimates, and that will make our model very inconsistent and not that much useful. So how to identify the homo sdsticity there are two things. First one is residual plot and second one is brisk pagan test. So we'll understand what is residual plot. One common way to diagnose homo sadsticity is by plotting the residuals against the predicted value. So whatever predicted value we'll get we plot the residuals against the predicted values or the independent variables in a residual plot, if the spread of the residuals appears roughly constrict across the range, the predicted values or independent variables, homocdstcity is likely pregent. However, if the spread of the residuals widens or narrows as a predicted values, the change of heterodstcity may pregent. The second technique is brace pagan taste. This is the statistical taste used to formally taste for homosdaity in regression analysis. The null hypothesis of the brace pagan test is that variance of the residuals is constant. While the alternative hypothesis is that the variance of the residuals is not constant heterosdtity. A significant result indicates evidence of herosdstity. Okay. What will happen? What will be the consequences of violating the homosdatcity? There will be the biased estimation. The coefficient estimates may be biased if the spread of the residuals varies systematically with the values of the independent variables. Incorrect standard errors, we may get the standard errors may be under estimated or over estimated leading to the incorrect confidence interval and hypothesis test. In efficient estimates, the efficiency of the parameter estimates may be compromised reducing the power of regression analysis to detect true relationship between the variables. Addressing the homo sadasticity, if the homo sdstcity is violated, several approaches can be considered. Transformation of variables, transforming the dependent or independent variables may help stabilize the variance of the residuals, weighted list square. Using weighted litch square regression where the observations are weighted based on their variance and can account for hero sedaticity. Robust standard errors, standard errors techniques can be employed to adjust standard errors for hero sedates without assuming the specific. Form of the variance structure. So homo sdsticity is crucial exemption in linear regression that ensures the validity and reliability of the model that estimates. Detecting and addressing violations of homosdsticity is essential for producing accurate and trust regression results. So this is what homodacity is and the importance be understood how to identify, there are two ways and then we have seen the consequences of violating the homo sadasticity. And then we have seen how to address the homo sadasticity. There are three ways transformation of variables, weighted shed squares and robust standard errors. Okay? So in the next lecture, we'll see what is normality. So she's inside the next lecture. 139. Understanding Normality: Hello, and welcome back. So in this lecture, we are going to understand what is normality. Normality is an aumpton of linear regression that refers to the condition where residuals or errors of the regression model are normally distributed. So when the errors or the residuals are normally distributed, it is called normality. In other words, it implies that the distribution of the residuals follows a bell curved or bell shape curve centered around zero. With most residuals clustered near the mean and fewer residuals in the tails. What does this actually mean? It means that when the errors in our model or the residuals in our regression model are normally distributed. What does it mean? Normally distributed means the distribution of the residuals follows the bell sip curve where the It is centered. It is centered. All the arrows are centered around the zero with most of the residuals clustered near the mean. So most of the if you look at the bell, if you look at the bell, how does it look? It looks like this right? Okay. Let me use a pen. If you look at a bell, it will be like this, right? So it will be centered around this zero, suppose. And most of the values are towards this mean value, mean value. Okay, so this is what it means. It should be like bells where most of the values are. Across the mean of the values. Well shaped curve centered arm zero with most vegetule clustered near the mean and fewer residuals in the tail. So now, importance of normality. Normality is an important ajumption regression because many statistical taste and estimation methods used in regression analysis rely on the ajumption of normality. When the residuals are normally distributed, it indicates that the errors are random and independent and regression cocenter estimated efficiently. How to identify normality using the residual plot, One common method to assess the normality exemption is by examining a histogram or a Q Q plot, quantile quantile plot of the residuals. In a Q Q plot, if the points fall approximately along the diagonal line, it suggests that the residuals are normally distributed. Alternatively, a histogram of the residual would resemble a b sep car like this. Okay. The next method is sapiro k taste. This is the statistical taste used to formally test the normality of the residuals. The null hypothesis of the sapiro ilk taste is the test that residuals are normally distributed. A significant result indicates evidence against normality. Okay. Consequences of violating the normality while doing the regression analysis is same as what we have seen in the homosedticity, biased estimation, incorrect inference. What is the biased estimation? Copicient estimates may be biased if the residuals are not normally distributed. Incorrect inference means confidence interval and hypothesis test based on t distributions may be invalid if the residuals are not normally distributed. Inaccurate predictions will occur if we violate the normality, prediction intervals may be unreliable if the exemption of normality is violated. So how to address normality if the normality is violated, several approaches can be considered to address it. The first one is data transformation transforming the data. By transforming the dependent and independent variables may help make residuals more normally distributed, common transformation include logarithmic or square root or the boxcox transformation. Robust regression, robust regression techniques such as robust standard errors or estimation can be used to estimate regression coefficient in pregens of the non normal errors. And then the third one is boostraping. Boost strapping is a re sampling technique that can provide robust inference in regression analysis without asooming normality. So what we can conclude normality is an exemption of linear regression that ensures that validity, efficiency, and reliability of the regression model is estimated correctly with no inference. While violations of normality may not always lead to significant problem, it is essentially essential to assess and address deviations from normality to produce accurate and trustworthy regression result. Okay. So I hope you got to know what is normality as well. In the next lecture, we will understand no perfect mult colinearity. So stay inside the next lecture. 140. Understanding No Perfect Multicollinearity: A and welcome back. In this lecture, we will learn about no perfect mult colinearity. So no perfect multicolinearity is an exemption that we take in linear regression that states that there should be no exact linear relationship between the independent variables. Okay. Independent variables are the variables are known as also predictors. Independent variables are the predictors and the value that we predict are called dependent variables. In other words, it implies that the one independent variable should not be perfect linear combination of others. So one independent variable should not be a perfect linear combination of the others. So there should not be any combination of linearity. Importance of no perfect multicolinearityes, multicolinarity occurs when two or more independent variables in a regression models are highly correlated with each other. So two or more independent variables should not be correlated, highly correlated with each other. While multicolinearity itself is not necessarily a problem. Perfect multicolinearity can lead to issues in estimation of regression coefficient and undermine the interpretation of the model. Identifying perfect mult correlation matrix. One common method to detect multicolinearity is by examining the correlation matrix of independent variables. Correlation coficient close to plus one or minus one indicates a strong relationship between two variables. The second method is variance inflation factor VF, VF is a measure of extent to which the variance of the estimated regression coficient is increased due to multicolinearity. A high F F means variance inflation factor, value typically a obtained indicates a problematic level of multicolinearity. What happens if we violate the perfect multicolary co linearity? It will lead to the unreliable coficient estimates. When the perfect multicoli colinearity exists, the regression model cannot estimate unique coficient for each independent variable. As a result, the coefficient estimates become unstable and unreliable. Infinite solutions. With perfect multicolinearity, there are infinitely many combinations of coefficient estimates that perfectly fit the data. Therefore, the regression model cannot uniquely determine the coefficient interpretation issues. The perfect mult coolinarity complicates the interpretation of individual coefficient, estimates as it becomes impossible to isolate the effect of independent variables from the others. How to address the perfect mult colinearity variable selection, remove one or more highly correlated variables from model, prioritize variables that are more theoretically relevant or have stronger association with the dependent variable. Data transformation transform one or more variables to reduce mult coolinarity. For example, creating interaction terms or polynomial terms can help capture complex relationships without introducing perfect coolinarity. Ridge regression. Ridge regression is a regularization technique that can handle mult coolinarity by adding a penalty term to the regression objective function. This penalty term help stabilize coefficient estimates and mitigates the effect of multicolinearity. These are the three very effective techniques that we can use to address the perfect multicolinearity, The one is variable selection. We can remove or add more remove one or more highly correlated variables from the model. Then the data transformation, we can transform one or more variables to reduce the mult coolity and ridge regression is regulation techniques to handle the multicolity by adding a penalty term. Okay. So ensuring no perfect mult coolity is essential for producing reliable and interpretable results in linear regression analysis. Detecting and addressing the mult collarity early in the modeling process can help improve the accuracy and stability of the regression estimates. So now we understood the homo sadisticity, normality, and perfect mult coolinarity. So now with this understanding, we are going to move ahead in our regression analysis, understanding. Okay, so see inside the next lecture. 141. Simple Linear Regression Concepts and Formulation: And welcome back. In this lecture, we will be understanding about simple linear regression, concepts and formulation. So simple linear regression, as we have discussed earlier also is a foundation statistical technique used to model the relationship between two variables, a dependent variable, and also dependent variable is also known as response variable and an independent variable that is called predictor variable. So predicted variable will be predicting a dependent variables. Here, we will deep into the concept of formulation of simple linear regression, exploding the fundamental principles and practical applications of simple linear regression. Concept of simple linear regression, first, we understand. At its code, simple linear regression aims to capture the linear relationship between a single independent variable x and a dependent variable y. The relationship plays through a straight line equation y equal to x plus c plus B t zero. B zero is zero, we have already seen in the previous lecture, is the intercept intercept point or the intercept x and y is the independent variable, y is the dependent variable, and B one, Beta one is a slope coefficient. We have already discussed this here. This is Beta zero, this is B one, this, and this is x. Okay. So now, the intercept term represents the value of y when x is zero that we have already seen and slope coefficient eta one indicate the change in y associated with the one unit change in x. So when you look at here, this eta one, will be changing. Now, formulation of simple linear regression. Estimation estimating the slope copesent Beta one. The slope copicent Beta one is estimated using the least square method, which minimizes the sum of squared difference between the objve and predicted values of y. The formula for estimating Beta one is Beta one is equal to Sigma of x i minus x to y minus y. This is predicted minus object minus predicted, divided by Sigma off, and here I is varying from equal to one two and xi minus x. Whole square. Where is the number of observations, x and y are the individual values of x and y and x and y, these are the sample mean of the x mean and y mean are the mean of the x and y. Estimating the Beta zero. Once the slope coficient Beta one is estimated, the intercept term beta zero can be calculated using the formula, beta zero, bar is equal to y bar minus beta 1 bar, where x bar x bar beta one is slope cofcient and x bar y bar are the sample means of x and y. So this is the formula of formulation of the simple linear regression. Application of simple linear regression, we already know in economics for modeling the relationship between the income and expenditure in finance, predict stock prices based on the historical data, in healthcare, and Ling the relationship between the patient age and medical expenses and in education in take like predicting the student performance based on the study s. So simple linear regression is a powerful statistical tool for modeling and understanding the relations between the two variables by stimulating the slope and intercept coefficient. Analysts can quantify the strength and direction of relationship and make predictions based on the observed data. Understanding the concept and formulation of simple linear regression is essential for conducting accurate and meaningful analysis. In various domains. So this is one step ahead, we have gone and we have understood how the simple linear regression formulation happens. And we have also understood the concept also. So we'll see more about this, uh, simple linear regression in the next lecture. 142. The Least Squares Method Theory Explained: And welcome back. So in this lecture, we will understand what is the least square method. So the least square method is a common approach used in linear regression. To estimate the coefficient of the regreation equation. It aims to find the line that best fits the object data by minimizing the sum of the squared differences between the object and predicted values of the dependent variable. Here is how the list square method works. The first thing is define the regression equation. The regression equation for a simple linear regression model is typically represented as y equal to Beta zero plus Beta one and two x plus or epsilon, where y is dependent variable. We also call it response variable variable, x is independent variable, we call it predictor variable. Beta zero is intercept term, Beta one is slofficient, and Epsilon is the error term, represented by unexplained variation. The second step is, calculate the predicted value using the given regression equation, calculate the predicted value, that is for each observation in the data set. Then predicted value y for observation is given y y y bar plus Beta zero plus Beta one into xi where Beta zero and Beta one are estimated coefficient on through the least square method. Now, the third step will be calculate the residual. The residual of for each observation is the difference between the object value of y and the predicted value, y hat colon i is equal to two, yi hat. To minimize the sum of squared residual. The goal of the list squa method is to minimize the sum of the squared residuals. Mathematically, this can be expraged as this equation. Sigma two n, Sigma F is equal to one to n i squared is equal to Sigma F y minus yi hat square where I will vary from one to n. By minimizing this sum, we obtain the best fitting line that passes through the data points as it reduces the overall discrepancy between the observed and predicted values of y. Estimate the coefficient, the fifth step page. To find the value of Beta zero and Beta one that minimizes the sum of squared residual calculus technique, specially partial derivatives are used. The formulas for estimating the coefficient are given as it, where x bar where x bar and ar sample means of the x and y respectively. After finding the value from here, next stepge interpret the coefficient. Once the coficient Beta zero bar and Beta 1 bar are estimated, they can be interpreted as follow. Beta zero bar is intercept term representing the y of representing the value of y where x is zero and Beta 1 bar is the slope coefficient indicating the change in y associated with one unit change in x. By applying this list square method, we obtain estimates of the coefficient that define the best fitting line for the given data, allowing us to quantify the relationship between the independent and dependent variables in a simple linear regression model. This is how shed square method works, and this is the theory behind list squares method. 143. Example Least Square Method in Linear Regression: Hello, and welcome back. In the previous lecture, we have seen the theory behind the list square method. In this lecture, we will run this simple program that I have written to explain the list squared method in linear regression. Here, what I'm doing, I'm generating a synthetic data. For that, I'm using set dot set 123. This will set the set for reproductibility. Here, I'm taking X as a value 1-200. This is independent variable, So x is independent variable. Dependent variable will be the y, and we know the recreation, sorry, the simple line formula is y is equal to x plus c. So here will be the two of x I'm taking two and two x, two x plus some random noise I'm taking here by using the formula, using the function norm, 100 and mean I'm taking zero and standard divation I'm taking ten. So this will give us the dependent variable with some random noise. Now, What I'll do, I'll fit the linear regression model using list square by third. Here I'll create the linear regression model by using LM functions, I'll create the model, and I'll use the function m, and here, y will be the dependent variable, and x will be the independent variable. I'm creating the linear regression model here. Now, I'll print the linear regression model. Let me run till here. See here now we can see here. Residuals minimum. First quartile median is this the third quartile and max and coefficient we can see here. Residual standard 9.1 on 90 degree of freedom, multiple R squared 0.0 978 adjusted R square 0.0 9763. Now, I'll plot the data and fit and find the fitted line. To plot, I'll use plot function x and I'll keep the heading as a title for the plot is least square linear regression and x x I'll say x and y, x p y, with the ab line, I'll try to find the fitted regression line. I'll use the model here. This linear regression model, and I'll use for the line, I'll use the red color. Now let me run this. See now here, let me do it. See now here, for every x, we're getting y and C I'm getting a linear regression. This is the regression line, the red line, I'm getting here. That is the linear regression line using least squared method. With this, we are getting this regression line here. Here we have used the least square method to find the regression line. See inside the next lecture. 144. Conclusion and Project Work: Hey guys, So we have learned the art programming and we're good to go with our, our programming knowledge. And we can now implement this art programming knowledge in data science and machine learning. Okay, so from now onwards, you are ready to implement and data science and machine learning algorithms using our programming. So our programming is clear. Now you have loaded. Now your next step is to implement data science and machine learning algorithms into our programming and try to explore. So this is the next step. And now you can move further and start analyzing the data using R and implement ML algorithms such as linear regression using R programming. So these are the things you can explore further after completing this class. So next thing is, what is your project for this class? So now you can create your own data, like I have created employee data in CSV file. You can also create your own data into a CSV file for this class, project work. And try to read that CSV file through art. Then try to perform some data analysis on that data, on that CSV file data using art like we have done in our class. And try to create some charts and graphs based on the data like rehab done for our employee data. Similarly, you can think of a problem, think of a data, and you perform all those things. And after you done with that, said, your final analytics report. Like what are the things you will have found from the data? And try to create a report and try to create graphs and charts. And try to put that into the project section of this class so that we all can go through it. Learn from your experiment, learn from your project, and fail constructive feedback with each other. So when you save your project, all other students. And even I can go through that and even I can say that feedback and others also look into it and say the feedback with each other. And with that, we can learn and grow. So start setting and posting your project. That's it for this class. And I hope you enjoyed this class and learned our programming for your future data science and machine learning projects. Bye, bye, Take care.