Data Science in Python Pandas, Scikit-learn,Numpy Matplotlib | Ankit Mistry | Skillshare

Data Science in Python Pandas, Scikit-learn,Numpy Matplotlib

Ankit Mistry, Big data and machine learning engineer

Play Speed
  • 0.5x
  • 1x (Normal)
  • 1.25x
  • 1.5x
  • 2x
61 Lessons (6h 51m)
    • 1. Course Overview

    • 2. Download and Install Anaconda - Python

    • 3. Overview Of Jupyter Notebook

    • 4. Introduction - Python

    • 5. Python - Number, String, Variable

    • 6. Python - List, tuples, Dictionary, Set

    • 7. Python - If/else, Looping

    • 8. Python - Function, Lambda, Map

    • 9. Introduction - Numpy - Numerica Python

    • 10. Numpy array

    • 11. Numpy array operations

    • 12. Indexing, Slicing - Numpy array

    • 13. Introduction - Pandas

    • 14. Pandas - Introduction to Series

    • 15. Pandas - Introduction to Dataframe

    • 16. Dataframe - Index, Multiindex

    • 17. Handling Missing Data - dropna, fillna

    • 18. Grouping data

    • 19. Read, Write .csv, .html, excel file

    • 20. Visualization of data with pandas

    • 21. Introduction - Matplotlib

    • 22. MatplotLib - Basic plotting, Plotting terminology

    • 23. MatplotLib - Subplots

    • 24. Matplotlib - Special plot

    • 25. Introduction - Web Scraping

    • 26. What is Web Scraping

    • 27. Web Scraping Process

    • 28. Search Element by TagName and TagByClass

    • 29. What is Machine Learning

    • 30. How Machine Learning system works

    • 31. Different Types of Machine Learning system- Supervised vs Unsupervised learning

    • 32. Parametric vs Non-parametric machine learning system

    • 33. Machine Learning system design and Scikit learn

    • 34. Plotly - introduction

    • 35. Basic plotting - plotly

    • 36. Plotly - Bar chart

    • 37. Plotly - Bubble chart

    • 38. Plotly - Histogram and Distribution plot

    • 39. Plotly scatter and line chart

    • 40. Reading Plain text file

    • 41. Reading .csv file

    • 42. Reading Excel and .m Matlab file

    • 43. Read Sqlite Database

    • 44. Fetch Data from Remote file

    • 45. Fetch Data from Facebook API

    • 46. Introduction - Data preprocessing

    • 47. Reading Data

    • 48. Handling Missing Data

    • 49. Categorical Data

    • 50. Splitting Data in Training and Testing Set

    • 51. Normalize Data

    • 52. EDA of pima indian diabetes dataset

    • 53. Visualize pima indian diabetes dataset

    • 54. Introduction - Data transformation and Scaling Data

    • 55. Rescale data - Standardize data

    • 56. Normalize Data - Binarize Data

    • 57. Introduction to feature selection

    • 58. Univariate feature selection

    • 59. Recursive feature elimination

    • 60. Principal component analysis

    • 61. Remove feature with low variance


About This Class

Data Scientist: The Sexiest Job of the 21st Century - By Harvard Business review

The average data scientist today earns $130,000 a year  by  glassdoor.

Want to join me for your journey towards becoming Data Scientist, Machine Learning Engineer.

This is first introductory course to get started data analysis, Machine learning and advanced AI algorithm implementation

This course will teach you - All Basic python library required for data analysis process.

  • Python  crash course
  • Numerical Python - Numpy
  • Pandas - data analysis
  • Matplotlib for data visualization

This course is for beginner and some experienced programmer who want to make career in DataScience and  Machine learning, AI.

Prequisite :

  • basic knowledge in python programming
  • High School mathematics

Enroll in this course and take first step in wonderful world of Data.

See you in field.


Ankit Mistry


1. Course Overview: Hello, friends. Welcome to the course on data science with Pytorch saying this, Coz we're going to learn about the basic knowledge of the fight on the basic replacement costs in a fight on. We are going toe cover about the basic concept of the fight on to little more advanced concept as a Chris courts. Then we will move towards our development environment for the 20 we did not. We will see about how to install on a conduct distribution for the fight on and starting towards the Jupiter, not look for the development environment. Then we will see about a craze scores on a new medical pytorch library on Numb. By Then we will go through the data analysis likely find us and then towards the data visualization library. Mac. Broadly, this is one of the really cuisine code's requirement for the machine learning I am on. Keep Mystery a software developer on a big day, dine genius in one of the private investment bank with eight plus years of experience, and I'm going to become a instructor for you in this course. What other pretty cuisine you should have a basic knowledge of the fight on programming and what should have a basic understanding off high school mathematics to move towards data signs? Who is the target audience? As this is a big note level course. Anyone who wants to start the carrier in a data science machine Learning or data Analytics , This is the best place to start. So what are you waiting for? Just and roll and begin your data science journey now. 2. Download and Install Anaconda - Python: Hello, friends. Welcome back in this video, we're going to learn about how to download and install and conduct distribution. So throughout this whole course, we're going toe use by tone as our main programming language. And for the I. D purpose, we are going to use this Jupiter notebook. So in this video we will see that how we can install this anaconda and as a part of this Anaconda distribution, all those hundreds of package will be by default available after you installed. So let's see what is Anaconda. Let's download it first and then install it. So let's go to the and a corn dot dot com So you can see this is the anaconda dot com. They're not going to be upset here. You can just click on what is Anaconda so you can see that Anaconda is one of the most popular pie tone based on data science platform, and it is used by almost 4.5 million peoples around the globe. 150 plus enterprise are using this thing, and it has a pre built thousands off data. Science packages are available, so in discourse, we're going to use lots off vital and data science package available in a pytorch like numb by new medical piped on library, Matt Bluntly briberies, Psychic Land Machine Learning Library or a lot of other visit logician, Library, Lakers, C'mon, library and other neurological based library we're going to use. So rather than installing all those stuff individually, this anaconda comes bitter. Hundreds of different packages. So let's see how we can download and install. So just click on this download so you can see that and organize available in for all three or major operating system. RINOs Mac and Alina X installation process for the Lennox, Mac and Windows all are quite similar and quite state forward. That is, very has a French traditions are available. And for all these three measure operating system Windows Mac and Alina X, that is a by turn, 3.6 base and a fight on 2.7 based wasn't available. Even for two different operating system 32 bit and 64 bit operating system. I would recommend you to download this 3.6 base by 20% based on your operating system. Just select this 64 bit order 30 Toby for the Mac also select the similar V. Either you can go with even graphical busy Staller for the Mac or even come online basis, taller and for the lean. It's also we have. Three different distributions are available. One is for 80 x 86 tolerate, and another one is a hearty debate installer. So, depending on the configuration off your operating system, Lennox Windows or a Mac, whether it's a started a bit or 64 bit and a fight on 3.6 words and so I'm using here windows so I will download this 64 big graphical installer on my machine. So if you just click on it, it will just take you to die chilled. Download pitch, and it will download almost find it and be around the full package installer for the anaconda. It is Anaconda 3.5, Windows Access 86 64 big operating system I have already downloaded on my local machine, so I'm not going to download it again, so just download it once. If you download it, just double click on that particular package and straightaway windows installation process , and it will install IQ. So I have already installed this Anaconda package Now. Next, let's go to the command line. Prompt. Now, if you were already installed this anaconda package, you just type this conduct, whether particular and when the package is available on your machines or not. Okay, so you can see that it and conduct among it has given lot off suggestion like mints and aggrandizement install on my machine. Now, as a part of this anaconda distribution large so far, there by pound packages are also install. So among them, we're going to use one off them is a Jupiter, not book. Now what is Jupiter not book. So let's go to the G, Peter. So Jupiter, not book is one of the replication and throw this replication as apart off the notebook replication begins light our piped on programming all those court directly into a browser and it will compile and execute. And whatever the output games directly, we can observe inside Web browser It just support another 40 different languages also. So that is the power of Jupiter notebook. And in this video of year, going toe, use how to start this Jupiter notebook So as were installed this anaconda Just go to the command prompt. And just like it's your picture notebook. Once you type this Jupiter not book, it will automatically start this Jupiter, not book server. And it will take you to the particular pace like a local host. And it will start on the port 8888 Now I'm going to create one more video which will explain you in a more genic way. How to use thesis Jupiter, not book. So let's just create a very simple Jupiter notebook by Turn three file. He just said he named me to the days Jupiter, not book. I already created taste here, so I'll just make it taste. 123 Okay, let's a sprinter. Very simple, common Laker helo. Okay, so it has printed hello that miss by Tony's install another more machine Jupiter, not boobies already install on my machine successfully throughout this whole courts for all programming assignment and all those programming exercise we are going to use this by tone is a programming environment and a Jupiter notebook as a Heidi. So that's all about this video plants. I hope you enjoy the sending this video and see you in the next video 3. Overview Of Jupyter Notebook: Hello, friends. Welcome back. So in this video, we're going to see about the very basic overview. Opta Jupiter, not book. So let's get started with starting Jupiter notebook Server to start the Jupiter, not book first start the concert Safe urine of indoors, missy, Just place moving goes button and search for CMD and it will start the console. Jupiter, not book now. Keep wanting in the mine. Whenever you type this command Jupiter not book. You are currently in a directory C Guerrero user pc. So inside this directly only it will start this Jupiter. Not so. Whatever Jupiter notebook and other file folders are available, it will be reasonable from this spot. If you want to start with some other directory, just change the part off that particular directly and start this Jupiter not book this place. Okay, so we have successfully started this Jupiter notebook so you can see here the directly through which video started this Jupiter not book. See drug user pc inside that Whatever files and folders are available, it has just listed here. If you start from some other directly, it will start from there. Now let's create our first very simple Jupiter notebook. So from the new by Taunt three Okay, so it has created a brand new Jupiter notebook for us. Let's rename it to some meaningful name like my first Jupiter. Not book severe successfully created in Jupiter, Not book. Now we can just start writing all those by tone chord inside this cell. So this is called a cell. Let's write a very simple one like a parent. Hello? We know to execute distinct we can just place it here, which will actually run the selector, excel. Or there is another option. Also, Altera enter or a CIF tender. So we have successfully run this first of all, Jupiter sell. So that is a two way one. You can place it here, or you can just type Laker sift enter our park and now how to say this thing. So you just praise this control us and it will be sealed. And there is a periodic saving are also available after every two minutes. US some periodic interval. It keeps saving your Jupiter notebook fight from the file menu again. You can just praise this sale and checkpoint and it will save the file. Now let's save you how created this full Jupiter, not book, and you want to make a executive Lee get copy. You can press it here like I make a copy. You want to rename it? You can see. Name it here. Let's say all those Jupiter, not book. You want to download it in tow? Some other form it something like a notebook for mate or not be. Why not go for make is by default. It is available. You want to convert it into pie turn dot your I file or not. It's team if I so you can just straight of it use. So let's say we want to create a new cell so there is a one. Button is available here. Insert sell below so it will create a brand new cell. Phone us. Now, If you see this individual cell, there is a two types of cells are available. One is a poor type, and when is the mark out? So in case of core type, you can just write like a by turns, according or any of the programming language quoting, or if you just convert it into markdown cell, it is just the plain tax, and if you just executed which sift enter are all tender. It is just a very straightforward, simple tax. You can just use all those markdown related syntax also, so just place hash. So it is the first level actually holding. It has applied on our tax by Tonko D. So it's so actually some marked down. So let's just change it to the markdown. Okay, so we have seen that visit two types of. So let's say we want to delete one cell so you can just type it here like a so meticulous electorate. Say you can even do this copy paste. You can learn it from here if you want to. Let's see intra Particular says. So they say some cells are keep running and it's going in tow. Some kind off in fine it processes. So you've been just interrupted. Let's say in between you want toe restart this full colonel so you can just press it here, leg a restart and run it all again, or you want to reconnect it full. Jupiter not, let's say if you don't get all those things which I house explain you in this video, you can just use this help and that is a lot of help are available. Mark down related help, not book related help. Simple fight on basic function Numb by library site by library, all those helps are available. That is, a one 10 year like a keyboard. Shortcuts are also so corresponding toe every extending Jupiter, not book finding the place. If you want to insert, sell what you can do. Insert self about below heading all those religious shortcuts are available. So that's all about the very basic introduction off the Jupiter, Not book. I hope you enjoy the sending this video and see you into next with you. 4. Introduction - Python: Hello, friends. Welcome to the section on a bite on craze courts. So in this Chris course, we will go through the different topics like data types in a bite on like numbers string William, Another data structure in a bite on like a list dictionary state A structure. The poise sets really won't go to the looping mechanism in a bite on the loops, like a four loop or a vile loop will go to their decision making statement. If hell safe, we'll see how to create a function in a bite on how to create a Lamela function in a beytin . And how do you leverage the one of the most powerful functions available in a pie tone like a map and a filter this six and covers all those basic fundamentals of a pie? Tony, this section is not for all those beginners who want to learn about the pike on programming language. If your absolute beginner better, you saw it for some other course, which will teach you in depth knowledge in a bike on programming language. And if you are to advance user, maybe all those information in this section will be leading them for you so better feel free to skip this section and go to the next section as we're going to program all those machine learning algorithm in tow. Fight on programming language, this sexiness for the refreshment off all those bite on fundamentals. So stay tuned. And from the next video, we will start with the basic day that I've seen a bite on. So thank you guys for the watching and see you into next with you. 5. Python - Number, String, Variable: Hello, friends. So welcome back to the section on a bite on Chris course. So let's start our journey with very simple date. Our type numbers. So all those positive negative numbers and decimals are considered as a number in a fight on. So let's see, What are they? So you can write like a three. 4.5 is considered as the number you can perform all those kind of arithmetic on the top of it, like 3.6, So it will give us nine. Let's say we want to perform like multiplication between 3.4 multiplied by five. Okay, so 17 you want to perform some kind of addict Matic like the reason rd when a model operation. So just like to buy for So as it is, a bite on the point X wasn't you do not require the explicitly right 4.0. Even if you supply to integer value, it will give us zero point fight. Let's say we want to combine multiple operators into single expression. So in this case by tone, well designed, a priority of individual operator based on more muscle thing plus six Indo fight. So what will happen here, the multiplication operator has a higher priority dinner. In addition, operator. So six multiply by five will give us 30 and 30. Blustery. 23. It won. I had three plus six first and then multiply bitter fight. So it just follows the same board muscle. Let's see another data type Laker string. Okay, so string is nothing but the collection off alpha numeric character. So character can be a one or more than one. Let's see. Hello. Why so in a single courts, we can light like a hello world, and it will consider it as a string. You know, bite on. We can even like the same thing in a double courts. So hello. Why? But no. Let's take another case. Like in sometimes in English language. We have a segments like I cannot. So it already contains one single courts. So how to defining her bite on programming language? So just put double quotes across it, and it will consider this single court says a not starting off the string, not ending off this tree. So this way we can define a string. So we helped define what is number in a bite on what is stringing up Eitan. No, let's see how to do this variable assignment so relatable. So there is no particular day that I really want to declare for all those particular variable at compile time in a pipe one Because by Tony za dynamically programming language . So you do not need to write like integer is equal to three like and you are trying to see fight on programming language Did the variable eser intention in case off fight on programming language and it said dynamically type language at the same time, it will automatically decide what type of that particular variable is. So just write the variable name and assign it to the value. Suppose we want to see what is the value of that particular valuable? Just put a and it will give us No, let's assign some other variable like X, and we will assigned it to those three. So hello way. Okay, let's bring the sex. Okay, so it has given us hello way. Now, suppose we want to see the type of that particular variably and a particular valuable X so we can use the function like type and inside that we can pass. Okay, so It says that he Tennesseean Taser so by Tony's automatically decided for us that what type of variable is and what type of variable access Let's apply same thing on X also. Okay, so it has given us str str is nothing but the street. But there are a couple of rules to define this kind of variable. You can start defining any variable or naming convention off any variable with the name like any alphanumeric character in an English language. But you cannot start put like well X is equal to 30 toe 20 so it really immediately give us that because it is not a valid bite on Centex. But if you liked like XT well off, it will consider this extra Lesa valid naming convention off the fight on variable. So X trail is nothing but the 23. So this way we can define ah different variable in a fight on. We can assign it are different variable and assigned different constant to the fight on radio with. So we have seen how what is numbers, What is string and what How did he find a valuable Now let's see what other operation we can perform on this string, so operation on Sting. Okay, so I have already defined one string like tax which contains this. Hello. Okay, now we want toe. Apply some matter on the top of this starting next. Let's see, we want to do upper cases off the sex, so just put X and dark and in place step and it will give us the suggestion. Like, what are the matters? You can apply on the top off this thing, so matter will be something like up, but Okay, so it has to be given all those upper case is so individual character of String X. Let's say lower cases. Okay, so all those character has been lower case middle pop matter this lower. So this way we can apply hell. Lot of different makers like let's see some more matters. His digit. Whether this axis particular digital. Not so. We applied this hex physical or disease. Okay. We need to supply the parentis is okay. False Because access Hello. Bolstering its not a digit. So this way we can apply all those operations on the top off string. Let's say we want to conquer next string with some other string like we have a string like X plus. We want to supply like hello by Okay, so it has immediately concluded netted our X variable, which is nothing but a string her lower and by and it is immediately upended this space by toad over existing x ray of it. Okay, so now next, see how to do this slicing or how to do how to extract the particular character out. Attack Street. So that is nothing but slicing. You know, I think, or how we can expect particular Katic in a whole strength. So let's just pretend experts. So xx indexing really start from 01 bo tree So zeroed element in acts will point to the edge, and that is nothing with the capital edge. First element will point to the a second element will point to dial. So let's say we want toe extract the very first element. So we'll just put X and then a bigger brackets. And inside the bigger breakfast, we'll just put Index zero. Okay, so he tells the bitterness, its capital it. Now let's say we want toe extract. The fifth element, as indexing is already started from zero. So fifth element will become like 0123 four. So 50 really element will become a blank. So let's extract the sixth element. So six Telemann We can expect winter in next fight. Okay, so it's a w. So 50 sixth element will be reference. Why the next fight? So zero first, 2nd 3rd 4th and this is a 51. Now let's say we want to accept the group off character, starting from some index and ending with some other index. So we want to extract, Let's say all a t l o. So what we can do? We can start with zero and we can end with 0123 as ah put Index is not being into consideration in case of bite on this string slicing. So we will just go toe fight so 5 15 Next will be this blank space and it won't be apart off all those number slicing from zero to fight so you can see it as immediately given us zero to Fort Element 01234 No, sorry. Actually, space has been included because we have given zero toe fight Now that we want to see all those elements starting from zero. So it has written full although string. Now, suppose we want to start all those elements which start from third Element and rest of the whole string. So third element will become 0123 So this is in the element that is 1/4 element, but Index's referenced by the three. So this way we can slicing and we can extract to some part off the string for the further processing. So the indus really over your seen what is number, what it's doing, What is variable? How Rio assigned this string and a number to the variable. And we have seen what operation even perform honestly, how you can do the slicing, how particular character and element you can extract out off the string. So that's all about this video friends. So see you do next video 6. Python - List, tuples, Dictionary, Set: Hello, friends. Welcome to the crease course, Sona Bhai tone. So in this video, we're going to see about by turn data structure like a list The buzz. What is dictionary and what is sex? What are the different operation you can perform on the top of this data structure and how it will be benefited toe create much more highly all functionality in a pie tone. So let's start with the list. So what is list list is nothing but an ordered collection off complex data structure arena Bite on soap I turn doesn't contain a, but it has given us another data structure. Alternative data structures like a list. So list is nothing but the alternative off are in a fight on, and list will give even more functionality. Dinner Ari in a bite on. So let's define a very simple list like three elements like 458 and you need to enclose Witter brackets. So it has a three elements 35 and eight, and it will return us a list. Let's say we want to put list within a list. So seem our Lear one, three five and the third element. We will put another list. Laker don't coma. Six. So this is a list of a dinner? Less. Let's say we want to assign this list to the variable like LST Okay, let's just bring this nihilistic. Okay, so that is a three element here. The first well, a ministry. The second element ISS fight and 1/3 element is another list, which contains another two Element two and a six. Now let's say we want to put any string inside this list so we can just straight away go here and we can put, like, any string, like have a look. Okay, now let's just sprint it LST So now. Element all change. The first element is type of string, which is indexed by number zero. Then the first element the second element tarred element enough. Fort Element. Now let's say we want to modify particular elements. Let's say hello to why we want to change. So something like love. Why I just make it come in. So hello to while I want to change it so we can just like LST off that zero to element we want to modify. So I'll just make it like right, Okay, so it has successfully executed legis printed because the first element has been changed. So this will begin modify particular element We can even a pendulum in Let's say we want to a pain LST has a one metre upping and we want to open like by Okay, let's just sprinted LST Okay, so we have ah Sendek how to modify particular element, how we can put even a complex data structure like another least regionalist. And then we hope undated One value. And every time when you applied this upended, the element goes into last index position plus one. Now let's school for the how we can extract particular element. Let's say we want to extract the fight So fires been index by zero Telemann This is threes off first index 52nd index. So LST second the Knicks. So it was three tennis. Fine. Let's say we want to extract this particular six. So six is nothing but the list within a list. So this is the first element. This is second element is this third element and this one is the fourth element and four Telemann is not a simple number, but it's another list. So it is index like 012 and three. So let's first extract first stick three. So it has the tennis lists. Now, out off all that particular list. Two comma six, we want to x like the second element. So second element, we will give the another indexing Laker one. So zero, it will be indexed by 22 And the 1st 1 will be Neck of One Index is the reference to the this six number sweet with the tennis six. So this way we can extract the complex Our data structure out off this listening I structure. So we have seen about the list data structures? No, let's go with another data structure. Laker the police. So I'll just say please. Okay, so we have seen that. Ah, list data structure and topples is exactly similar. Liker list only. The only difference is that the police are the immutable. Later structuring a fighter. So you cannot Marty fight. So once it is defined, you cannot modify it. Okay, so let's define the same Some couples like let's say the u P. Is equal toe, but in case off list, it has texture were given this bracket. Now, in case off tapas, we need to give the brackets like this. So same comma for comma. Let's say six. Okay, so it is. The return isn a couple now. Suppose we want to see what is the type of variable U P So just like a U. P inside the knife function. Okay, so it's a type off double. Now let's try to modify particular element the U P off. Oh, let's say zeroed element. We want to modify through five. OK, so it has immediately given other that in a couple Object doesn't support the item assignment. So once it has been defined, you just cannot modify it. But all those operation which were earlier seeing, like not related to any kind of modification on the top of this list of concept can be applied. So let's save you want toe extractor first elements a couple zero. So it has given a zero. Let's say we want to see the it's hard elements or indexes reference. Baidoa number sotto So this is the city, wrote element. This is the first element. This is a secondary wants a second element this toe. So now let's go to another detached actual liker dictionary hooking so dictionary is ah, like key value pair kind of data. So in but the earlier data structure we have seen, that index has been given by the numbers like 0123457 and list will keep going on. Once you are paying something, it will just up end to the last index plus one position. But let's save you want toe? Make this g r indexing like any string value like let's say, even how we can do it and corresponding toe this, given our correspondent. Oh, that particular we we want all sorts. It's some value. So in this case, Wien leader they touched actually, like a key value pair and a dictionary data structure will provide us. Let's say we want to define Ah, one dictionary object with given associated with value in and key to is associated with value toe. So let's define one very well. Like a dictionary D I c. Variable. And don't define this dictionary data you need to provide all those Were you in a Columbus is, so I'll just create one Colliver assists and inside that provided key one and a valuable so one element we have defined after they put the comma and put your another element, Laker. Keep to welcome. Okay, so like has been different. Let's just printed. So keep unwell when key to welcome. Let's say we want to extract the key one value. So dick, just supply this. Keep one instead off that numbering index index based on the number now we can provide. This index was down the street, so it's a Cuban. Okay, so it has return us the value one. Now here we have all provided this way. Llewyn well, one and a valuable in terms off string deserved of a loser initialing. But you can provide any other complex. Take a structure also possible. So this is how you can use the dictionary value? No, Let's see another new structure like a sex. So sets are you can consider something like no unique sets off element. Let's say we want toe create the sets off. 1 to 5 went two for five, so it has created a one set having a 1245 Now let's just apply. Let's has up in some more element, like a 4567 so you can see that Ford has been depicted. Time fires mainly predict two times, but it has not given as output for two times sort of by two times. So sad is nothing but the unique collection suffer number or unique collection off complex object you can even ask. Apply all those matter Laker up in and even a delete matter on the top off this certain element. So let's just assigned this set toe the variable like a steep. Just bring it and let's try to open some maybe some number upend mints. Nothing but the here ad. It won't go directly at the end, but once you supply what numbers in in this argument, let's say you supply to the eight, so it will just stay at the end of the dissect. Now let's say we want toe area for So it one pro Arab that Ford is already exists, but it will just written executives same list because it will scan through all those numbers, which is already available, you know, set variable esti and found that four is already available, so I do not need to upend out. I do not need to add one more four, because the kind off characteristic dissect the cash structure Conde has is like a unique set off character We want to put in this particular later structure. So see, after leading for it has not given ever, and one super in this nasty again. It's a 1 to 456 on it. It is exactly similares. I'll ear. So in this tutorial we have seen about the four very important retouched structure in a fight on. Like I said, next study the poise I know listed as So that's all about this video prince. I hope you enjoy the sending this video and see you into next video. 7. Python - If/else, Looping: Hello, friends. Welcome back. So in this video, we're going to see about what is decision making and what is the looping mechanism available in a fight on So disease and making will be something like if this kind off condition like if condition one is successful, Lynn perform, toss one or else but from Casto. So this type of scenario always comes in vials. You do the programming. So how we can do all those diseases and making, you know, fight on programming language. So to evaluate all those condition, we however, bullion, data structure bullion data type available in a pie tone. So fight on provides a two billion daily lives of one is a through and another one is a price. So this condition has been evaluated and it will return us whether particular condition is true or not that particular condition or expression or whatever mathematical and logical things we have written here. If that returns as a true then perform tasks, one will happen otherwise. But from us, who will happen? So let's try toe boarding a very a simple example. So how condition will be evaluated was done are some mathematical and logical operator Let's say we've right like a Feist, our forces greater than five. So this is definitely a false statement. So by tone will return us voice. Let's say let's change it to the for his legs. Then fight. Okay, so it has a return is true because this is a true So whenever this kind of condition becomes to this first line performed task, one will be exhibited. A device performed Pasto will be executed. Let's called it very simple. Let's say we are writing like who is always greater than zero? Then we will just printed Tosca one Hells and I asked. Okay, so it has given us Adam. We need to semi column. So the first time the condition has been executed like who is greater than zero? So it has evaluated as a true So what? They were the order block me on returning it inside the if statement that will be executed . So task well, let's make this condition it diluted toe voice. So too, is less than let's say zero. So it has. The return is the task to so control goes to the else block this way. Even else condition has been executed. Let's say Apart from Task One and Pasto, we have Ah past like basketry and we want to perform this task. Three, based on some other condition gets true. Let's say after two is less than zero we have evaluated. We want to perform again. Better five is less than three or not. So let's say we will light like hell. If so, that is nothing better. Health savings FIEs Lee greater than lets it three and you want to perform task three. So what will happen here? First condition has been evaluated and it is false. Sorgen control goes to the next block so five is greater than three. It really well it and it will check that fights greater than three. So it really, really toe the true statement and it will go to the this next immediate block off execution which is nothing but past three. Okay, so it has successfully return is the task three So control has not gone and reach to the last block. Let's make this condition also Eveillard toe the force and let's see what they will print. So it has printed task for us because booed off this first and second condition evaluated to the falls. So this way decision making happens in a bite on programming language. Now let's see about another mechanism like looping mechanism. So for looping in a pie tone, we have a true loops are available. When is the full open another rings value? So let's see both off them one by one. So let's say we have oven function available for the attrition in a bite on Laker. Arrange function so vain function what we believe in you. If you provide, let's say six. It will return us the range between zero toe fight and let's apply the list on the top off this range. Okay, so it is that it turns the list. The all numbers start from zero to fight. Now let's save you want to irritate him individual element off this list, and then we want toe print all those elements. So in that case, and we can use this for look so four and let's make some temporary very well, like a damn in a loop or in a list. What we're going to use it is nothing but the lane six. So what it will do, it will take all those individual element in Lane six function will provide us, and that will assign for each iteration toe the time valuable. And then we'll see how toe print this. Okay, so it has a threat to every single element, which he's been given by this range function. Lane six. So at the first tee prison it has given the function like function is returned like zero and zero is going to the camp, and it s print. Take time. The second name, The Rangers provided us the one and it is printed one. Because the time variable is now one. Let's say we have a one list. Let's create a list like 35 and then put some high like a string valuable. And let's spring another like Kate. And let's say we want to eat threat this Alice de Variable and we want to separate individual elements. So what we can do? We can use this for loop, and we'll use this temporary variable inside this list because here also, it was returning list for us and Individual Limited we were treating here. In this case, we hope one list is available with the LST very human and for each way. Temporary variable Laissus printed. Okay, so it has been that 1st 3 by then high and eight so incrementally it will do so. This type of operation is very much common, you know, fight on programming and we will be keep using this type of thing in over further future lectured throughout discourse. Let's see another looping mechanism like vile. Why you So what is the difference between Via Look on our four Lopez that for Lobi trades through all those element available in honor of a different data structure like list? Are you an addict? Study data structure and it will keep in implementing the variable after every one interval . But here, in this case, violin Clement station. We can explicitly need toe right? So let's say we want to perform some action on Listen until this particular condition is being satisfied. Let's say we want to print high 10 times. So let's they find one variable like eyes equal to zero, and then we will assign it to the eyes list. Then then. So as soon as the eyes less than 10 I first I will become zero. So zero is less than 10. It will evaluate toe the true so it will enter inside the loop and inside the loop we want to print high. Now what will happen that in the next attrition again I become zero. So every single it prison that is no implementation off eyes happening. So this via love will go in tow, are in fine ish Look So it want the dominant? We need to explicitly tell minutes So here, in case off this via Look what we need to do we need toe incriminate Desai variable periodically. So after everyone attrition we will incriminate this I value icicle toe I plus one So immediately in the next station I becomes too. I become three I become for the moment I become 10. 10 is less than 10 Which will Evolet toe the completely false statement Once it is fall, it want entering toe look and our whole looping will become dominant. So let's just friend it. So it has spring Take this high. This is high related to first time. Highly to the second name. Okay, so we have seen about how to evaluate the condition, become true or false? We have seen about the disease and making fl statement in a fight on we have seen about four and by a local looping mechanism in a fight on in this video. So that's all about this video prince. I hope you enjoy the sending this video and see you in the next video. 8. Python - Function, Lambda, Map: Hello, friends. Welcome back to the course on a pie turned racecourse. So in this video, we're going to see about what dysfunction? What is Lamba expression? How to use this map and filter functionality off a bite on environment. So let's start with the function. So what dysfunction. Why that function kind of functionality is required in the programming and monument. Let's say we want to perform some task repetitively in a programming. So if you want to perform some task in a repetitive manner, you need to write that Gold also emancipated humanity. But that will be a too much Liberace process to write the same thing again and again in a according for me. So function will provide you with such a functionality that your light function once and use as many times as possible. So let's create a very simple function, and let's call it so we'll understand. So to create the function in a bite on the function will start better keyword like that. That is nothing but the definition. And let's create a very simple function. Laker friend. Hello. As a part of argument, we won't pass anything right now, and we'll just print Hello. Okay, so we have successfully defined a function print. Hello, Legis colleague. Okay, so he has painted Hello for us. Now let's make it a little bit more complicated. Will increase the complexities. Step by step. Let's say we want toe print not hello, but our user defined some other street. So as a part off argument, we can pass some other variables. So let's create the same Think again. Let's say another function like a print. Hello one. And as argument we'll just pass. Barham is equal to better and let's print it. Hey them! Okay, so it's Bs defined another function bring hello one But what it will do it will bring the value of it is coming as a part of argument. So this function and the argument name is Adam. So bring hello one x sprinted So instead of hello, let's say everyone double in high. Okay, so we have successfully printed high? No. Suppose we print hello one without any argument. So it is immediately u N Azadeh It is saying that missing one required traditional argument . Barham So you help to explicitly light one argument because you are not supplying any arguments so it has immediately given us Carol. No. Instead of just passing this, Batam Villagers assigned this Brademas summly fold value Laker hello or we could say like defiled default value. Okay, so if you don't supply anything, it will automatically take this in default value. So let's just execute dysfunction after supplying high again. It has printed high because, as you are explicitly supplying some argument it won't take into, Consider sent this default value rather than what year supplying that has a higher property . But now again, call the same function and we're not going toe supply any argument as a part of input. So see it has a printer a default value because, as you are not supplying any argument it has, they couldn't Disney fold value as argument, and it has printed. So we have seen that how supplied argument out to set the default value argument and hope, without even argument, hope to write the function. No. Let's see how we can return something from this variable or beat returned something from DYS function and that function output. We can catch it in tow. Some variable. Let's define some other function maker, Let's say addition and we want to supply to argument like me. So let's lead another valuable like see single A plus speak. We just want to add and really just every time I see Okay, so we have successfully defined available our function head. Now if you see just plane dad, it will give us the function mean it. So it's a type of function. Let's say we will supply like four comma six. So it has immediately returners. No. Then now, if you want to catch it to some other variable like oh, so it wanted it done. It has returned, but it has not print daily. Now let's just plain to be so. These Siegel do 10. So what? They were Zickler after executive execution of dysfunction. Whatever output came, it has caught it in tow, variability and very really become then. So we have seen many more flavors off how to write the function hopeful you turn available from the function. How to supply argument hopes that put the defiant argument as a part of function? No, let's go to the another concept. Liker Lambro expression. So what is Lambrecks prison? So before diving into Lamela ex prison, let's first creates very simple function like Er I had one. And then we will try to see that that particular function, how we can re present as a Lamela expression with a minimum amount of syntax or minimum amount of courting. Let's define like air five function. So what it will do whatever input are given, you will supply. It will just add fight and it will return it. So time a plus fight. Okay, now, if you call this function or if I would, let's say argument like eight. So it plus fight should written for 13. Okay, now the same function. We will try to light with lemon Expedition. So how we can do it. So let's first create one variable verifying what we will use that 51 in disgrace. Then let's start fritter Lamela Now, After Lamela, you just supply your input argument. Let's several input argument will be in after Colon. Just put whatever the variable you want toe right on you. So you want to return it? He bless. Fight. Okay, So become very minimal amount of set texts. We have created the lamb or expression or Lamela function held fight one solicitous, executed so we have successfully executed it. So lamb lies the key one easy. What argument and whatever. How could you want toe return it or what they were the variable. You want to return it? Just put after semi colons and this input argument, you can use it in your expression, whatever you want to return. Now let's just call. Haired Firing has supplied is late again. Okay, so it has the return, the 13 for us. So the same way we can write the same function with a function V or either Lamoreaux expression. The main difference is that if you need to write a very minimal amount off according with just the one liner, you can do the same functionality. Here. You need to write a letter multiple lines, so that is the advantage. Lamela expression will give over a simple function. Okay, so let's moto another concept, like a map and then freak. So what is map map itself? Suggest that it is mapping on one sequence off number two, the another sequence up number and what the transformation from one sequence off number two . The another sequence of number he's given by us as a programmer in terms off function. So let's say one sequence we will define like a We have really find number off list in earlier total U. So let's define one list like 257 Hey, and let's everyone toe apply this whole sequence off data over a real function like air. If I I had a fight. So what do we can do? We can use this map functionality. So if you see this map functionality, you can just supply this air fight first argument and then another argument is LST So what will happen that every single value our every single item contains in this LST variable will be applied one by month to the function at fight and whatever will return that is on another list only. Okay, so it says that Ethan, honor that a map. But we need toe wrap up to the list. So do in return us the least porous. So Okay, you can see that every single value, like a who has been applied to the function verify so two plus five becomes 75 plus five become 10 simplify become well, and it plus fight, become typing. So this were map functionality. Will be very useful that you want to apply some function. So the whole range of list off numbers like listing a fight in a case off, bite on so you can use it. Now let's see about the filter functionality. What is filter? Filter? The name itself. Such is dead. Let's say you want to filter some data so we have one sequence like LST celestic intends to 57 and eight. Let's say we want to filter on Lee does record even in this case, hardly reach at art and discuss So how we can do it. So saving we can do I think that as a part off first argument, we need to supply the function or the logic or the condition with the condition match toe individual element. If that condition has been match for this individual element, then only it will return is so here. We need to write like a function that we will write it waas and then this, as a part of second argument will supply the whole sequence off number. It is nothing but The LST network is now how we can like the function. Just now we have seen that we can write a very simple one. Liners function, Get dealt book Lamela So Lamela and then as input argument. Let's stick. And in a value like a the eyes are coming. And just that he done veni where he is. Even so, a Model two is equal to zero. So what it will do that every single number from this Alice T our list we'll take like a to . After that you will be applied on dysfunction. Lamela. Now to we'll go inside dysfunction Lamela and it will be valued the condition. If that condition become true and only it will output. Otherwise it will. If the condition become force, it will just feel tell that number and let's just wrap it to the list. Okay, so it has written only those will you feature even for us. So who is even eighties even and remaining to? Well, you find seven. It is Peter. Let's take Onley or a value. So instead of July, just make it so it has returned only fine. Seven. So this way, if you want to filter date hour, if you want to remove some data based on some condition, this filter functionality will be very helpful. So that's all about this video friends. So we have seen in this video like a map and filter functionality how to write a Lamoreaux expression. How creator of functioning a bite on. So I hope you're in Julie's and in this video and I see you into next video. 9. Introduction - Numpy - Numerica Python: Hello, friends. Welcome to the new section on a new medical fight only library. So in this section, we are going to learn about a new medical bite on library. So what is numb by why we need to study about the number? So let's try to understand Num pies and New Medical Pytorch Library. It is one of the core library for the scientific computing in a fight on it is giving on many advantages over by Tonalist, in which we have seen in a last section. It has even one dimensional and a two dimensional vector and mattresses related processing at a very high performance scale. Lots of linear algebra related or prisons are available. Many of the fight on scientific computing library use numbers Abakan throughout this whole cause. We're going to use this psych it learn. Ah, machine learning library for fighter is using numb pies there back in system. So whatever input data will supply real supplies and Mumbai curry. So that is the all reason we should learn about the number library. What topics we're going to cover in this section is we'll see what is number Ira, how we can create this numb Pirie from the different races. What are the operation we can perform on the top of this number theory. What is number indexing and how to do slicing on a number three. Okay, so now next task is how to install this number I library. So I have already created video that ho toe installed this full and a condom packets. So if you have already installed the full anaconda picketing by default, this whole library is already available as a part of anaconda distribution. But if you want to install this separately, or if it is not install on your machine as a part of cornered exhibition what you can do for the installation purpose, you can just write like cornea installed numb by. There's the one way you can go ahead and install this number by Liberia's Apart Off Condom distribution another ways that if you haven't installed this anaconda, you can go with a pimp installer so you can use this paper in stall number I. If people is already stole, it is nothing but the package manager for the pipe. So these other two way you can install there's not by Liberian on your machine now Let's see how we can import this number I package. So for the importing of number, I pick, it will just import important numb by And we will give early as name like and peaks a citizen standard convincing Whole community will use So every function we can address with A and B we do not need to like this whole number. So it really bought all those functionality available in a number library. Let's just executive it. So we have successfully, I guess. Executed. Okay, so I took little make time. It has successfully executed this number library, so stadium. And from the next video, we will get started with the number IRA. What other? Numb by Ari's. So, thank you guys for the watching and see you in the next video. 10. Numpy array: Hello, friends. Welcome back. So in this video we're going to see how we can create a numb Pirie with the help of different built in functions available and with the help of quality available by tonalist. So let's start creating the num fire and bitter building by tonalist implementation. So let's create first a very simple list. So we create the one dimensional list first. And from that one dimensional list, we will create the Num Pirie. So one dimensional lists no umpiring. And then we will create a matrix which is nothing but the list within a list and from that vehicle it on number theory. So let's start, let's say Dr Ill defined and really supplied it three value. I want to win three. Now we have one list name as a vector. Be easy. So from that how we can create a number very so inside number. We will reference every single function with MP as we have already imported, not by library as a in import package in the last, really only so and be dark and then place Just step and it will give us the suggests index . What other operation and my time we can apply on the top of the send me So we have a microlight and inside that we will just pass the value like back so it and be underscored record. I just assign it to die and be in this, Corvette said. Now by Vector, this is a normal director. Okay, let's just bring, take and be back. Okay, so it has created one dimensional numb Pirie from the over simple fight on list. Let's see what other type of this NPR It is a number I dot n dimensional type. Let's create a story metrics. I'll give the name like Mad Max my picks and we will provide what Two dimensionality? Let's see Doak Omar, six common eight and another row like three. Coma, seven, comma zero. And everything will be in close by another briquettes which will indicate stick. That is a new element in this list. This is Father Frost Row, and this is for the second. So there is a total and total three column Sunday. So list within a list now the same function. We will use it, you know, umpire, and we will supply Mike and let's assign it took her and the younger school. Mac. Okay, lets you spring this name Be okay so you tested it. Dinos are a with a two dimensional. There is a to grow and a total three column Suddenly so in this way from the by Thanh Basic Implementation Off list We can create a numb Pirie and all those operation we can apply after deck on the top off written Now let's see what other? Another different ways We can create this numb Pirie So there are a lot of building functions are available in a number I library to create the number Iran. Let's see some of the special one like Zito's once I How we can create it. So let's say we want to create numb by two dimensional array with all values become zeros or already has become one what we want to create identity matrix. We can use this I function Okay, let's start with the creating zeros so we'll use the function like and we got zeroes and inside that we will pass. But the pies the person we have seen in a fight on basic racecourse that topples is like immutable bi tonal object So once it has been defined. You can't change it. So you need to supply topples and opposing the farm off. First value will be number off. Rosey want opiate and a second will you will be number off columns. So we want to create a two dimensional array with all value zero where number of rows is equal to three and number off columns will become four. Okay, so it has nicely generated. Total three years and four column. But for us and all value feel bitter. Zero. Let's say we want to create all values with the ones so we have a function like ones. Same way we can apply. Couple three Common for all their user one. Let's say we want to generate the identity matrix so we can use hi function and inside die again. You can supply this. Okay. Here you need to supply only one value. So you just supply well so it will create a three cross three square matrix diamond salary which having a three goes and three columns and all diagonal element really become one. So it has nicely generated for us. Three. Those three colors. Let's say we want to create a 606 column. So all those diagonal elements will become one and total number of Rosie's six and total number of Collins. Also six. Okay, so let's see some other B to generate these numb pirie we can generate with random function . So all values inside this numb pirate will be filled bitter, some random value so we can use a Brendan function so we can use our random function on the top of this and be so we have a random function and inside the random there number of ways you can generate, like rain, random in major land. And so we're going to use this landing so all of you will be standard distribution with it regionally, let's say we want to generate off I value. Okay, so it has generated all value somewhere in the negative form, some marina positive for, But whatever it has tended it disease in a one dimensional vector for me. Let's say we want to generate this in a diamond snow metrics permit. So in paedo trend, not save you want does NT $3 a four column so total total value it has generated, which is in a form off total credos and a four column we can even use landerman teacher So friendly lock That is the that I love. So if you see this, the randomness major we can supply like total value of one is the start. Anyone is in so between which range you want to generate this value. Let's save you want to generate a number between five and if it clean. Okay, so it has donated one number, which is an intention number between find 50. But as the tired argument we haven't provided, the car argument will tell them how many number between the range fire and 50 you want to generate. Let's say we want to select a fine number between five and 15. Okay, so it has generated 14 2 times 779 let's say between fire and 151 to generate total final so total final but has been generated so deserve the random one can generate our random number and it will return as the num pirie. There are another two way one can generate this numb pirie. So one is a range function in another one is the line space for so let's see both of them one by one. So let's say n p dot peerage. So inside the earrings you want toe give the three arguments total minimum to argument you can you? Let's say you want to generate the number from 3 to 12. Okay, so it has started from three and 12 has been excluded. The upper moment is always excluded in every single function in a fight on. But all values has been implemented by one so 3 to 4 to 6 to 7. In this way, let's say we want to go from 3 to 12 in a step off. So all value has been implemented by five and it will wish upto 12. But well has been excluded. So this way we can use a range function and erase function. What it will do. It will start from the starting number two trending number. And this story is nothing but the incremental value that between two number how many interval you need that is another way is a line space function. So np dark line space here also do argument unit toe three argument. You need to supply minimum to argument. So let's say between to win then you need toe, generate the number so it has generated between two and 10. So if you see how many numbers it has generated, uh, that moment is that they're so let me just make it like it will do in a fight. Okay, so there is a 50 value it has generated. So by default, might be has taken a 50 value toe, get generated between two and five, and fives also included in this case. But let's say we want to generates just the tang value between to identify with the humanly space in earlier cases. The last hard argument has the significance of how much you need to incriminate from one number to the other. Another number. So what is the difference between corresponding to number? And here in the case, off this line space, the tar argument value is related toe. How many evenly spaced partition between start and your ending number? Okay, so you can see that there is a 10 part it has done between two winner five. So this way we can generate the number with a range in a line space function And the third argument the first how much unit increment in case of earrings and how many evenly spaced partition unit to do between starting and number. Okay, so we have seen a lot of ways to generator and dimensional are a in a number higher library . Now what are the operation we can apply on the top of this numb pirie, or what are the matter or meta data we can extract out of this numbering. So in our little we hope created to worry like and be in the school rack is one dimensional vector, and under one is and being this good is it told I missed. Now let's see what other things we can do with this and P in this school. Let's say we want to see what other dimensional So we just apply, sit, attribute on the top of this and be back, and it has given us their total pre elements. So it's a one dimensional area. There is only three elements are there. Let's say we want to see and be in the score. Met seep. So there is a total Turow's and at three columns it has given us. Let's say we want to find the maximum value out off this MP in the score back so you can see how many matter you can apply. So next is dead. So that is the maximum value is three. Let's say we want to see the what is the minimum value. So minimum values one the same thing. You can apply on the top of this, NPR underscored matters. Okay, so the housing how to find the maximum and minimum mode of vector and my crisis open number by end Divisional. Now let's see whether we can reset this number. Aerial. Not so we have, ah, number Ira like and be in the school man, which is over two rows and a three column can be converted into three rows in a two column . So we have a functionality available like and be dot and the school make having a function Laker the re ship and inside the reception just supply. What is the output dimension you want now? The multiplication off this rows and columns is exactly same as your original source matrix multiplication of rows and columns as there is a total six values are available. You cannot enjoy it in tow to rose in the three columns. Now you want to recep to the three rows in it column. Okay. It has given us total three years in two columns of 26 hate three and a 70 Now you see supply toe three rows and three columns immediately. It will give Harris because Dimension is mismatching. What, you want to go with one dimensionally like a six rosen on one color. So it has given us all those value in a single single row. Let's say we want to receptive of one Drew in a six column. Okay, so total one Drew and a six column it has given. Now, let's say we want to see what other beta type of individual elements are available in a number fiery. So and p underscore Met Doc. You can supply like a type any type of it. So it has given us all a loser in Tasia Tactic a bit. So we have seen in this lecture a lot of things about the num Pirie how we can create this new empire. Every tell popular, built in function like hearings, lines press at our number of generally the NGOs, Delta Pi, Tony's and what are the number of different attributes you can. Oh, play under top off. No. By getting so that's all about this video friends. So I hope you enjoy the sending this video and see you in the next video. 11. Numpy array operations: Hello, friends. Welcome back. So in this video, we're going to see about what are the operation you can perform on the top off number very In the last video of your seen hall to generate Numb Pirie in this video, we're going to see what that operation you can perform on top of Numb Pirie. So let's get started So bored that let's just create a very simple one Vector based number very so will create like p dot r a. And inside that will pass like 01 toe tea for five. Alexey Very simple. No. Suppose we want to add seven to each and every number we can just straight away go like so it has Alex seven to every single number. So Jiro plus seven one plus seven to plus seven is nine. Let's say we want to substrate two out of every single number, so zero minus to become minus 21 minus toe become minus one. Let's say we want to perform some scaler operation like we want to multiply every single element. So geo multiplied by 31 multiply very 35 multi fiber, 15 Same way you can go with the wide So let's save you want to do right by crew so every single value has been divided by So this way we can perform all those scaler operation on the top off of one dimensionality. All those functionality is same as applicable on two diamond Snell area also. So let's see some oregano metric function, which will calculate the barometric value of each and every value available in a hurry so we can just write like me not okay. We need to use this and p dot sign, and inside it will pass the A. So for each and every value hope containing a any, it has Cal collect a sign value object particular all those other car econometric function , like a car sign changing all those you can find it. So this way you can perform all those trigonometry competition on the top of fighting. Let's see somewhat that one. Let's say we want to do some off all we're losing. He so you can see that zero plus one plus two plus three plus four plus five become a 15 so already will be summed up. Let's see some other function like we want to find a mean off all values in our A so mean of all values in a hurry become 2.5. So this way we can perform all those addict Matic operation on the tops. Now buy it. Let's define some more complicated operation like With which will perform on multiple Aries . So I have already defined 011 day, Mrs Hillary, Let's define it. Told them It's Naledi. Let's say one is even and well defined to damage Hillary. They say 146 and really find an adorable visa. Be one. You says you two okay, be I father to write the brackets. Okay, So it has successfully created to worry which is both off damage. Who crossed your limits in the two rows and two columns is this spring even And there Reza Leven now based on this to what we can do begin just right even bless me one so corresponding element in the first row. First column it has addict. So one plus zero become one t blessed to become fight four plus three become seven and six plus one becomes saying Let's do this obstruction. Okay, so one minus, you know, become 13 minus two become one even for minus three. Become one and six minus one become fight. Let's go element by element multiplication silly. One multiplied by. So all those element has been multiply corresponding. So that's a normal dot product element. Let's say we want to do them at tracks multiplication. So we have a function like dark and second argument. You can pass it as a leaving. So you are applying this metrics. Multiple isn't dot is nothing, but I just liked as a comment you got here are the first to metrics occasion. So let's just perform even dot even so, even has been multiplied by be even as a matrix multiplication. So there are a lot of other like a statistical function all those conditional arms function . Also, you gonna play? We already defined won t. So let's apply this condition. Let's say we want to compare each and every element with it's a two so is greater than two or not. If he is greater than to that miss, every single element in a is greater than if it is good that then toe it will return a proof authorizing should return. If I so it will just compare it with every single value. So 1st 3 value is less than toe for equal toe the remaining three way loser Good intento. So it has a return. But so we have seen in this video that how you can perform all kinds off automatic between to better to my crisis, how you can put form element by element multiple it is. And the reason how you can perform very simple trigonometry function. You can go ahead and apply toe statistical misery, technique like mean median also, and a very simple my scaler operation on the top of our thing. So that's all about this video evidence. I hope you enjoy listening this video and thank you guys for the watching and see you next week. You 12. Indexing, Slicing - Numpy array: Hello, friends. Welcome back. So in this video, we're going to see about what is indexing in a numb Pirie What is slicing and what is the broadcasting kind off concept? So we have seen in earlier videos how to generate a number by a ray. Then we have seen that more that operation you can perform on the top of number theory. Now we're going to see what is indexing. So for that, I'm trying toe so you of it. One dimensional array of what is indexing. And then with the toe dimensionality, let's create one plenty. We can't be dot Aaron's and we will create a 25 element array. Leven. Let's just spring leaving. So it has created a one dimensional array. It is having a total one draw and a 25 column. All value will start from zero to 24. So total 25. If you see here indexing, the next thing is started at zero position. So this is the GE wrote element is the first element. As here, the indexing number and a value itself becomes same. So don't get confused. The value can be anything. So this is a geotel Ament is first, 2nd 3rd 4th in this week. 24. Let's create on another Ari are another Mac prices, which is anger 25 element only. But let's form it into a fight Crossfire metrics for So let's get another and Peter he range will create Identify really 25 Elementary only, but we'll just reshaped. Oh, fight Crossfire. Okay, so let's just print it. So here you can see that all those 25 element only there Joe 2 24 But it is arranging Firoz and a fight column. Former So you can see that this is the first row. This is the second room and Rose has been started from And next thing zero, this is the first row. This is the second row. This is the third row is a four through index by zero for second tour and 1/4. Actually, it's a first row second target. Ruin of four trying. A 50 same way column will start from 0123 and four. So let's try to see that this is all about indexing. Let's try to see how weakens extract the partial portion out off all those full. Let's say we want to grab element from 4 to 10 from the first hurry leaving. So there is a common Syntex year Leven, and then you can light like a start, indexing and then end. Indexing always and index is not always included, so you can just light like N plus one if we want to grab from 4 to 11. So what we can do? We can start fit of four and we can go up to 12 but valleys upper bound that one be included. So this is the comments in text. We are going to goes, so will say, like 4 to 12. So you can see that the element has came like 456789 10 11 up to 11. Let's say we want to see all the rows are all all the element in a row, which is starting from for So what we can do begin start from four and ending. We don't know we want to go till and so instead of putting any end value, we just put it as a space or we don't put anything so you can see that the star value start from four and it has gone up to end off the elements. The ending of element is nothing but 24. Let's say we want to grab the all elements starting from zero or element Start from very first element off Cotwell. So how we can do it? So very start. We want to start from the very first element so we do not make to require to put any starting here. But we want to go a put well. So we will include Tartine Leah. So it will be included. It well, als zero toe. So this way we can extract the part off the number early with such a kind of slicing mechanism. Now let's apply the same thing. Honor code. I miss Mallory. So let me just print this to the area. Let's say we want to grab the very first element, So I need to hear the comments in Texas. A role number and then column number Certainly, too. So we want to grab zero pro henna zero column. So it is Jiro. Let's say we want to grab this 11 so 11 come says, Oh, zero toe, first row in a second room. So it's a second row and a column will become 1st 1 So this is the first column. This is a second column. So second row in the first column look. So till now we have extracted Ah, very single element. Let's say we want to grab the whole row or even whole color. So how we can do it? They say, We want to grab this 56789 the very second row. So how we can do it? We want to get a second rules will just write the second and then comma All column we need to expect will just putting Colon. So all element off that particular colon. So we do not need to specify any column because we required to grab all the values in particular color. But the very second row we need to grab it. So OK, sorry until its first row real first row, we need to grab it. Okay, so it has grabbed this 56789 Now let's say we want to grab this fourth color if it is indexed by column number three. So we in this case we need to grab all the room, but column number only fourth column and four column is indexed by the number three, so you can see 38 13 18 and 23 has been expected. So this way you can extract the piece off the functionality. Let's say we want to grab this small square. No, in between, so square will be something like 678 11 12 13 16 17 18. So this piece up square out of this hole met. Prices will start from very first row, second room and 1/3 room. So indexing will start with, like first row. We need to go. We need to go to second row. We need to goto third row, but third row. If you want to include thoroughly. So we need to reach and we need toe include this four toes. So we have grabbed from first to fourth. So first throw it will have included Second row will be included. Third roving big Krulik and four will be excluded. Let's Rabia put this forth Same way for the column. Also, we want to include this second column torque Parliament, fourth color. So include this 2nd 3rd and fourth column. We need to start indexing from voiced tow fourth so you can see that 678 11 12 13 16 17 18 If you don't understand anything, don't worry about it. Depressed, practicing and keep trying together. Some piece off information out of this hole. Bigger. So that is all about indexing and a slicing. Now let's see about the broadcasting. So we're going to do this broadcasting on the even, which is a one dimensional early, Let's say the first fire element zero to fourth. We need to cite it to the like 15. So let's first grab this 0 to 50 l. J Toto Filemon So we'll start with zero. So we do not need to write anything or we can even write. Also zero toe fight so it will include 01234501234 Total Fire element vo extinct. Now we want to set all this element to them. Let's say minus 10 and let's desperate this demon so you can see that the very first fire element has been changed to the minus 10 remaining. All elements become hunting, so that is what nothing but the broadcast that if you want toe change the piece off the our number off are part off some elements in the whole Sorry for my prices. If you want to change, you can just Www assign with tell pops this slicing notation and it will be changed. Okay, so we have seen about indexing. What is indexing off number Marijo. You can slice the part off this number theory. What is the broadcasting thing? And there are many more concepts are also there in a numb Pirie. You can just go through the number website and go to the documentation. You will get it. So that's all about this Radio France. I hope you enjoy listening this video. And if you haven't understood anything, if you have any kind of confident police, light it in tow, Cubana and I'll be very happy to help you at last. Thank us for the watching and see you into next with you. 13. Introduction - Pandas: Hello, friends. Welcome back to the section on data analysis Library. Find us. So in this section we are going to learn about one of the very important library for the data analysis purpose in a fight on it is nothing but find us. And it's another advance wasn't off the Excel, but is having much, much more feature than a maxim. So in this section, we are going to learn about phenomenal data structure off find as likely one is the city's data structure, and other one is a data frame data structure. Then we will see about what other operations you can perform under top of this data. Friend, have you can handle the missing data when you our data available in multiple places like in a multiple data frames or multiple sees form objects, how you can most it and how you can join those object how you can grab this data from the varieties of different sources, like a file reading and how you can save it for the future. Use like a file writing. So these are all topic. We're going toe cover from the next video and worse, no. Next us is how you can install this. So you fear already is told the Anaconda full package, that is no need to install it. But if you're installed and a condom mini package, you need to install despond as library explicitly. For that, you can just write the command like Gonda install. Find us and it will install. Find us library. Now, if you want to install the Standalone Pandas Library, you can use the standalone way to install a package manager in a fight on like a pimp installer. So just write the people still find us any quint installing I have already stolen my machine. So let me just make unnecessary import further point. That's library to get starting. So know my naming convincer for the partners. Like important. Find us and we will use the Elias name like er whole community used, which is nothing but the PD. So from the next video on bars, all the function which were going toe using a finder's library, we will reference it with the name really, rather than using the full name partners. So let me just executive import. Fine Does is a PD. Okay, so I have successfully executed. That means that it has been installed on my machine. So stay tuned from the next video. And must we are going to get started with a very important part off the dynasties process our data from data structure in a series. The constructor. So I hope you enjoy listening to sweet here and see you in the next video. 14. Pandas - Introduction to Series: Hello, friends. Welcome back to the Jackson on Late Analysis Library. Find us. So in this video, we're going to learn about one of the very important data types cities in a bind us. So first we'll see how we can create the city's kind of object in a bind us. But basically what this city's is, you can consider this city's as a one dimensional number. Very then how it is different from the number here. As we proceed, we will see one by one. But there is a more than that number, Harry, this city's functionality has. So let's see how we can create this city's day that I've been a pint us. What are the different ways we can create it, either through lists. So let's start first important number and we'll import find us so we can create it with the city's function. We'll apply on the top off this baby, and we can just supply one least like to comma six comma can. Okay, so it has return it. One looks like one dimensional array was in next week's starting from 012 and all those value has been starting from 26 and 10. So this is a que lindo, the one dimensional numb pirate in one diamonds and umpire. All indexes are like individual based indexing, but cities is giving us one more and wanted dick. Let's say we want to keep the indexing like non American nature so we can supply here the second argument like Index. And here we can assign them three Index because we are supplying here. That three value So index will be something like a recipe. 1/3 1 Let's call it. See? Okay. So you can see now in database indexing has been replaced with this non numeric custom, annexing what you provided as a second argument while creating the city's data structure so easy called Toto bees equal to six and a season called 10. Now, suppose we want to grab particular element so we can just directly grab it like OK, we need to assign it toe some cities. They got structure. Let's assign it to the S one. So s one. Let's say we just want the very first element. Okay, element which indexes a so easy going toe toe. Let's say we needed more than one element, so we just need to pass as a list, so we'll pass it like a and C we want Okay, so easy Goto and a seasonal toe. 10. So this way we can create the city's let's see some more way. Let's say we have a built in data structure available in a bite on Lego dictionary data structure. So how we can create it with help off dictionary. So instead of passing these values and in next individually, we can pass it like cities data structure also. So we will create one dictionary object and we will pass it here. So let's say that's not the object will be like a one and value will keep it like PFI another Leven value. We will keep it like seven. Let's say 3rd 1 for okay, so it has the Eternal Cities data structure was index are even be one and see what it is equivalent to the G. Bai losing a dictionary later time and all their user 57 and a four. So it is automatically taken. This key values in additional it infrastructure to the indexing. See these data structure arena Find us now every time we host a plate just a very numeric value as, ah, valuing, vile, creating this data structure cities. But instead of that, we can create even more complex object was. So let's say you want to create later. Hello, Sound will keep. Remain as it is, Let's say, 3rd 1 we want to keep it like through a 1,000,000,000 value so you can see even is reference, even indexes. Referencing to the hello string. Beaman is pointing to the numeric values seven and see Venice pointed the bullion value. True. So in this way you can put even more complex. Object was a possible so this way number off. We we can create the city's data structure, you know, find us lately. Now let's see what other operations we can perform on the top of it. So for that, let's create a very simple to string like as one musical cities. And let's create it with indexes like one comma, three comma five and we'll put the next, like Lexie, A, B and C. Let's create another one like us to, and we will do some mathematical operation like element by elements, multiplication or element by Element. Addison's obstruction. Let's put it like to seven skipping nine. Okay, so instead of in same indexing will keep it in next thing like a B C and a D. Okay, so we have created two cities Data structure in appoint us. Let's say both of Denver one by one. So easy pointing toe one in the first cities Beast three C's eagle to fight. Let's put S please hold these two season below seven and these ballooning. Now let's do the multiplication off. This s one and s two. So it will do this element by element multiplication wherever there is a next will match. So let's see what the result will come. So you can see is indexing witches spine to do not a number. Why? Because is existing s one, but he doesn't exist in a yes to has is doesn't exist in as toe. It has not given those multiplication of value and it is leaving. It's not a number, but in case or B b index like this limbo That's one in us to be here. Isabella three and your busy little toe. Three multiple vibrato become six same way for C. C's will If I see Ziegel to seven. So seven multiplied verified certified and for D again. So he doesn't exist in the first data. Uh, cities. And it existed a second cities, but it has nowhere Luna one cities. So it has just thrown us not a number. So this way we can we can even sub apply subsection. So for that year and Levi didn't get a value because there is a data missing it for B and C v. Carter value, it has done index based element by limits, obstruction or multiplication. So in this video, we have seen better One of the very important data type in a fine dust library hope to create the seas from a varieties of different way. How we can a Sino very custom in next to the particular cities they got I bought at the operations we can perform and how it performs. All those are prison. So that's all about this video. Flint's. I hope you enjoy the setting this video and see you next with you. 15. Pandas - Introduction to Dataframe: Hello, friends. Welcome back to the section on the finalists Ponders Library. So in this video, we're going to learn about one of the very important data structure of the Pandas library, which is nothing but a date offering The bathroom is nothing but the two dimensional are you can see And this is the concept of later frame in a pond, as has been inherited from Oracle Data Frame are our programming language data from So this concept has been inherited from them. So I helped created one sample data from here in a visualization form it so you can see that there are a total three goes and a four column started individual column has been identified with name column c one, c two, c three and I guess it's a B a C for actually, and the rule has been identified with R one R two and R tree. So in this video was create exactly the same data frame inside the panda slavery. So let's make some necessary inputs. So now we have one function, like a data for him and inside the data free involved first, real fast. The data here there are total 12 values we need to pass. And this is arranging a rose and a column for Mitt, so we will passed it as a list. Individual row has been a list within a list, so we will pass it like this is an inside that this is for the first row. Then this is second row and this is true. So first role will be something like 246 minus one, then three minus four minus one and a terrible minus 10 35 minus seven. Okay, data video in sudden. Now we we need to supply that. What does the column name? So columns. Physical toe like C one, c two, c three, c four. And there is another one argument we need to supply. Like indexing. It is nothing but individual whose name like R one R two in our three. So another argument like index are one. Let's assign it to the deer that common IQ running for the data from Okay, So we have successfully created this dear let's just displayed so you can see that exactly three rose has been created with four column individual to as a total four columns and each has been identified with the name R one R two, r three and individual columnists E one to C for No, let's see some of that crippled. What we can apply on the top of this leaf lets everyone to see very first few rows, so we have function like er head. But as there is on Lee, three rows are available. Head is minimum giving Firoz so it will just display hold a different Let's say we want to see just first Turow's so it will see just r one and R toe. Okay, so we have created business data ranveer with allies and well displayed this day. Cover him. Let's say after creating this data friendly nikto are one more column. Let's say, like column. Lee will be something like New column Let's put the sea so column named usability. Let's assign it to some constant. Well, you like a six or next seven. Okay, now let's just displayed again so you can see that new brand new column has been generated , and it is our upended had dined off all this column newsy and all value has been seven. Let's see a rather than a sign in the constant value. Let's see, if you want to assign some value, it is computed based on some other column. Let's say the F C one plus that's a fight. So everywhere. Luna column C one has been added. Phi Tau individual element of columns even, and it will assign it to the new column C. So let's just displayed so you can see that plus five becomes +73 plus five committed minus 10 plus five become minus five. So this is where we can add a new column. So we have seen that how to add a new column? Let's see how we can delete the new column so far that we have function like a dropped. So let's say we want to drop a column like whatever the newly created column like new C. Now this argument is not sufficient, because data from cannot understand that whether you're particularly referring toa individual lows or individual column. So for that, another argument like access so if you are providing access is equal to zero, it will reference to the euro, and if you provide X is equal to one, it will reference to individual collar. So let's provide of one and let's see. New See column has been discarded. Let's see by putting accessible to zero so I can say it will immediately throws the error that new see label not contained in access because by default this excess is equal to right now. Zero. So it is referring to individual inaccessible particular row and this new see it didn't find into particular role index, so it has immediately throws the other. Okay, now let's say we want to drop like our one so we can say we want to drop arm. So in this case, we need to supply like access is equal to zero. So you can say that our one has been dropping. Our point are three only limits. Now let's just display this DF again so you can see that we held drop per one column via Drop windrow, but still, after this plane is the ever again, it so is the full isn't all credos in all those fight columns. So why this happens? So along with that too argument which roar column unit to supply. To get the litter and access value, we again need to supply one more thing like a in place in place is evil. Toto, now in place is going to throw indicates that whatever your existing the F four data frame objects out there inside that only it will do all those kind of modifications. So if you do this thing, it will actually affect the this Dear Faymann, and if by default is in place, is equal to falls. So what happens that in case off the simplicity called falls, new data from has been generated? Whatever the newly has generated, a Dublin has not affect over or is null data from the air. So let's just put this in places a call to through and let's execute it again. So after doing this thing, it has not returned a new later frame because it has done this modification on original data for him. Now let's just display this in the U. F again so you can say that new C has been discarded. Same way let's go with this hard won and we'll put like in place and let's just displayed the F again. So are one also has been discarding. So you can say that by putting this in place is equal photo you want toe modification in your own business data offering. So we have seen that how to create the data frame? How to add some new column or ho toe drop particular rows and columns. Now let's see how we can select this particular rose are how Then we can select a particular column. Okay, so that's a selection part. Now let's just display this the year first so you can see that there are two rows and we have a total four columns. Let's say we want to see just Sito our second column so we can just write like this. So minus four and a three. So this minus four and three has been visibly. Let's say we want to see C two and C for both, so we can supply it C two and C full as a part of list so you can see C two and C four so minus 43 and minus one GMA minus seven. Okay. And along with that, there are another two functions out of debt. Those selecting particular rose and a particular column. One is the function, which is nothing, but you want to grab some location and the other one is I look both is having on Lee. One difference is that in the case of this lock function, what you can do, you can supply rose on a column. That video you want to select and which column you want to select, based on individual names. Off particular Rose and a column coma Been separated for Rose and a call the same thing you can do for I Love. Also, you can supply this so on a column. The difference is that you can reference individual rows and columns by endangered value, so it builds on in cager value. Here you can supply exactly the same name it has been given to particular ruin, particular color. So let's see how we can use this lock. And now I look function. So let's just displayed is the F. Now let's say we want toe grab this Arto is a starter and we want to use this lock function inside the locker. We will pass it like art Tacoma. So Roe v want to grab his heart and all. Four column, you know, need to grab that Sawicki will keep, like, a cooler. Okay, I guess it's typo. Really Tours supply the square brackets instead of the surrounded records. We need to supply the square big so that this first part indicates what are the row unit to extract? In the second part here I open the column that meant all column. We need toe extract so we just need to extract our to so you can see that it's a real given a column. So it's our Toto are treat will consider. So let's not put this colon so you can see that are too has been extracted three minus four to win minus one, and all column has been extricated. Let's see, I want to put Laker C two and C three only. So we want to grab like a minus four and two inside the Rose Arto Secret, its ah, minus four and two, as we've actually so this way, with the help of this lock function, you can directly address your particular room with whatever name has been given as a next and individual column name. And you can grab a piece of data is nothing but this slicing. Now let's see how we can use this. We have not. I look, so let's say you want to used like we have not I Look, we want to grab this R three r three is nothing but the raw veggies index with the number zero for our two and one for us. So we grabbed this one coma in Alcala minutes to grab it. So it has grabbed his minus 10 35 minus seven. Let's say we want to grab justice. Even so, Stephen will be something like 0 to 1. So upper bound is always included. Someone will be excluded. So only zeroed collonville cup and fast. Oh, so this is the 1st 0 and zero column. So it should return as a minus 10. Okay, let's say we want to grab the value C two and C three so we can start with column number one, 23 so one and two will be included. So first column will be this 1 62 which is having right next number one. Because indexing here starts from zero and second will be seats. You can say that three and fires came as a return. So in this video, we have seen that hope to create the data frame, how to select a particular piece off elements based on some rose number in a column number . How to create a New Qala Moroto Drop Particular column. Andrew So that's all about this video friends in the next video, we'll see about some more functionality while you can apply on the top of this data frame object. So I hope you enjoy descending this week here and see you don't next studio. 16. Dataframe - Index, Multiindex: Hello, friends. Welcome back to the section on data analysis with help Find us. So in this video we're going toe extend over discussion on our data from object. We'll see some more concept associated with the data from, like, indexing or changing indexing how you can create a multi index. So let's see. So I hope they can have lashed produced they can frame only. So let me just run it and you can see that there is a three row for column they do every year with C one c two c three c four column name and our when our plight freezing. Next thing. So if you'll just right like index indexes are likely next object, it is having a value. R one R two and r three. So individual, though you can access with R one R two and R T. Let's say we want to resect the index, and we want to make a index starting from zero. Okay, so you can see that our when our to our three has been discarded as index, and it has created a new column like R one R two and r three. It is having a name index but your original index Now it's not referencing with our one. I'll put an R three, but it is referencing with 01 and two. Let's say we want to changing next thing. So how we can do it? Let's say we want to make this seat three column as our indexing. So we have a function like sad index and inside that we will pass the column name like Citrus, and we'll make it like in place through. So what will happen that our original data from the U. F only will be in effect? Let's just sprint it so you can see now the secret become now Index and C one, C two and C four color become, as is will. Now, let's say if we want to grab some information like Prince A Sito. So this column we can extract it like this, but with the help of this lot function, let's save you want toe? Grab a second room. Let's first displayed. We want to grab leg, say second, So second row has been indexed with the new values, so we light like it too comma and all columns. So that's why colon So you can see that three minus four minus one. Okay, so there is a second, Let's say from second run waas 2nd 5th row. You want to grab it? So my understand three and minus. So everything you can index it with value, like 62 And if I know if you apply, let's say this index had trouble. Don't not data free. So now index has been changed. So this in major 64 index object and it's where lose at six to fight. So individual low for second and third row, you can at a forensic with the index number like 6 to 15 So this way, in a data flame weakens. Recite the index, we can change indexing mechanism. Now, let's see about the another concept. Laker multi index. Know what is multi index? So for the explanation purpose, I hope created one dummy data for him, which is having a multi index, actually. So this is the data from my hell cooperated. I'm just pay steady. So did the tougher him. Indicates that individual students performance at a class y. So you for individual to class like C one and C two and inside the three different subject . Math, science and English. Which particular student has got the maximum score? So let me just execute and display several Understand more. Okay, we haven't defined his and P and P's nothing but the Numpty library. So you just need to import this number I live in San. Let me. It's important. So I'm just queen hooking. So let's just bring this year data frame object. So this is that in two different classes Stephen and Sito for three different subject math , science and English. Each other student has scored a maximum and year. Our indexing is like 012345 for all those six different stolen. So if you apply this in next sacred book on the top off this later frame object, it says the range indexes, starting from zero, six and six, has been excluded because Upper Bound is mostly excluded. Nina by 10 programming. Now let's say we want to make this a glass, and a subject has over indexing Nichole's. In our literal example, we have seen only single well column. We were making it as a next. So, with the help of that particular value in that particular column, we were referencing to the whole room. But now let's see, with the help of combination like mats, even formats Ito English see touring receiver in this kind off boo primary key. We want to make it for our table over data frame. So how we can do it. So in that case, we need to go for the concept Laker Multi index. So for that, we can use the same function, like sad Index, and we will make it two columns like 1st 1 is a class and under one is a subject. Both of them will make it as a index. So you can see that now completely over their differences religious and has Big Jane. So it becomes like, see, one class has ah, three different subject math, science and enlists. He took plazas or has a math, science and English. And for all these three, which student has got done? Maximum score. It has been indicated. Now let's say we want to grab some data based on the combination off this class and a subject, because now we have, er indexing mechanism like the f, not people. Flight is index. You can say that. Okay, we need to put like in places equals true, Otherwise it won't take into effect. OK, so you can see that now. The multi necks are there, so levels are seven Cito and under level is English maths and science and indexing. Name is letting but her classes and a subject. So now let's grab some data. Let me just sprained it before that. Let's say we want toe grab all the data corresponding toe class. Even so, we'll use the same of a lot function and we will supply this. Even so, you can see that all three subject corresponding to class. Stephen, which student has scored a maximum? It has given, Let's say we want to grab for classy too. Again it has given for Class C two Max Science and English. Which student has scored maximum 58 71 16 and corresponding to subject Max English and science met single since sense. OK, so it has written another data frame because it is like a date, a friendly dinner day covering, Let's say for class, Cito, we just want to see which student has cord maximum in just the subject, letting lists who again, we need to apply like a law function from the top of because it has again returned this data offline and this data from again, you're supplying our human desert English so it will give just correspondingly Cito and English so classy toe and inside the classy toe. Which student has scored maximum in a subject Inglis. So as fighting it should be 71. So it's correct. It's asked by any several different. So this way, if you want to make Multi Column as your indexing, it's something like a two column become a primary key to access the data that is nothing but he Q and concept up our relational database system. So in this lecture, we have seen better how to resect the index, how you can change the index, and we tell about new index, how you can grab the date of your sin, the moat. How to create the multi next A system and by multi indexing system is required in each other scenario. You can take advantage off it and how to set this multi multi indexing mechanism and how you can grab this data. We tell pop this lock function and money people time you apply the lock functioning. Grab this particular later. So that's all about this with your friends? I hope you enjoyed listening Dispute here and see you do next with you. 17. Handling Missing Data - dropna, fillna: Hello, friends. Welcome back to the sectional data analysis with the help of ponders. So in this video of year, going to learn about how to handle the missing data. So for that purpose, I hope created when data from it is having lots off missing values, that so in this video, we're going to learn about how you can handle those missing value. What are the options you have available? So let me just print it and see what distance NATO frame looks like. So you can see that there are a total four rows and a four columns, and there are many more musing values that then on all missing value has been indicated with an end and then stands for northern number. So what we can do? We can think from the rose perspective or from the Colin prospecting. Now, if you see particular column like a C, all values are not a number. So it is not adding value while creating the model on the top of this data or, if you see particular rose like a four through which is indexed by the number three. All values in that particular rules are not a number so we can just simply discard it, so one day you can discard it. But there are some columns like fourth column, which is indicated by the number Me it is having just a one value of it is not a number. So in that case, resident discarding particular column you can go for the feeding value. Or it may happen that rather than discarding or feeling, you want to replace some number of it. Other number. So let's see all three approach one by one. Let's say we want to discard the value. So for that in a pint, as we have a function like for a penny drop, any means you want to drop, which is not a number. Orbit is kind of missing value in your data free. So the one argument we need to put you want to think from the column perspective or rule prospecting. So let's say we want to think from the column perspective. Support that we need to provide access is equal to one for the column and for the role excesses. Zero. So you can see that although has been discarded immediately because every single column has at least one value of it is not a number. So by default it is assuming that even if at least one billion has been not a number just discarded particular call. So for that we have another argument like how how you want to discard it. What is your strategy? So let's say we put any so any of the number is not just discarded. So this is Executive Samos without throwing the argument, like how visible to any, Let's say we'll put argument like all, if all values and that particular column is not a number. Generally, please discard it so you can see that the column C has been discarded because all values in that particular column only are missing. Remaining all three columns A, B and E. At least one values it is available. That's why it has not discarded. So this is this true strategy. You can vote on it. It's like how all well you want to discuss or even if anywhere, user, nor the number you want toe discard it. Let's do the same thing again with respectable room. So here we'll put our human like X is equal to zero, so you can see that all true has been discarded because by the fall, how is equal to any because every single row has at least one value of it is not a number. No, let's change our strategy like how any so, if any off the values are not a number, at least one will use are not a number across the single row. Then it will discuss, Let's put it all, and let's see that is it so you can see the large throw has been only discarded because it is having all values, which is not a number. So this way we can discard particular Rose in a column in a Find us and whether you want to discard or whether you do not want to discuss, that's up to your skills and kind off model. You want O build kind of data you want to build, whether particular features or particular record is how much important to you while building your modern. So that's the kind of philosophical question, and based on experience you will guide it whether you want oh discarded don't know, let's say something like courts or some particular features are very much important to you . In that case, you do not want to discard it, but you want to interpolate the value. So in that case unit toe, fill the value with some value in your data free. So let's say we want to fill all the value off Northern number with some other value. Let's say we'll put like a constant Let's say it will be a minus 10 so you can see that all the value which is having a northern number has been filled. A Peter Value minus 10. Okay, but does it make sense to put such a kind of constant value? But we can apply some little bit smarter strategy on the top of it. Let's say we want toe fill the value with some mean value of that particular color. Let's take up be column. So will apply. Just Feel and Yana be calling, and we want to fill with the value. Like, what is the average value of that particular column? So every value of the column B is nothing but do plus four. The right way to it is nothing but the average value. All elements available in a column. Bees nothing but three. So let's just apply mean value of that particular. Let's just first find a mean and then we will apply so you can say the mean is equal to three so that exactly three will be this. An end will be replaced by the number three. So again, see that only second column has been into effect. And all the values which is having a northern number has been changed with value. Three. Now let's say you want to apply are rather than discarding some value, all rather than feeling some value. You want to replace some value with some other value, so we have a function flight. Replace now with what value you want toe. Replace with what value. Let's say whatever the values are available. Zero. We want to replace with some value. Let's say the regional value. We want to put it as a first targeted. The second argument will put it like that was a Jiro earlier, so you can see that the first rule. The Last column. Zero Kalmadi Plantain zero. So it has all the values which is having a value zero where it is being replaced with. That was a Jiro year. Let's say we want to replace this is not a number, not the number has been represented by this And be dot n in and lets us write it like De Rosa missing values. OK, so wherever there is a not a number, it had just replaced with a missing value so missing while you were, there was a missing value It So all those kind of strategy you can apply Violeta feeling or by discarding particular who, in column or while replacing the particular rose, Our particular column You want to apply? So in this video, we have seen that how you can handle the missing data tell about three functions are available in a pint us which you can apply on the top off your data from object. It is a drop. Any is a feeling. And under one is a very police. So that's all about this lecture, friends. I hope you enjoy listening this video and see you next video 18. Grouping data: Hello, friends. Welcome back to the section on data analysis. We tell Pop, find us. So in this video we're going to learn about two concepts related to the data frame in a pond. US. One is the grouping off data, and under one is a applying custom function on particular column or whole later frame. So for the illustration purpose, I hope created one data from this data frame is about individual players for four months in one particular tour coming. And this data offering indicates that where the particular person is sex, male or female, which particular team it belongs to. What is his performance and what is this? Eight? So let's just execute and visualize the data from okay, so you can see the players name. I would just cap it like P one p two p three up to be 30. So we have total 13 players, and each player belongs to some particular team that the name is like. Am I really hara? It's sex. How much scored? It has perform and what is the age of particular player? So it's a player database. Now let's see what is group. I mean, let's say we want toe group this player across the team. So am I. Belongs to Oh, there is a total for player be one Tito B six and a pillow, and we want to perform some more pleasant related to just, um, I only or we want to group despair based on their sex or particular players is in a meal group are it is in a female group, and after that we want to put form some statistical, a logical opposition on the top of it. So before they let's start with a very simple one like let's supply how many unique teams at their total in the state of is so we can apply the function like Count, So they'r totaled 13 that court Saturday. Instead, off counts Beacon Light like a value count. So it says that there are a total five teams out there. An individual team has a total four player M I T. T is a key player key, and there is a three player are as a player and lady is a one player. Let's apply some more functionality like a unique, so this will give the total unique teams are available and my DDR agility and gala. Let's say we'll put like an unique and unique is nothing but how many totals. Unique teams are there. So that way we can apply, like a count and count a unique team. Or how many unique teams with Dell pop and unique function. And there is a important monitor. Value counts. So where you can't will give genetically the hist o gram. That am I. As a four players City has a repair. That same thing. You can apply for sex wise also. So sex toys that are totals seven. Player belongs to Meal Group and six player belongs to a female group. So this way we can apply this unique functionality on the top of particular column. Let's see group by. Let's say we want toe group by particular team so we can write like a group by and we can apply team. So based on team, it will do this grouping. So let's assign it better by being, and it will return the data structure like the frame group by object. I play tough removal by uptake. Okay, now we can apply lots off different function on the top of this biting, So let's say we want toe make us some off all school so you can see that by team leader he has scored 69. So it and what it has done that it just accumulated all led am I into different, different groups. And for each of that group, it has just calculate that the some of the school. So for M I 54 plus during the one that's 51 plus 87 big cyclic well in tow 230 same age. Also, it has summed up, but it doesn't make sense. Toe sum up the age. No, let's a play the another function like a means. So just do the grouping off pace Dunder thing So mean value off this needy team for the scores 69 for they ate it during the six for GT Score is deeply but ages 29. Save me. Now let's do this Grouping is done the sex. Now you can observe wanting that when we applied this mean function or some function he just calculate. Based on this course and age column, only he didn't go for the player and sex, and it throws there. So pandas is enough smart debt. It automatically takes care that we had a planes supposed to some function or a mean from Genesis related to the numeric value. So that's why it has just applied on the about. This scored an agent. Now let's do based on sex. So let's to sex and we want to do the some. So in a male group, the score was 330 but in a female group to score is 3 69 total ages 1 48 and one for 1 80 Fine case of me. But in case of this mean value makes a lot more sense compared to some. So female group has a rages off 24.66 mail as a 26.42. Let's apply some more functionality. Laker leave not group by. Let's City, and we will apply. Hold those juristic like describe function. Now what does describe will give us all those statistical information based on the grouping off them. So it has group. This stream individually because of lady, contains some portion of record, and it is applied discount mean standard deviation, minimum value, maximum value for 25% of peoples, and it's 75% for attack, particular age and for school. Also, it has done the similarly same thing, so it has doing a lot off varieties off statistics based on the grouping Off team. Let's say we want toe do or get the same statistic based on the sex. So for the female, a discount like a six total 36 female candidates are there based on the eight and the average values 26.66 the standard deviation from the meanest 6.0 e. So that way you can apply this grouping. You want a group based on some particular parameters, and you want to perform some logical operation and statistical operation on the top of it, you can use this group by functionality, so that is all about the grouping. Off data have you can group the data in a bind us. Now let's toe another thing, like applying custom function. So what is applying? Custom function lets us observe this DF once again. What is applying customer? Let's say some people have scored like a fib before. 21 are we want to create. Let's say another column are we want toe change this players with, uh, we want to give some gift to display. Let's say those were scored more than a 70. We want toe. Tell him the congratulations. So let's say this player is 96. So we want to write like a congratulations before and whenever there is a scores less than 70 we will keep that player name. He wouldn't be Toby three, as it is, so let's create how we can do it. So for that, let's create first of function. So let's define a function. A very simple one, like Congress function. And we will take access argument. Let's say, if X is greater than let's is 70 we will just return. Congratulations. Plus, we'll just live in the Congress. Chelation okay. Else will just return a blank. Look, we just need to put here for lunch, okay? So well defined A function like a congress now how we can apply dysfunction on particular color. So for that, we need to apply this function on score table, and we have one functions available in a partners like apply. And inside the play, we will go with Laker. Apply this Congress function on the top of it. So what it will do? It will scan through every single rose off this core column. And for heat score, it will just call this Congress function and argument will be in element contained inside this score. So scored, 54 will go and it will check whether it's greater than 70. It is immediately Cruz return her congratulations, otherwise it will return a blank. Okay, now we want to upend this Congratulations. Before this, players need B one b two b three people. So how we can do it? We just supply this thing. So you can say that just by applying. We got to know, did the person or player which is having a next 38 10 and 11 is called greater than 70 sort ? 33 8 10 and 11. Now, for all this four players, we want to put the Congress situation in a column player. So how we can do it? The same thing we can apply will just redefine our clear column. And this is nothing. But it will return us the congratulations. Thus we were playing the same thing. B f prayer. So what it will do This portion will just return. Congratulations Ways don't whether player as score created in 70 or not he s scored 78 Villagers threatens Congratulations on the rise. It's written blank and it will be perfect. Stow the original P one p two p three I need So let's just execute it and let's see. Okay, so we got this. The player number in next 38 10 and 11 Got the congress elation because there's course 70. So this where we can use this apply function like you can create your own custom function in it. In the apply matter, you can pass on dysfunction and individual rose. Our individual element of that particular column has been applied dysfunction. It will return the output based Dunder largely corrected inside this function. So in this video of you seen about how weaken group the daytime, how we can apply the custom function on the top of data offering particular column or data from So that's all about this video friends. I hope you enjoy listening. Listen during see you don't next with you 19. Read, Write .csv, .html, excel file: Hello, friends. Welcome back to the section on data analysis with the help of binders. So in this video, we're going to learn about how we can lead and light device. So if you have a let's say, data available either in a data frame object and how you can store it into P different kinds off five like a Nazi s we dot html or not Excel file. Or let's say you have a date available in three different form. It's how we can read this file and put it into later frame object. So in this video, we are going to learn about this two way process off reading file and a writing five. So for the illustration purpose, I hope created. One simple data frame, which is having a very simple three, goes on a two columns and individual row has been index Twitter Name R, one R two and r three. So let's see how we can store this file into dot CS three for me. So for storing into dot CS, reform it be helping you in a one function like to see a sweet and inside that legis pass it a very simple argument like file name, which you want to get stole as a dot CS three. So let's put it like a sample CS dot CS Wi Fi. OK, so we have successfully story no less. Just see so you can see that one file has been created. Sample CSE. Let's just open it. So this whole data frame has been converted into Nazi esque reformer, and it has opened in tow. Microsoft Excel file So you can see there are total three rows in a two column an individual who has been indexed with the name R one R two and r three. Now let's see how we can lead this fight the same thing again in tow. Another data from Let's Now call it the F one. And for that we are going to use the function like a real sees. So sample CSP Nazis. Wi Fi will be inventoried. So were successfully really. Now let's just display so you can see that it has successfully. But our First Index column has not been identified, so we need to provide one more argument like index column issue, because Gerard column is nothing. But it's not a particular column, but its index column so you can see that now. R one and r two and r three r indexing and so three Hello all and 34 through. He's nothing but in the begin rose value, and it has a total local. So we have seen their how we can create later frame object out off this Nazi s we find. And if you have data frame object and how you can convert it into dot C is refined. Okay, Now let's move to another kind off like Lord Extremely. Let's say I hope kept you here off when you are in, which is suing the void. Bologna at least Wikipedia. Pete. Now we're going to read this speech and despite us, library will automatically capture whatever this HTML tables are available. It will capture this automatically and put it in tow. Data frame object for us. So let's start so far that we have a function like read html and inside the Reid HTM, and we will pass this year while and let's assign it to the DF. Okay, so now if you see about the type of decently, if you can see it's a list, it's not a date offering because why it has given a list Because there it may have wondered there are more than one tables that may be available, so let's just assign it to the list off device. Okay, so now this list stopped. Label Okay, it has given list now. Individual list. Let's extracted. So let's stop Chico so you can see it's a table and it's nothing. But the type is data from. If you're play like a length, their total four table it has extract. Now let's see another one, like so against the all those below Analyst has come into second David like a Bill Gates is on a number one Carlos Seimas on a second number, which is having a network 77 billion in a time of 2000 took. So this way we can read the file html file and grab whatever the HTML tables are available . Terek in Jindo find us library, and it came as a list for me. So it's a list off all data frame object, an individual element. Opted Particular list is nothing but a data. OK, so we have seen that how we can grab the data from extremophiles. Okay, so Let's Moto another kind of five Laker Excel file how we can build with the Excel file. Let's say the same day Katherine, which we have available earlier like a r one R two r three. Let's say we want to story doesn Accel. If I So how we can do it so well. Just apply like toe except on the top of this data for him. And let's give the name sample Excel file. Excellent. So let's just save it. Okay, so it has successfully sailed. Now let's just see an open it so you can see sample excellence has been created. Okay, so r one R two and R three and hello tro two or 3 34 through. So it has successfully created this excel by the same way we can read this Excel file. So in this video, we have seen that how to deal with this excel file dot html file or not see its refined. So that's all about this really difference. I hope you enjoy listening this video and see do next with you 20. Visualization of data with pandas: Hello, friends. Welcome back to the section on data analysis with the help opined us So in this video we're going to learn about data visualization, but find us a window. Find us is using back and software like a Mack totally library only, but with a very few amounts off holding, we can visualize all those basic parts like area charge or line chart box Plourde Ben City function or scatter plot with Dell Pope Pandas Library Very easy. So for the illustration purpose, I hope created one data from which is having a total three column and 100 total rose and individual column has been identified with this even C two and C 33 So let's just with July's couple off a few records. So total there is a three columns on 100 close. Now let's see individual chart one by one. Let's say we want to blot the area, Chuck and we want to apply light on the top off. Let's see even column. So there is a one common Syntex for blotting the chart. In case off this binders building, visit logician function like you first provide on video later, you want to put the plot after the night is a common Centex like a block. And then your play the name of the plot. Okay, so you can see that it has returned. One Mac blood lip exists object, but it does not display the graph. Now, if you're using this Jupiter, not book. And if you want to do is less all those crap built in inside this Jupiter, not book. Only then there is about one thing you need to apply, like percentage Mac broadly in line. So which indicates that you need to plot all those things inside the Jupiter. Not because you want to get display, although scrap inside your Jupiter, not book. If you're not using this Jupiter, not book. In that case, you can just write like a plt dot So So that will actually display Grab in another. Bingo. No. Let me just execute this again and it will display the graph. OK, so you can see that desert area attack. So every single value which is in a c two 65 82. It has displayed it and it has covered up with the line. Okay, let's see some other charge like a bar chart, Let's say, will apply on top of this. Even so, Lord Bar. So it is displayed this bar, but there is a too much We lose that available, so it becomes a little bit clumsy. Let's say we want to display same thing in a horizontal bar chart. Okay, so it's in a horizontal weight has displayed Okay, Now let's say we want toe display like a same bottle. Check on all the column so large, like instead of just the one collar so you can see that it has displayed for every single record it has displayed, the three values are available. Oneness even want to see you and corresponding to minus 63 So it has displayed the Bartek. Any individual recalled as having a total three, will you? Same. You can apply it here. Laker for isn't all part checked. OK, so either you can apply this blotting functionality on particular column or group of column , or even entire this data free so you can see that just by putting a lot of blood function and after that a plotting function name, you can just immediately grabbed the visualization. You do not need toe right have a lot of I would like a Mack bluntly been all those things. Now let's say we want toe Visualize the line, Pollak. So let's just apply on a C three and then block. Okay? So you can see very fast. You can visualize it So all those 100 record has been displayed for the C three. Now let's say we want to see the box plot. Now what? This box blood Wilken, give us marks. Blood will give us that for all individual column. What is the average value? Where is maximum distribution Life? What is this? A mini momentum Maximum value in just one single political. Give us so we have a function like a box. So you can see for C one c 23. It has displayed three different dimension. One is that to see when it is having average will you somewhere around four. Fire 47 somewhere out. Same way for C two and C three and this is the maximum value in this is minimum value for all this individual column like the C one C 12 city. So this way box blood just by writing off very few lines. Are you in just within a single line. You can just start plotting your functionality. Let's say we want to visualize the colonel. Lindsey, off particular, like Lexus, will apply on a seat only so you can visualize it. Clearly. Support large Katie. So it's a kernel density because displayed it seriously. Very nice way with just the one line off court as display the carnal best city estimation. Now let's say we want to apply like a scatter plot or scratch. It flows in the sense, Doctor, let me just utilize the vehicle. Let's say we want to visualize c one and C two column in a dotted formatting to diamond snow space. So it is something like on a Hot isn't Alexis. We're trying to plot X axis this see one column and are vertical axis this c to call. So let's just displayed. So we'll just apply this Clark on the top of this year and then Philip leg a sketch and incite the skater will pass the toe argument like, how isn't Alexis? You can take it like a see even column and work legal access. You can pick it later, Sito. Okay, so you can see that C one and C two has not very much highly coordinated. It's not moving toward a particular direction as it is a random date. Iogen Richard. But for corresponding C one and C two, it has just put the dot that is nothing but the scatter plot. Now let's say we want to visualize like a sta gram how we can see so wait can justly split this instagram photo. That's a column c even plot twist. So far, all corresponding value between zero. And let's say this is, I guess bring Ben. Is he doing 10? What is the frequency between 10 and 20? What is the frequency? So number beans. It has divided whole things in tow. Ah, 10 differently NGO 10 10 to 2020 to 30 30 to 40 this may. Now let's say we want to increase this. Been size are like let's say you want to put Ben's like 20 focus. It has divided whole range between 0 200 for the column. C went into 20 different place, so 0 to 5 fight 10 and took a plane kind of that way. You can even decrease on immune increase or so let's put it like a fight. So it has divided the total 22 42 60 to 1800 so that we can visualize this history number. So we have seen in this video a lot of different plotting functionality with just a single line off Corina. Find us building Brazil additional really would like just talking about scraped a plot. How you can display the density from sharing a box plot line chart mile charge. Are you an 88? So that's all about this video offense. I hope you enjoy listening this video and see the next video. 21. Introduction - Matplotlib: Hello, friends. Welcome back to the new section on data visualization in a fight on with the library. Mac bluntly. So what is Matt Blood Lip? It's one of the booty plotting library available in a fight on, and it's a mother off all visualization library available in the pipeline like data analysts library like a finder's building visualization. Or, if you say, like a statistical with villages and library like a Seaborne, are all built on the top of this Mac bluntly bribery. And in this section we are going to learn about each and every nook and corner of this matter bluntly property, which is going to become a very helpful for us for the data visualization while will build , ah, model off machine learning. And this is one of the library, which is completely inspired from a Mac lip plotting package. So what other topics we're going to cover in this video? We'll start with a very simple, basic blooding go through the concept. Like what? This figure? What is access? How you can create the subplot, how you can assign X label horizontal axis label in sort, or why exists label how you can define a tighter legend changes figure size. Or you can play with a different color how you can save your graph. And what are the different kind of special plotting like history? Graham Plot a scatter plot or a box plots. So all those topic we're going to cover from the next video and was in this section how you can install it. Sofia already installed the foot, and I found a package that is no need to install it. Just import this Mac blood lip dot by plot, and we will assign it to the if it rains successfully deafness. It is already install on your machine. And if you have installed a mini conduct baggage in your machine, you can just install this with the command like Kanda, install Matt blood lip, and it will install this Mac bluntly package as a part off on a conducted division. If you want to go for the standalone installation off this Mac broadly package, then you can go with the pip installer available in a fightin, and you can just execute the commands like a peep installed Mac broadly, and it will install it and just import this matter blood Liebgott pipeline and give the leader's name like a B L T V. Now we want to get all those with realizes until we happened inside the stupid of notebook . So for that purpose, we're going to use this another common Laker, Michael Oatley in line. Okay, so what will happen that do toe this line? Whatever the busy ledges and we will do well can see inside the Jupiter, not book only inside our house around. So that's all about this with your friends. So stating from the next video and once we will start working on very fun, part is is nothing with visual addition. Potion off over data. So that's all about this video evidence. I hope you enjoy listening this video and see you next for you. 22. MatplotLib - Basic plotting, Plotting terminology: Hello, friends. Welcome back to the section on data Visualization Library. Mac bluntly. So in this video, we're going to learn about how to do the very basic kind of plotting. And what of the terminologies our associate it with the Matt brought Lib Library off plotting terminology. So for that, I owe make some necessary imports like a number library. Find us library and Matt broadly dot by plot as a P. Eltingh. And one more thing is that Matt Blood clip in life to get all those specializes and inside the Jupiter, not book on. If you're not using Jupiter, not book every time you need to write the function like plt dot So I will so you and that will open it in tow. Separate window, although sprouting ground. So for plotting purpose, we need faster data, so let's generate some random date off us. Let's generate some X Y and Z diamonds, and so will use function like, let's say, inside then P number library. We have functioned like a line space, so let generator data between 0 to 10 equally spaced 100 data, which is equally space between zero and 10. And let's create another true graph like, which is dependent on X. Let's create like a very simple trigonometry function. So NP signing. So it will take all those values off acts, and it will calculate the sign valuable. Let's create another one in a form off. Let's it costs booking. So let's just execute it now You can see the whole generated around 100 data point X, Just see pie. Same Levi. So let's start about the very simple, basic plotting stuff. So first for the plotting purpose Vini off figure. We need some kind off canvas where we can draw all those kind of exes or particular tax. We want to write particular graph you want upload. Have you want to display this crap? So for that, we need some drawing paper and trying to every canvas, and that is nothing. But we call it as a figure. So we need to create a figure so we'll just apply this figure on the top of this PLT function and that will return us figure to us. Okay, so we have successfully created the figure. Now the next task is we need to add the access. So inside the figure really put Texas like xxx Antibiotics. So we have of microbes like axes for taxes we need toe past the total four values. One is from the bottom from the left. And how much weight up the graph and how much height off the ground. So let's take from the bottom and from the left we want to keep their distance like a 0.1 is all related distance and which we want, like a one inside the full can must be want all our full graft to get displayed and other one is a high treaties. Lexie put money and it will return us as a excess. So it is X now once be God this e X so you can see that it has displayed a very blind craft from 0 to 1 from 0 to 1 in a boat that animation. Now all those plotting functionality, we can apply it here on the top of this axis. So e x has a one matter like a block and inside that we need toe plus like a X dimension on a wide emission. So we'll just possibly x comma by. So why we have a signed graph and there we have a Cassandra. So let's just display Sign one so you can see that just passing the X coordinates and of I coordinate it has displayed a very nice sign. CO for us. Let's display more than one graph. Let's say we want to display both of them in the same place, so we'll put like X commas. So excellent women, our horizontal axis, Mrs Common. But the vertical axis we want to change now it will displayed has a course in graphic. Okay, so it has displayed total to graph. The 1st 1 needs Siggraph. Another one is a co signer. Now let's do what other things we can apply on the top of it. Let's say we want to change the color of individual Graham. Let's say the 1st 1 We want to make it like a raid. So sign graph. We want to make it like a real So we have one argument like a C and inside did. We can pass Leggett lead and let's say the 2nd 1 because I love you want to make it like a green so you can see that we got this sign co. As a waiter and a court sine causa green. Let's say we want to make this graph a little bit more transparent. So how we can do it. So we have argument like Alfonso. Let's put the Alpha Lakers, you know, point fight here on. So let's go later. Zero point fight so you can see that it has given a transparency at a level of 0.5. Let's make it a little bit more darkens. List Looks nice. Let's say, for both of the graph, we want to highlight some daughter or some marker point. So if you see that particular signed up on every single point on this graph, you want to make some marker. So we have one argument we can write. Laker Marker. Let's say you want to keep the market like this. Sign in a sign, grab and for co sign grab. Let's keep it like any other thing. Let's keep space symbol X so you can see that this market for sign it is this one and father cost sign. It has displayed like a XXX, so we can visualize it a little bit better. Now let's say we want toe chain like a line with here. Whatever the line Without there, we want to increase it or we want to decreasing. So we have another argument, like a line with LW. We can keep it lined with, like a 1.5. Let's just experiment it so you can see becomes a little bit more ticker, I guess. Let's make it 2.5. Okay, again, it is more taker and ticker. Okay, so we have seen a lot off things. What we can play with that. How graph will be visualize. Now let's see some other functions like, let's say, we want to change the limit off the Zach sexes and a varieties So we have a function Laker set. Excellent. So we want to pass it, like from which range or packs toe, which range off X minimum brain toe the maximum lee we want to supply. Let's say you want to get display between minus 10 and 10 so you can see that the graph has been displayed on a horizontal axis between minus 10 intent, but we hold data available between 0 to 10 only, so craft has been displayed it a portion of Jiro 10 only, but the whole grab has been visualized in a form of X limit minus 10 putting. Let's make it the zero same way we can apply on a violin. It also so let's supply why? Let's make it minus fight toe fight. Okay, so you can see that the graph becomes a little bit more. Cite them because the whole Graf why axis ranges between minus fine fight. But our graph is range between just minus one and one because all those sign values and co sign values are varies between minus one and one. So let's make it between minus one and one. So this way you can change the graph. X axis ltda. Why Access limited? Let's say we want to or see that over the particular Greengard belongs to call sign graph in a raid graph belongs to the sign grab. So here there is no indication addict, so we can add a like a legend. So gonna like alleging So legend will tell us that particular radiograph is signed graph and green grab is cost sank off. Okay, so it has given us warning and there is no label objects, okay? It is so knows that there is no label object because we haven't defined any sign or co sign here. So that case we need to light one label here like this is the sign graph, and another label is like a course sign. OK, so you can see that at the bottom left corner. It is displayed like a near grab is a sign grab, and a green graph is a co signer. This way we can add the legend. Now let's say we want to add, like a X Axis title or a Y Axis title. How we can do it. So there is one matter, like a set X level. Let's say X label is like a from the physics point of view. Let's say it's a time and via label is amplitude, so it's kind of intensity, so you can see that's nicely displayed this time on the amplitude included on a vertical axis and a time on a horizontal axis. Let's say we want to keep like, a title, so this is a title we can keep like like say, this is my graph. Jupiter not cook. Okay, so it has displayed this very nice title there to be a given his argument to the site title function. My first matt bluntly graphene a Jupiter, not book came. So in this video we have seen that what are the matters you can apply And to make your graph even more appealing? More visualization. You can see there to how you can different parameters and you can supply to the this excess object to create your graph. More visualize is and powerful how you can put this limits on X axis y axes and the labors . What is the marker and how you can change the color off particular car? So that's all about dispute difference. I hope you enjoy listening this media and see you in the next video. 23. MatplotLib - Subplots: Hello, friends. Welcome back to the section on data visualization Library, Mac Bluntly Sina Last video We have seen about how to do very basic plotting and some of the terminology liquor toe plotting and Vo displayed a photograph in a single canvas, a single figure. Now, in this video, we're going to learn about how you can create the subplot. So subplot in the sense that let's say, we want to keep this sign graph in a course in graph in a two different figure on a two different canvas for two different graph. We don't want to get it overlap on the same figure. So how we can do it. So let's start it. We're going to use the same depth X, y and z what we randomly generated. So we have a function Laker subplots, development and inside that we need toe past like how many rows off graph and how many columns off if you want to display, Let's say if you want to display the program, which is having a in a one horizontal row, so one draw in a total Kalala, so once it is displayed, you will very much on the student. Let's create it. Let's use this first to cross toe so you can see that to cross to metrics will be something like this. Total tour and two columns. So total four graph it will display, let's say to cross for so Total Turow's and a four column. But here we want to displayed, like one draw and two quality. So total telegraphy want toe display. And for the argument, name is number up arose and number of column musical. Okay, now it will return the to object. Basically, so one is a figure in another. One is a access, so figure and access board it will return it. No in if you see like, let's say this type of this success so you can see it is a two dimensional Ari. So one element of that are a is corresponding to this first graph in the second element corresponding to this second graf, and we can apply all those matters which we have seen earlier in this nexus. We can apply now individually. This excess first element and exists second element, so let's just do Geo X zero is corresponding to this first craft and exist one corresponding to second round because it's a body now. The same matter which is nothing but the block we're going toe plan. We will pass it like same day will pass it like X Komazec so you can see that in both. The graph has been displayed in two different places. It is not overlapping on each other. X zero is corresponding to this four scrap. So let's say all those changes we want to apply like a color. Let's say we want to change the color like a late first craft to sign become now that in the second graf apply Laker green, so second will become green. So display all those property we have seen in the last video. You can apply in this x zero between taken into effect on this first crap and exists what will be taken into effect in the second round. Now, if you see if you want to display more than a one graph, this can must looks a little bit smaller. So how we can make this can was a little bit more bigger. So let's say we want to me little bit more widen in this cold graph. So there is a one argument here available, which is nothing but a figure size. And there we need to pass it like a topples that how much of it and how much Heidi one. They say We want a bit like a 15 and height. We won't like it, so you can see that Let's make this height little bit smaller, nicely visible. Seven. So it looks very much nice now, so height will a little bit increase and it is drastically increase. But that will be in a proposing off 15 gem seven. So in this video, we have seen that how you can create a subplot and on individuals subplot, you can apply all those kind of functionality. So that's all about this video friends. I hope you enjoy listening this video and see you next for you. 24. Matplotlib - Special plot: Hello, friends. Welcome back to the section on data Visualization Library. Matt bluntly. So in the last video we have seen about how to do basic plotting how you can create a subplot. Now, in this video, we're going to see about some of the special plotting like a bar chart. Oregon's a hist o gram box plot or a scatter chart have you can generate it. So let's start with generating very simple parter. We're not going to use that data of your randomly generated so we can create like X physical do very simple. Let's write like a 1.34 crisis. Early line and corresponding violence generated like so we held generated accent light. Now let's see how we can generate like a bar chart so we can use the same plt and we can apply like a bar, and we can just pass it like packs. So conceded corresponding toe one that is, a values three corresponding to two. Well, Lewis for corresponding to three values toe. So this American generate a bar chart and all those coloring and all those mechanism you can apply as argument to disbar function. Let's say we want to generate a horizontal bar so you can do it this way. No, let's go for another charge, like a scattered attacks so we can send it like a scattered. So what the scatter will do? It will take individually accent by, and it will just put a point so corresponding to this accent by whatever the values are available, let's say 11 entry so you can save one comma trees available here. So one minutes three So this were scattered shot can regenerating. Let's say we want to see the history graham off via exists. So historian we just need to pass in case of his totem. The one column only, And it will give us the corresponding frequency. So corresponding Toto. The frequencies toe corresponding to three also available two times four is also available . Poor times. This is like a coincidence, but it's available. Toto Times only. All those values. So it has divided all things in tow. Some sort of rains like a 2 to 2.42 point for 22.8. So 2.8 to 3.8 3.2 so that we're actually all those values which lies into this particular pocket too. That's why all those graph, we're finding it as a kind of similar graph. Okay, so let's see some other child like a box block. So to see the box plot before the Legislature delays, despite what other way use that they're in a bite. No, let's visualize this box pluck. So we have a function like a box plot and we will pass it. Plus why? So you can see that the average value of this all those value available, Lena, why? Element is nothing but corresponding tigers around up four and we lose lies range between 2 to 6. So this way we can generate the box blood. Okay, so we have seen in this video about how toe gender this box plot hist o gram, scattered shot, horizontal bar chart or normal vertical bar chart. So that's all about this video friends. You can go through the documentation of Mac plot lib library and you will get to know about their lots of other graph was so you can generate with. So I hope you're enjoying the sending this video and see you into next with you 25. Introduction - Web Scraping: hi prints. Welcome to the brand new section on Web Scrapping with my talk. Web scraping is one of the very important skill for the data scientists, the journalist or data engineer people. So in this section we are going to learn about what the Web scrapping is. Why we really need this Web scrubbing what other articles uses up this group scrapping What is the actual Web scrapping process we are going to follow? And once we die Minto Web scrapping part, we will see how we can extract ah particular title link image or individual back off the webpages through we're scrapping. We'll go through the complete example off How to scrap this? I am Lee B. Movie listing page. Then we will see another example of one more important and one of the famous consumer website as a practical activity. So these are all topics were going toe cover in this section, and for that, we're going to use a bite on beautiful soap library. So if you haven't installed this anaconda, or if the beautiful soap library has not been installed on your machine as a standalone installation, you can use this pimp installers and you can just fire the command people install beautiful . So by default this beautiful soap has been a part of this anaconda, so do not need to install it. But if you want to explicitly install inside the anaconda so you can just fire this common Condon, Stop! Beautiful stop. And if you want standalone in stolen fire The common we've been stood beautiful So I have already 10 on my machine. So let me just import necessary model. So I'm just important this Bs for and inside did I'll important model dutifully show. Okay missile successfully exhibited Commandant mints Beautiful soap library has been installed which is required for the web scrapping. So that's all about this video friend self from the next video. And was we'll see individually all those topic related toe web scrapping. So see you next, will you 26. What is Web Scraping: hi pins, so welcome back. So in this video we'll see what the Web scrapping is why we need a Web scraping and what other ethical uses off this Web scraping. So let's first try to understand what Web scrapping is. So Web scrapping is nothing but are extracting a particular data. So let's take example of this I am DB site. So I opened this IMDb website of it saw the top box office US movie during the weekends 24 to 26 November 2017 so you can see that all those movie has been listed as a accordion. Now, if you want to grab the data like, what is the title of this webpage or are, you can save what other links associated with individual course off this top box office movie off this weekend. What of the image you are a lot associate e. What other there begins are things out there so extracting those kind off data it is associated with different HTML tag? Because every weight page has been associated with HD image sort scored and out of that source score. Extracting this particular data is nothing but of Web scrapping now why we need a Web scrapping now If you see Google, Facebook, Amazon, this kind of company already providing that very p a. So with the help of that very baby, I you can grab those data. But there are some website or some company. They do not provide such a kind of very pay. But still you want to grab the data associated with their website. It is visible on their website. So how we can do it so you just cannot use the Web. AP A. That's why you need to take a help of Web scrapping. So grab this whole HTML pit and trying to expect whatever the important material important , particular tag important informations are there out object. That's why we need a rap scrapping. No, it looks like that you are stealing some kind of information. So what are the ethical uses off this Web? Scrapping his Web scrapping is completely illegal or it is not a single. So let's try to understand this thing now. As a consumer, I'm just typing www dot i am david dot com on my browser, and it has just loaded with the webpage corresponding toe. That particular address now what this Web scrapper will do that also exactly do the same thing like it just affects the content off this MTV dot com website and scrap particular thing. So there is no technically difference between whether humans are reading particular weapons or some Web scrapper related program are particularly downloading particular. You really are particularly maize from that particular you are. So technically there is no difference, and it looks like there is a completely vertical to use it, and there are some good advantages. Also, let's say some website it is a very specific to personalizing your account like a financial dope in our in a banking domain. Some website don't. I love you to download some kind off data when you long meaning toe your account. But in that case, you still want this. They turn into some kind of form it Laker Excel Form eight or C s reform it on any X file form eight. So, in that case of unit to do the reps Kevin So that is ah, looks like a completely tactical. He uses up Peps coming, but that is on the ah side. Also like a you cannot. There is No, it takes associated with that. You just cannot still the data. So let's say someone has already build up a large amount of the time. They'll get out those data from last 10 15 years and you are just scrapping this daytime you're posting toe somewhere else. So some other website or some other open black form where you just pose those the kind you're tryingto claim that that is your own data. So that is not completely accepted. 10 copyright and all those things will come into picture because you are trying to stealing data on you are trying to claiming that this is your own data, so that is not accepted. So there is a very much blood boundary between article uses, off web scrapping and unethical uses off web scrapping in discourse were going toe Grab the data from this particular you are in for the illustration purpose. There is no intention toe crap. The whole data associate it with this. I am living website. This is just for the learning purpose. So this is the disclaimer. I'm just posting here for the I am the Vedic. I'm just grabbing the data from this particular one webpage for the learning purpose in this section. Now what are the upside and what other downsides off this Web scrubbing? So let's see. I'll just make one more topic. Like what? Upside and downside. So if you see that if you start reading this big as a human, it went a good amount off time, and it will take a lot of time for you to capture some particular information out of this webpage. But instead of that, if you write some program in any of the programming language as a Web scrapple and Ted scrapper with millions off webpage, you know in the billions off webpage in a very sort span off time. So that is kind of upside, because that is a completely automatic process. But there is a downside also. Now let's say you helped build this scrapper and you want to grab this particular title and title is associated with some kind of development that we're going to see now further video . But after some time or after some periodic interval, let's say I am to be here, decided that I want to check the tag name I want to change. Those CS is associated with me. I want to change the class name object particular take on ideas associated with that particular time. In that case, you're Web scrapper will be completely broken. So that is one of the downside of this rap scrapping. So that is all about the Web scraping. So in this video we have seen what Web scrapping is why you actually you need to do this Web scraping wide Web scraping is required because you don't have a PS are available from all the website. What are they tickle use? What is the actual blurred boundaries is between article using and non ethical uses up this scrapping and what an upside and a downside to be hosting that is a one more downside. I forget to mention that once you start fetching data from the particular server that servers efficiency also goes down on. So that is upside and some of their downside related to grab scrapping. So that's all about this video friends. I hope you enjoy listening this video and see you do next video 27. Web Scraping Process: Hi friends. Welcome back. So let's see about what are the Web scrapping process. What are the steps are associate it with the Web scraping process. So in the last video we have seen what rep scrapping is so Web scrapping is nothing but extracting data out off this such statement pate. So inside the Web, scrapping or to make the Web scraper, what are the steps we need to follow? Extracted it out of HTML. You first need the HTML data or replicate who saw scored is returning html ceases or a JavaScript. So first us is toe download this ex Tamil paid. So let's me just write down Lord X TML were page. So for that you need a you are in. So when you give the particular you are ill, just download the estimates so scored its associated with them. The next house is located which particular element you want to extract? So let's say out of this weapon A. I just want to extract the title of all those movie or the Weaken earnings off all those movie. Or let's say the grass income of that particular movie are you are ill associated with a particular image. So locate in which particular part off this whole webpage you want toe grab. So that is what? The location off information off interest. Okay, Now you have located what part off this weapons you want to grab it Now the actual technical part will start. So for every single browser, either in a Google chrome, Mozilla Firefox or Internet expert or even a safari on a marquis just right click and inspector Element. So once you do this, think it will just pop up another window, associate it with that particular weapon, which is nothing but a developer console. So if you see that one particular information, which is of interest to you, if you just likely can inspect it, it will just point out you the actual estimates or school. Or you can just right click and see control you, which is nothing but the view paid source for every different individual browser has. Different sort codes are associated with that, but very commenting is just right. Click and see the source code. Now I proceeded Source code a horse or Scott is made of off a lots of JavaScript. HTML and CSS score out of that we need to grab this particular data. So here we want to grab the data associative detect title. So if you just select particular element and if you right click and inspected, it will just look attack particular element. And if you see the If you just click on a view based sauce, it will just open another window and it will solve the whole page reps or school. Now you located the particular element which is up impressed to you. So it is nothing but a locate. The tar part is located element, which is of interest to you inside html source Score. OK, so we hope located this particular. So it is nothing but the thre f link and was but regular. Let's say the titles are given the history of the link. Values are been given and exit. The name is Coco. So that is the part off this pretty lady's nothing but particular cell value off this particular table, and this class has been titled column. Now, if you want to grab this particular date off in plus to you, you need to find the very common pattern associative detect. So every eliminating estimate has been associated with some kind off classes, or either ID's associates of classes like a car moniker, similar kind of element and ideas like a particular unique identified associated with particular element or itself a tag name. So either tag name will be a development or table element or any other kind off E Bratcher pH area for image element. So based on that, if you have identified the technical part, so you in third party oh, identified the common pattern. So once we see the more example in a future video, you will get to know about Dick. What other common back unassociated with the element off interest to you now here we want to grab all the titles so title has been associated with the class title name. Now, if you see this, just this'll IQ now another movie, which is also associated with a title column class. So that is a common thing associated with all the title name whose classes titled Column. So now you're grabbed a particular data which is of interest to you. Now The four task is tech in your scrapper. Give instruction to your scrapper, for you can see the extractor, but do me all element. Bruce classes this title column So it will be something like, Give me all elements. Whose glasses? Fightin column. Now this class is always different varies from element to element. Website to website. So in this way there is a total four step process of web scrapping. If you want to grab any individuals, single element, or even a multiple element or more than one kind off element out of this I stammel pay. First, you need to download this webpage. You locate the information off interest to you, then locate the element inside decks. Timmons Sore score. Once you got this source scored inside the element which is of interest to you, find a common pattern. Once you find a common pattern based on that common pattern, you've given instruction to your extractor that give me all elements. Associate it with this particular Kyla batter. So that is all about the process off Web scrapping process. I hope you enjoy the sending this video and see you in the next video 28. Search Element by TagName and TagByClass: Hi, friends. Welcome back. So let's see about how we can extract the particular tag in this video. We can try to search for particular tag, name or bag by the class, or tag by the idea tribute. So let's type two sides. So before that are we need to make some necessary imports. So we are just going toe import request model just for the downloading off. Full tax from the particular you are and another one is a beautiful. So from me. Yes, for really important, Beautiful. This is the Ural, which we want to fit, so I'll just copy paste it. So let's assign if you are in tow well street and will use the request mortar to fetch the data, we will apply the get matter and will supply the you are Let's assign it to the extreme away, not get and see the content which we all received. So you can see the whole HTML code. It is exactly nothing. But if you see this view paid source, this whole source scored has skin. Now let's apply. This source scored toe the beautiful supply, Billy between nothing but hierarchical way. It will convert this whole HTML. So school, which is available in a tax form it it will convert in tow soap objects. So let's assign it to the extreme in sex. And which part, sir, we want to use it as a next her a second argument, the real supply. So we want to use the parts of like Alex Salmon and let's assign it to the soap objects or BS for beauty. So it's a beautiful soul logic. Okay, now let me just sprint it so you can see now it's ah, it has printed in a very beautiful fasten compared to earlier. What we're printer. It's terrible, not taxed. So this beautiful soap object is nothing. But whatever that the ITT has been available in HTML form it, it has converted into beautiful soap of the soda begins try particular element based on tax and particular or element based on some class or some other kind of attribute. Now let's say we want to search. Let's say the title the title is you can see here it's I am top box office space hype in space. I am leaving. So how we can search it. So if you see this source, score just type the Stichel so you can see it has named with stagnant title. So how we can do it in your different so so exist. Use beautiful soap object dark and we have a matter leg of fine. So which Bagby want defined? So this was a bite, just executive, so it has returned a full title. Now we want to just see the attacks inside that So we have matters available. You get tax so you can see it has expected the particular tax. So in this way we can grab the particular element Name. BaFin already know about the limit name. It says nothing with the title here we can grab. Now let's get some more element. Let's say you want to grab this stop box office us whatever has been done. So let's just right. Click it selected right click and inspect element. So if you just inspected, it does type off each one. Thank So let let's search for a gigantic now. It may be possible that title is already off level in on leave one, but this a joint, like maybe more than one, also possible. So in this case will use the matter like, find all okay. And we will supply, actually. Okay, let's assign it to the actual list over to an objects or Richman list dessert the least of objects. Or let me just say pretty true. So much money. It's one list. Let me spring, Dick, actually, so you can see there are more than one inch one. Ah, exiled there. One is a top box office us. It is nothing but this one. And there is another one. Also the leading information. The source for the AG detainment industry. So that is under the to actually tax. Either way, well, let me just search it here and let's just ready fright. Okay? See, this is a first statue in the leading information resource for and the diamond industry, and this is the second selection. Had a top box, obviously us. Now let's start for another living. Let's say let's search for this one. So this one is like a hatch for Let's apply the same thing. So it would such published and same way we'll just bring pledge for so I'll just trade over the edge for list. Okay, let's assign it here or search for Let's assign Bridgeport Okay, so you can see that a total one elements also available Lal etc. We want to see just the string available inside the edge vortex so we can apply the matter like str Okay, that's so so we have what other matters we can apply Get next So get bags again we can apply So weekend off number 24 to 26 2017 This is nothing but the here. So that is that for technical help grab Let's grab the title off. All movies has been listed on this page, so let's search with let's say, this CD elements A TV elemental number off really elements. So how African differentiate distantly element with other kind of PD elements. So for tech, we have our class available title color and inside the title column. Behold one always a funding. It is actually flanker available now If you see done the movie title also sector ivy, title color and inside that there is a one. So how we can differentiate disbelieve it other dd. So let me just use the same one. So So I'll just assign it to the ah Titlist title list and the tag name is speedy. How long it did. If you just see the documentation off this find all function, just pray safe tap. And it will. So that what other other troubles you can apply? So our director beauty is nothing but a class name is You can see here the class name is a title column. Okay, so title column so directly with drinking. Supplied, like in terms of dictionary reusable glass Portland, fight their corner. Okay, so we're successfully executed. Now let me this organic trip through this fighting title list, like spring this title so you can see all those TV elementary tests display all those DD Element, which has all movie names. Any feelings has been pretty loud. Let's say we want to see just contained or let's they will use the printing of the content won't work here. So let me just driving. Some of them are attribute, so I just used again to get X maternally. Okay, so we already state this all than movies name. So 1st 1 is a cocoa than just technically wonder. So here are so you can verify again. Big cocoa Justin league wander in this week. So this were you based on the class name. You can differentiate that particular TD element or any other kind of element and all those common element attribute you've been supplied as a dictionary. Form it in a second argument off this find all function If you hold same way, let's say idea so. Some elements are associated with Laker idly so you can just supplied here in this next department, they say Islay physical toe, kind of victory or any other attribute. It is not very common use, but some peoples. Are you using those red of the 90 or class some people use like a day car, so even just supply it like that and you can just crab. So, apart from fighting the particular tagged, isn't class ideal someone directly well done? A murder for drugmakers? Start available. Laker. You want to sort for what are the next sibling up that particular element? Or find a parent element off that particular element? What is the next element? Are all previous elements are available? Pile All know does next element you concert. If you want to find, let's say the Children up that particular element, you can just search for this fine all matter Stop. The matters are always really studied. Documentation. So this way we can grab all those data automatically with the help off directly from this webpage with the help of this beautiful So play. Very so in this video of you have seen that how we can extract the job is done The back name based on the tag, by class or by idee attribute. So that's all about this video friends. I hope you enjoy listening this video and see into next Really? 29. What is Machine Learning: Hello, friends. Welcome back to the second part of the course, and one of the most exciting journey we're going toe start begin in this section. What is machine learning? So in this video, we're going to learn, but a very simple example what machine learning is. So let's start it. So how many emails we get on a daily basis, either in our personal life or in a professional life, we get huge amount off. Email which of the emails are important and reach of the email are not important to us. How we decide whether particular email is very much important to you or particular email. We need to take this type of action so we create the rule in other G mailbox foreign over Outlook box. The rules will be something like if email comes from such a recipient like R one, R two and R T moving to some folder. Even if email has a title like a job, it's a very much important image because, as right now, I'm searching for the job. So it is one of the excellent example of automation that you create a rule and based on the rule system, will pick the disease, it on behalf off you. But what is the problem associated with such a kind of approach? One thing is, all rule creation is completely manually. So man usual creation. It's a quite Liberace tusk, and you need toe oblate this route periodically as you get new kinds off email that is not a part off your rule, so you need to update those kind of rule. So that is the problem. So can we think off something that is, Ah, head off automation? Can Such a rule can be created automatically something like automation off automation in a leered videos in its example of automation. But can we even automate the automation itself so that automation off our Thomas itself is nothing but a machine learning? So machine learning is just one step ahead of automation. So it's automation automation. Let's see what machine learning is with the help of one diagram and with the same example in the last example, we have seen that with our traditional approach, we keep creating rules and system will act based on the rule. But in case off this machine learning system, you supply all those email to the machine learning system. And it will create such a kind off rule which we all manually created in our last example, which it will create automatically for us. So that is nothing but a machine learning model. And when you supply this data or your email so the machine learning system that is nothing but a machine learning algorithm. So it's a best example off automation automation. So now when about the new email will come, you just feed this email directly to our model and based on rule, it will just act. So machine learning is one step ahead of automation. It is nothing but automation off commission. So I hope you enjoy listening. What machine learning is in somewhat layman terms. We will try to see more and more definition off machine learning toe grass, the actual concept up machine gun. I hope you enjoy the sending this video and see you in the next video 31. How Machine Learning system works: Hi, friends. Welcome back. So in this video, we're going to see about how machine learning System Works Machine learning system are like a training plus tasting kind off system. So training you can say like a learning happens. You fit the huge amount of data to the machine learning system. And in the training face, the loving is going on and vile the tasting. You provide the new kind of data toe test. Whether your machine learning algorithm, he's working fine or not. Generally, this training happens enough offline, more so when you have a data in offline war, you create the model of representative off your input data, which is nothing but the learning process, off machine learning system and distaste ing generally happens in a online more. So let's try to understand how this training system exactly box. Let's say we have a huge amount of data in a form off. Either images speech are attacks. So in a training face, first parties toe p process this data and extract features along with individual image speech or tax. Later, we have a corresponding labels are available. So this extracted feature out off this input data and label associated with this in individual image, we will supply to the machine learning algorithm and this machine learning algorithm eventually generate Ah, model our representation off our input data. And that is how the whole training system works in a gay shop machine learning. So how much he learning generate this kind of model are actually input re presentation off your input data. So let's a little bit more into deeper sense that how this model has been generated. So machine learning process is kind off a tray due process. So after every treason, he just keep learning new and new ting and it just providing better and better accuracy to represent your input data. It is nothing but modern. So same way you how input data in a form off image, speech and tax and you have actual output label associated with individual rose. Now you fit this input data to the machine learning algorithm and it will predict the output level now, at the very first pleasant, it may happen that your predictor output is not matching with your actual output. So what we do, we will just compare this to output with a predicted output and you will find the difference. And based on those differences, you will update your machine learning model. You will object to the machine learning algorithm so that whenever the next operation start , you take the next set off in mid speech or text later into a machine learning algorithm and based on this abraded model, it will predict the output again. You compared this predicted output with actual output and you find the differences based on differences. Argan, you are pretty so this is like a ongoing process. So every time you've encountered this actual output and a predicted output, find some differences. Based on those differences, you update this model so you can say this machine learning processes kind off a treaty process. So after every it listen, it keep trying to improve the system. It gives learning new and new things kind. The vein human learned in our childhood way we learn a number of different things. So that is how machine learning it play due process is on going on and after certain interval, you are really that machine learning system or machine learning generated model has reached to a certain level of accuracy. In that case, we will just wind up. So this machine learning algorithm passing, predicting output and comparing with actual output. And based on this differences, you are break this model and that is nothing but a treaty process is a keep going process. So this is all about the training face and inside the training. Chris, how actually this aggressions are going on now, let's see about that testing system. So when you have a model already generated, you fit this new case data to your actual model. It is nothing but the representation of your input data venue supply this new test data to your model, it will actually product the label off your new later. So that is how the training and testing system works in gaze up machine learning. So I hope you enjoy listening this video and see you in the next with you. 32. Different Types of Machine Learning system- Supervised vs Unsupervised learning: Hi friends. Welcome back. So in this video, we are going to learn about what are the different types of machine learning system at abstractly will in how many different categories you can classify your different machine learning algorithm, so there are mainly three types of machine learning system. One of them is supervise way off learning, unsupervised playoff learning and under one is a reinforcement kind off learning. You can further subdivide this supervised way off, learning in tow two different categories, like a regulation and a classification and discourse. Mostly, we are goingto deal Vitter algorithm related to supervise way off learning regulation, classification and unsupervised be off learning. So let's try to understand all of them one by one. Supervisory off learning Let's say we have been given a data input data input data as X and output data, which is in a form off label by and in case of supervised learning. Our objective is to find or predictive function f Let's say we help. Details are available and we need to supply this data to the supervisory off loving, and it will generate a model for us so except by become your input and output data you supplied to supervise were flooding and what they were F has been generated. It is nothing but the model off your let's take one example off for the subcategory off. Supervisory off learning. One of them is a regulation, so let's try to understand regulation with predicting house price. Let's say we have been given a data in a form off pair where in put his the size off your house and output is the price of the host. Let's project this data in tow two dimensional graph and let's try to fit this data with a simple straight line. So whenever the new data will be available with you, project this onto line and it will give us the price of your house. So what we did here we have been given input data as a size of your house and output data by and we have found the objective function are modeling function F, which is in the form off line, which characterized by individual slopes and intercept. Let's see about the classic village. So, in case of regulation, we found it all output way lives that continues in nature. Prize of the house are continues in nature, it can take any integer value. Where is in case off classification algorithm? All outputs are this creating nature. So let's try to understand which one example. Let's say we have been given data and each data has been. Label is one of the classes, so this set of data belongs to read classes, and they set up green classes. So are leery. Model, the state arbiter. Help open straight line, which Titley presenting put data. But in case of classification algorithm, we try to discriminate either into category one or category toe, so outcome values is either. Finally, it's a true or false idol. It's in a zero class one in a one class or it's a great class audit. Sina Green class It can be a multi class classification was, But the basic fundamental differences between regulation and classification are. Regression is continues in nature in case of output varies. Our comes in case of classifications. They're discreet in nature. Let's see about the unsupervised the offline in case of one supervise, we're we're learning. We have been given Onley input data and our objective is to model dysfunction f There is no output. All label data has been given to us. Let's take one example. So in case of classification algorithm, we already know that particular sample belongs to either late class or green class. He don't know where tech particular sample data belongs to, either in a class zero or class. But objective really is to form the cluster automatically either in a Cluster three cluster based on the requirement off the problem. But you have not been given the out label associated with individual data. Let's summarize all three techniques off the machine learning so increase over immigration and classifications and these other supervisory off flattening algorithm. And this is unsupervised me off learning are good a case ups, regulation and classification. We have been given input and out board the things input and output has a pair. The busy differences that regulation are continues as output, whereas in case of classifications, they're discreet in nature. Output are discrediting niche. Where is it? Case off unsupervised? We are planning. We have been given on lean put. There is no output has been given to us in our objectivity is to form the cluster off data . So that is what the basic differences between regulation classification kind of supervisory off learning versus unsupervised me off. Lucky. So that is all about this video and see in the next video with some more machine learning stuff. 33. Parametric vs Non-parametric machine learning system: Hi, friends. Welcome to the machine Learning System Design. An introduction to the psychical and Lively, one of the very well known library available for the machine learning system designed. So before diving into actual a p I level implementation off Psychic Learn Library. How AP eyes off psychical and nobody has been designed. Let's try to understand what other common machine learning system designed step. So the very first step is to acquire data up to import the data so they does is available in a huge variety of different kinds of farmer. In one of the section I tried took over debt from very, very idea of different file invite their idea of different sorts how to import most data in tow. Fight on programming environment In the last couple of section we have seen, How could you explore ity, the Thai analysis and how to visualize this data? Victor Elbow Ponders Library and Matt Broadly plus Seaborne library. We need some kind of pre processing stuff, like whether some letters that have some missing where the courts are not or some attributes has been missing, how to do normalizes and scaling minor tradition or standardization off your date. So desert to some of the section from importing data does scaling off and transformation Daytop, which we have already covered in the last couple of section where were mostly used. The Finders library and for the visit logician purpose. We use Matt Barkley Blade now it to machine learning religious stop restock like a feature selection features Helixon is equal and to removing some unimportant feature. Also, So what features are very important, which contribute very well could pedicure output way. But then one of the very important part is a modern selection. Whether based on your problem statement off machine learning system will you applied the supervisory off learning algorithms are unsupervised. Be off lending if we are going toe supply or use supervisory off learning algorithm board particular Martin are you going to use? So that is nothing but a model selection and in discourse, we are going to see lots of different kinds of modern, from simple linear regression to live in it. Lenient regulation, Toby. Far from the way far toe random forests. This is entering and support that kind off. Martin. Once you select a modern, you foot your data to particular morning and transform this data if required. Once your model has been really you applied this this thing set off data to your morning evil. You again, the performers. If you're not satisfied, video performance. Veto it with some sort of other hyper parameters, like change, some sort of pre possessing stuff, whether you're missed something or not. Scaling part features, selects and part include some sort of those feature. Important, but you remove. You know your first victory sent off system design jean some sort of modern selection. Use some different modern or seem model with different kinds of hyper parameter. And keep recreating the steps here until you're satisfied. Vitor Mortal and you're testing results. So disaster. Some of the common step toe designer machine learning system now for doing mostly the past like modern selection, feature selection fitting data Evaluating the performance. We are mostly going to use a psychic land library, one of the well known ivory for the machine learning system designed. So let's go to the psychic learned lightly documentation. So let's just go toe psychic documented. So if you see in a psychic learned documentation, that is a very good quick stock and who do you really for the development purpose? We're going to use this in detail? Wasn't what this AP I how that EPA has been organized? Let's received his use of aid. So I said so. A p A are organized into a number of different back is like supervisory off learning algorithms are available. So hello, supervisory off learning algorithm like nearest neighbor algorithm, Colonel, for the deregulation Linear models in Goshen process recently New Base Classify, Really, but in a supervised view of learning, sat supple going and all are implementing psychic find supervisory off learning algorithm. Make sure model clustering my clustering Nancy estimation other parties of modern selection and evolution. So you select the modern do you need and again devalued the results of that part of implementing. It's like a later transformation. What we see in the last section, like feature extraction part or repossessing off your later how to normalize standard days . Finalize your daytime later, Skilling or presence of this oil compounded. This sets a VPN data loading utility that is a pretty bitter. Lots of different sample data sets are available. If you want toe send, generate some sample because that you can use the sample generated class. Also, Preble data sets are also really well, like I the state as a most on house price that doesn't get started. I just based cancer with schools in a classic diabetes status itself and about from that related to some, compute additional performance related steps at every level. Now let's go to the actual a p A documentation. So AP a documentation has been organized, like based glasses there calibration religion. One is full flush during class, that is a very nice documentations are available which psychically library. As we approach ill, keep putting the link related to particular class, which we are going to use. So sample generator has a Lots of different classes are like, Oh, you want toe randomly generated like circle. It looks like a circle or looks like a moon. There is a huge amount of things that I really would like for the composition, like pcldy composition or singular value. Decomposition related stuff, sir, are really large sum and symbol. Learning algorithms has been implemented in the psychical and library like Abbas Stern extractor classifier mostly exploited, classified. We're going to use poor feature selection. What is the important features that they're so feature extraction related stuff for immediate. There is a different kind of feature you can extract, and for the tax, you can expect a different kind of feature. Gaussian process related steps out completely different Colonel Approximation. And there's lots of generalized linear modernist implemented a simple, linear regression less or regulation largest immigration. And that is a lot so regeneration Saturday. So these are some linear model. Many Ford Learning has been implemented, so there is a huge amount off algorithm related to machine learning. Statistics has been implemented are paralyzed metrics, calculation matter, paradise calculation. How to select particular sets up mortal For each individual section, we are going to use different, different sets off a B A from this psychic learn documentation. So that is all about this video. We have seen what other common steps off machine learning system designed for the next section on what we make our hands actually dirty. And we'll start designing the June machine learning system from the feature selection to the modern selection devalued the modern, applied immortal and then disaster, some common step. And for each of this step, like a feature selection, modest Erickson. That is a standard set of AP is that I really But in a psychic life. So that's all about this video. See you in the next minute with some more machine learning stuff. 34. Machine Learning system design and Scikit learn: Hi, friends. Welcome to the Biljon parameter like and a non parametric machine learning system. So at the Broadway, we can classify machine learning system into two different categories. We have seen in the last video supervise, be of learning and unsupervised were planning. This is from another dimension to look at the machine learning system. One is the parameters set off algorithm and under one is a non parametric set off guerena. So once you learn a lot of different machine learning ovary, don't. Then you come to know about that. What parameter is machine learning and non parametric based muscle and each any every individual machine learning algorithm? I will tell you that it's a parametric. Mids are non parametric, Miss. So before that in this video, let's just learn about what is the basic definition in criteria to qualify any machine learning algorithm, which is a Parametric machine learning system or a non Parametric machine learning system. And what is the pros and cons are associated with this boat kind of system. So what is the whole object to open a machine learning system? Let's say you have important later and output data and the whole goal of machine learning system to find the best printing Co from input to output. So let's take very simple example. Let's take input their input output Paris like 12357 committee zero. Now let's say, with the help of machine learning system, we want toe define some sort off formula which model this set off. It put an output passed so we can define like a very simple linear regression like a linear . Let's save Isaac Aldo, I m explicitly where am is a slope of the line and me the intercept of the line. No, we can define this formula and we'll tell that based on this data, you find out the rate parameter like M and beefy. So in one kind of machine learning system, you straightaway passing the clothes from solution. So the machine learning system and you tell that you generate those repair omitted for him . And that is another kind of machine learning system where you do not supply such a kind off close from solution. Instead of that, you just pour or fit all your data to machine learning system and system be automatic collegiately. This kindof complex model and formula for you. So whenever you supply this kind off waiting perimeter based close form solution to the machine learning system and you tell system toe fit your data to attack particular sex off clothes from solution only, so those kind of freedom classified as a parametric sets. But once you do not tell machine learning system that you fix your data, could this particular sets of equations are close from solution? In that case, machine learning system would automatically generate such a kind of solution so it can be such a kind of very simple linear for muscle. But it can be highly nonlinear. Form ourselves for ready, highly nonlinear hydro Mr Off formula. It can Children. So let's try to understand what are the parameters? Machine learning algorithms, Saturn and what other known paramedics. So I have created differentiating factor because parameter and a non pyramid. Now, as we're telling in case off Parametric model, that you have to fit your data with this particular set. Sophie question. That means it's a very much simple. We require a very less amount of training later, so in practice, the training time is also very low. So speed off training your data is very fast. But now you are unstinting. Your input data took that particular specific formula to your martyr. So it want particularly categorize any non linear or any other kind off pattern associative video data. It is not captured by your input, Martin. Because as you are beautiful supplying that model to your system, this kind of system may be good freed on your training data because as you already know about that, the behavior off your training gator looks like this. So based on that, you just supply that particular sets off closed form solutions. But whenever the new kind of testing data will come in front off this model, it will foot very poorly. So algorithms like a linear regulation. Our largest degradation are perceptron deserve the kind of algorithm which is completely parametric miss the case of linear regression. You cube like this formula. If you have just one feature If you have multiple feature like X one experts three you just apparently can supply largest Gregorys and similarly, But you just supply you are my default. Assuming that your system will be held, Esper or your data just captured this kind off patterns from it on Perceptron is on the other side of algorithms. Noodle neck of miserable little bit is a parameter is future section. We're going to see all those algorithm one by one, and then you will actually realize that what does the barometric meat and what doesn't known parameter. Let's see the non parameters now in case of non parametric basil Karina. We're not supplying any close forms, Alison, so it can generate any kind off ArmaLite congenital any kind off part, and it can capture. So that is one of the good thing about the non permit with al Qaida. So but it's a very much complex in that case, as we require a huge amount of training daytime case off non parametric visible So it's a very slowing gives up, and it captures a last set off. A huge huge actually captured a huge set of nonlinear function, but it has over fitting risk also, because you're not telling machine learning system that you just constrained your daytime figure data to that particular set subject prison. Instead of that, you give you go anyway, forfeiting your data. So in that case, it can fit all of their data so that particular sex or ready, highly complex, non linear model. So in that case, that is a fear off over fitting response. So, while giving them like a neural network are presently base algorithms support vector machine. In that case, we are not supplying any sets of particular question toe. Your machine learning system and machine learning system will automatically generated model for you that this classified as a non parametric. This so that is all about the perimeter biz machine learning system and non parametric this machine learning system. I hope you enjoyed the sending this video and see you next video. 35. Plotly - introduction: Hi, everyone. So welcome to the brand new section on another visit. Logician library. Larky. So we already gone through the one of the basic broccoli lightly or one of the basic purpose logician library macro click It is so you can consider the mother of all the visit Logician library really bolena fighter. And in this section, we are mainly going to focus with this bluntly lady. So what is bluntly and how plotless going to give much more advantageous over the might Broadly. So that is one basic misconception associate it with deserve broccoli that broccoli is a kind of company. So broadly is one of the company picture main degree really exist visualization electric products. And they hold a Lisbon product visualization electric one product which is also the need blocking. So library name is also plucky on a complete name is also bloody. Now how this sorrow partly is a different from this Mac, broadly so bluntly has been released in our multiple different languages, like a bite on hard. And we are mainly going to little practices are fighting was in office block. Know how that is a friend from and what are the advantages display likely will queue over the Mac project. So if you're seeing this map, luckily gravity or a basic utility library, what we have seen, which is also exactly return on the top of this macro clip only in a fine dust with the logician library. It's once Ah, you could get any kind of blotting functionality. Whatever the output will come as, ah, kind off images that is highly stated. Huges cannot interact with this thing or you cannot do mean or you cannot do Mo. So those kind of functionality is not available in case of this exported output. In case of this Mac bloody but display likely will provide hoping track, pick your our plot. So kind of Jew mean or Juno or some sort off small modification inside the graph that place only. So that is the big advantage. Bluntly with you over the mat blocking and the different plots realistic Collects a line chart, a bar chart, our haste O gram or heat myths. All those things were going to see talk this whole Shenson. Now, how do you start this blood? Lee? You can just simply use this peeping start broccoli and it will immediately install on your local environment. So I am already installed on my local machine. So let me this make necessary import important bluntly, not offline as a fee by you so bluntly, successfully installed on my local machine. So that Solomon decision in the next video we'll start with a basic plotting with this block I hope you enjoyed in sending this video and see you next to you. 36. Basic plotting - plotly: how you would. Even so, let's start with a very busy and a simple plotting in a broccoli lightly, so before drying, are plotting anything with the help of blocked reliability. Legis politically are draw something with Matt Brooklyn lighting. So you will understand the basic difference between what is the advantage display likely will you over the My problem, lady. So let me this make necessary, important and now will just import important numb pie, who generally some sort of random Rita a really important point, that's like and another one is a mess. Okay, so let me just generate some random later and we'll create one data from Just visualize it . So for inland grab display, let me tissues Matt Barkley in Lane and let's create a number Enough. Yes, okay, here. I did a mistake. Actually, I need to use like a baby. So let's is a generate data for one single column money and having local end, of course. So go to Denver and when on Let's just give the columns need there is involvement. Next. See? See one. Okay, so get everyone's been generated. Just display this year, so they could 10 records and one problem is only did Okay, so let's just use the basic plotting functionality of this Mac blood and made up. We can just simply use the f not block. So it has immediately displayed are this particular line Jack for us? So that propelled 10 values it has considered and for each up closer, our value it has printed. So what isn't Alexis? There are total 10 values and vertical axis corresponding values. What? We have random region it. Now, if you observe here, this graph is output. It's like a static images on, so you just cannot interact with anything you just cannot do mean or the human. Now let's see the same blood of the are going to generate with help of this bluntly library and what kind off interaction we can do. What extraordinaire functionality is a block level providers. So data be here all originating so same datable supply to the broccoli like so before that , Let me just make necessary, Important, important, likely not offline has a B by you. Okay. No, toward the speed by you, the black alligator. I need to use this block function and I'll just supplied So first I need to create a data. It is so table in a farm. It which is partly like we can accept, because now we have a data is available in a late offering. Objects from data from object are we need to create into this Nate object. So let me discreet. One video will create the list off this object. So horizontal exists. What we are going to supply it is equal in tow. You can consider, like, in next off this particular later offering. So this is indexing. So horizontal axis will be 01 night so that we can call it as a leave doctor index and vertical surviving you the actual value of this first column. So the f see even we can just simply excess. We have seen even in a bracket. So the first column value weaken extract. So data has been created. Now, let me put this liquor inside this block function and let's see what so you can see immediately. It has a creative graph. So let's just see here. So once I fired dysfunction be by, you got blocked, and I am supplied this later. It has created the standout block html five and it is opening another being built this browser. So you can see this is a kind of very nice interactive visualisation for this line chart. So the value of this particular one location, our second position card petition that then a number of other functionalities also they are providing. Let's say you want to download it as a PNG Me, George, you want to edit those thing into charge studio so it will take you to another website or you want to do mean somebody. This one are you want to jump out bang or you want to see, like, some part. So this Ray, you want to see, like, some parts or you want to select with the lesson select or Jimmy Jamar? Oh, you want for our cost? So there are number of interactive functionalities available details about this broccoli library, and that is what the basic at one is. Display likely will pull over. These are stated blocker with help Mac broccoli. Like now it has automatically generated this spam plot. Employees got estimate file. You want some custom filing so you can just simply give like a file him later. Basic blood market. Let me this execute sitting, create another fight, and it will open also immediately for us. Okay, so it is a very nice of visualization in a very much interactive you can in practice craft and there are a number of other options are available to make this graph even more beautiful and much more highly interactive. Real just given our very basic input values like Hard isn't Alexis. Were you in a Where Dick Alexis making. So that is all about a busy blocking. We tell this block library in the next few videos. We'll see some more plotting related French, 90 places, character brought on a bar chart and all those things we see in the next video. 37. Plotly - Bar chart: hi, everyone. So in this video we'll see how to plot this bar chart. So where this bar chart is generally useful, So let's say you have a date are available in some sort off this correct number off category. So finance number off creditor you can see. And for each of the category, you have some sort of value or some sort off aggregated value is some kind off maximum value for that particular getting your minimum value are you can consider some sort off Opel value for that particular category and you want to display in a disguised manner. So in that case you generally used this mark, so let's just started. So we hope I already copy necessary imports. So same as ah, earlier class now lettuce A. Generate a very simple later. So in earlier cars have used this are on Go procreate, Theis, go for the scatter plot of later. Now in this Ah, we'll use this in court, Doctor. See body simply real supply. Laker X and Y. That's an excerpt we give the horizontally are Let's see three different. Let's say our for the class and great one student great close to a really great tree student we want to display get how many total number of students exist in that particular greed. So that's a creative one. Okay. And in 33 Equalling viable with supply. Laker 23 student integrated for defined in 27 Greatly. Okay, so let's just assign it data, and I just don't close it this break ins. So this is a dick. I will become a list off. Lose. So do you guys be generated? So you used to seem be by you Look like supply like a later. Let's give fighting him bottled or excuse look, So you can see three different bar has been generated so great. One incorrect buena greatly so very simple Graph soul horizontal value will be greater. One big group Pretty it is nothing but the x ray you what we'll supply. So instead of using this and Goldar scatter, he have used the colder. Now this very simple example I hope, are taken. But I'll put it does some sort of experience. And so this particular a bar chart a za practical activity in a next lecture like you can extend it toe multiple diamonds and like, let's say, you you just want instead of just a single bar, Jack, you want her system sort of stacked bodies back Order what? Isn't a bar chart or multiple bar checked. So how? Bill you supply It's multiple values or you can see, like different colors Former deeper values, how you can customize the title of this graph. So kind off, Leo. So lay out. I'm not going toe explain you in this. So I would highly encourage you to go to the documentation and completely next activity toe extend this bar chart trunks landed. What? I hope so New. So that's all about the CDO C next. 38. Plotly - Bubble chart: Hi, everyone. So we have already seen about those scatter plot in a line chart in this video of you see about how to draw the double check. So there is no basically difference between this bubble chart and now scheduled palak. Both are kind off exactly similar, except one difference is doctor in a scattered plots of Aegis supplied of a two dimensional . So what is or isn't telling vertical dimension only, but the size of individual bubbles so you can see Marcoses kind of constant, but we can add another dimension also told the size of a critical market. So let's say we want to display the size of the market variably with respect to some other third name is so I have already pasted discord for this scattered blood. So let me just execute it. And I was. So what is the exact difference between the scattered and so I'll generated this ex invite earlier and we just supply this except by Judas goat outskated function data. And let me just block this so you can see every single market is a constant because this has no dimensions associative. Now let's say that with respect to some other tar dimension. The size of this market will very so in that case, how to behave so same court. Everything is same. We are just going to use it. Let me just create another car, diamonds and let's adjourn. Yeah, so with respect to chair the size of this market will reaching and will adhere another pyramid like a marker. So my career is equal sizes. Okay, Now let me this execute distant data and let's just change it here. Mobile Barnett, Stephen and you can see the tar diamonds that has been a so if you just compare with the scatter plot, all the Marco sizes constant. But now, with respect to the surgery value, the size of this market is way. So let's just again do some more modification to discern Let's gender a range of this size . Let's it toward aiming, said Jay. So let me just 600. So between 1 200 genital region little 20 venues. And let me just execute later. And just again, uh, getting this mobile check so you can see No, you can clearly see that the size off the store diamonds injured has a direct effect. Bitter size of this markers. And that is what the basic idea behind this in mobile shots, a bubble child is like a tar diamonds. An addict. You are scattered plot and scattered black. Individual market size depends on her dreams. So that's all about this. Will you see the next video? 39. Plotly - Histogram and Distribution plot: hi, everyone in the city, you will see about a different kind of block. So one is the history Graham kind of black, and another one is a listed building Mr Bishan Park. So let's proceed the body system. So let's say you have ah continues variables off one feature available in your data set. So in case of Instagram, it is time to predict the overall shape for distribution available for you later. So let's say whatever the continues data you have available, you divide those continues data into different, different marriage. So let's say all the details is available between the reigns of I 900 you just divide all those 100 in tow. 10 different markets. So let's The first pocket is from 1 to 10. Another one is a 11 to 20 so kind of end buckets and each of the size 10. And after that, we'll try to find out how many total data set value lies in that particular each other markets. So resident of frequency off values available in each of this bucket, it is tryingto blood. The bar chart off connecting. So let's first brought very simple history, Graham. So you get the idea. Addict. How does for it is trying to predict the overall distribution off full later sex. So let's first create a simple his toga. So I have made already all those necessary imports. Let's first it let some random but you it's the X the kingdom. Did you? Let's say between 12 Andre, I want to send it somebody I don't find it will use. Okay, so find it really has been generated between 1 200 And let's just define over later point. So later point is a list off all places. So if you tell pop this grab or media smarter will create a little point. So go dark hist o gram. We will supply over Dick. I exit here. Also, vo defined dekeyser X. So it's and another argument we can supply like X means so expensive is kind off the value , which is trying to say that you want to divide your data in how many total number of beans . So let's say I have a date available finding data points and all those data are okay. It's arrange. It depends on rain, so ranges 1 200 I want to the right dose arranging toe are 20 different buckets. So 20 different been sequences are 10 different breeds sighs Let's make it 10 for time and data point as we define no lettuce, simply part block. So look data and let you just kill the filing. Hist o gram, Not next demon and let me is executed so you can see immediately The program has been created. No, if you have seen nice instagram has been it has been blotted. But if you are 30 year we have given to see size is about 10 and our data has been distributed between 1 200 So it has to be divided and different markets. But now, or all those that has been divided into 20 different pockets. So no market size is something like a fights off certified to 39 42 44. So why is it like this? So we have supplied any kind off start and stop, will you? So let's give the start Where you as the one and stop musical 100. So you divide your take out which lies between the range 1 200 divided into 10 different buckets off size 10 just give stock is really one and stop is 100 divided 10 sites. So let me just execute it again. Oh, there may be some problem. It has to be and not stop. So now it's getting so now you can see the all the details were divided into the pocket sizes like of incisors stand so 1 to 10 11 to 20 Don't even took 30 kind of region even are modified like let's do 15 So now later has been live I did into somewhere around seven different channels So 1 to 15 16 to attack So this Ben size will actually define doctor how granular or how little lesson program part unique. Let's give the size very small. So very detailed blood you will get to know about the frequency of data between the 79 80 is somewhere around 18. 80 Numbers put 80 number randomly real generated which lies in this er vins 79 80. So you'll get to know about when you have very a small number off size are small number of men size in this case, you captain, about very detailed. You can see history. Let me disclose all those now thesis all about his toga. Let's just move to another kind of plot. So it's a distribution. So distribution Lakis you can consider like it is trying toe blocked the three different views. So one is by default Instagram It will brought it there. Another one it will block like overall distribution lines. So how the shape of your distribution will look like. So it is tryingto protect the carnal dusty density estimation for that particular arrange off later. You How really so, Let's ah, block this distribution block rate. A very simple data would be help generate. So we have our religion. It'd ex later, So let's just lock the distribution. Funny here we need to important one extra figured factory morning. So let's just first important. Yes, sir, So it has been important. Now let's just generate some big So I live via genetic all the data of its lies between the rain 0.200. Now let's just end. It normally distributed data because here in this case, what date of your worldly agenda date techniques a uniformed distributed day castle and density estimation will be something like a very much flat. So instead of better Alexis. And with a normally distributed data, No look arraigning. So just 1000. Okay, now we need to create a date out off. So and we just seem to supply X. So it's a list of and another thing we need to supply Laker group play with. So just like Level six distribution block No, from this figure, factory will supply this data and labels and we'll create a figure. Create this block. So the first start coming to get us a play like a detail. Another one is Labour's. So if you got has been generated now let's just block it. B y u Not like so the first time doing supply Laker figure and the second out human find. So let's just give us this plot Large team. So, like meters execute and see so you can see direct three kinds of graph eaters gender. So the 1st 1 based on whatever the DW generated, which is a randomly distributed how normal distributed data. So it has generated the historia what we have earlier see. The second thing is Decter. It has trying to predict the overall shape of the distribution of your data, so it's like a bell Seco offer normal distribution. And the target one is that, you know, resent early on, lean of one direction or a one. Diamonds anarchy. It is trying to paste the data so most of the data lies between this particular range. So this kernel density it is trying to predict the coroner density estimation was over all the staple your data looks like in terms of distribution. So in this video, you seen about the kinds of plotting. So one is a system alerting and another one is a distribution blood. So I hope you enjoy listening to city and see you next week. 40. Plotly scatter and line chart: higher. Even so, in this video we'll see about how to block this scattered type of block and line plot. So both of this car has their own intention by plotting the date on a block. So let's see you want to find the correlation exists between go different, very about how much do very well is. We call it reiterated that gives this cattle prod will be very much useful. Let it just based on what they were. Your holy sandal. Invite the vertical axis ter instead of just put one daughter some sort of point so will come to know about how much you are. Horizontal axis is school working Galaxies, whereas this line port is a kind off you can considered as a similar to deserve scatter plot, the very musical differences that all the points are connected in some sort of order. So let's say you want toe find out what is it trained with respect to some sort of horizontal axis on some sort of time. You can use this light block, so let's just start drawing some of with some random later. So I have already placed it and this is an import so bluntly dot offline and bluntly dog crap or BTS. So this is for offline gaffe during. And another one is they are putting those. They're trying to proper form in which this black like an unlisted and last one is a number . So two generator. Okay, so let us generate some random numbers. Excess. Look at him. Indeed. So what of the argument? Begins supply like a high low fantasize. So let's see you want to generate between one 20 Alexey 20 number between one. Wasn't it same way for violence so excellent. Why has been generated. So this is the axe user. So all values in letting between 1 20 upsides 20 So direct, potent frontin about your gender. Now let's this block are baseless. Daytime groups get a block far. So So First, we need to put it into some sort of later for me support that we are going to use this graph of median smarter. So not skater. So pushed arguing X dissipate his ex. Why would like did what you said more Just use the market So simple, Doctor. Put it Let us assign it to go. Let's see later and all those later I should be in a list for me. So we just enclose victor brackets. So you guys mention drinking. Now let's just use the simple P value block blocked. Just start up so far, starting with supply leg later and under one is finally So finally, let's just skip it. Get a life, not HTM. Okay, so you can see immediately. It has generated, But there are very few points I have taken so like me Just generate some more, leader. So let's just send it no handed. Let's just execute this later part. So you can see there is a good amount of data it has generated. Okay, so this is all about scatter plot. You can do all those kind off modification like a title name. Change on a horizontal axis. Level change or vertical axis. Slated level, chain. You can do all those kind of things. Now, let's say like medicine decrease. Let's a 10 only revealed with Okay, okay. And instead of scattered blood Isis Oneto draw. Let's say the line plot. So let me increase. Let's see between 50. So what isn't Alexis between one and 50. I want toe Arjun, The 10 different points and all those 10 different point. I just want to get panicked some sort of line. So I'm just going to use the same exact thing like you just executed. And instead, off markers will use Laker lines so it will be connected. And let me just use, like, line block, so data. Okay, So you can see it looks like a very much of rare stuff because our all the horizontal axis value. Let me just sprinted you, dear. Not enough sorting order. So it is not it all in starting on. So let's say we want toe generate all those were, you know, starting order so we can use, like, so I'm just going to use I will use, like, a challenge. One of the argument we can supply to this eating like a start stop and a step. So I just want to start from one, lets it parenti in a step off. So then you'll come to know about it. Write it Black Looks like this. So, except by its virginity. Now, if you just again bring the sex, you can see 1 90 and corresponding. Why? Details of Jimmy secluded, uh, 20 value. But now here by religion. Ready? That is it just that language. So let me just make it 20. So access 20 value and we have a VII's also granted. Let me just execute data, part and light. Look. So now you can see very nice, beautiful visualization off this the line Pollak has been generally who here you used like a lines for the line plotting and Marcus for the scatter plot you can use both of dem wants So line Marcus, you can see now markets also now you can do all those kind of modification like a changing off the color of this particular line. Our value Do some sort of mouse over or whatever the value has been generated. You cannot be in some sort off some what? Little more meaningful stream are you canteen Laker? Eight years for radios off this marker or color of this matter. So all those kind of modifications you can but the idea be indebted. How do blocker very basic scatter plot in line plot now are Bojan Rated dessert later for one blood on Now let's say instead of one line plot, I everyone degenerate father Coolen plots. So how we can go ahead. So, like, let's just use this accent. I seem so I'm just going to use down ago. So, accent, why has been decision? Really? Now, instead of using this league elected, we'll use the same one and like me, this assigning to the even. So later First blood is even another one. I want to send Greek Laker next. Did you okay? Let me just share some key. Will you incited? And the second child I just want, like, lines on. No, no markers of this. There are a total Stephen and Peter. And out of this, Stephen and Tito will generate so deepened, even call Mark. Okay, so now Dargan has been generated. I can use the same one like me by You're not block region, Victor Lang. So before they let me disclose all other caps. Okay. So you can see that a full line block has been generated. The second line plot. I'll just add it. Three. So all value has been sifting a little bit off fox, but otherwise it we would helping overlap. So if you want to display the multiple, let's say cities. In that case, you can use the multiple crisis like this and you can club all those cases dresses like you've been, put it into some sort off list and you've been calling as in data and supplied was later to the block functionality. It's all about this video of you have seen how to do the scattered block and line blood in this we, you know, next few videos will see some sort of more some more blocks, like a bar chart. Or he stopped crying up blood so that salable deceive you seemed next. 42. Reading Plain text file: Hello, friends. Welcome back. So in this video, we're going to see how we can lead a very simple flat file in tow. Fight on programming environment. So I hope created one sample file for each of those example, like a database file or remote HTML Excel File, Metal file or plain Tax es three file. You can download all those pine from the resource polar. Okay, so let's start with a reading. Very simple text plane file. So we have a new approach. We're going to use one, Is it context manager and one is the without context. Many. So let's start with the without context manager and then we will see what is the difference between So we are going to read here one file new start. Txt I have created so invite and we have one come on like open and inside the open well past the two argument. When is the new start the activities? Nothing. Ever filing feature help cape in the same folder where my Jupiter notebook recites and under argument is like a more so in which more you want to read this file? Let's say we want to just read the file so we can just like our so artists for the reading . Let's say we want to read plus write so we can write Opened this into more off the blue. So here this section is all about importing data acquisition of data We're going toe little bitter more are And let's just assign it a variable five. Okay, so we have successfully open this file. Now let's disagree to fight. Okay, so at the one shot, only it has that all those available inside that file. So I hope synthetically and manually put some tax out off the some newspapers. And it has read all those files after everyone knew the reserve. One slash and it is nothing, but it should indeed. Gettinto it is in available in the second line, but double Jupiter notebook has not displayed it, so don't worry about it. But Slesin indicates that he should come into next line. So with this file, Daughtry, it has read completely the whole file at a one shorten. Okay, so we have successfully read the file into one single variable and we heard displayed it. Now, if you see one property off this file variable, it is nothing But whether we want to see whether, despite lists closed or not after we have left this pile, it is always good practice to close the file. So it says that it's a false so we need to explicitly close despite so for that we have, of one matter, like half close. Okay, so we have successfully closed the fight. Now we can check whether the file has been closed or not. So it's a true so in this kind off approach, without any context managers that this approach is like a without contracts manager, we need to explicitly close the file. Okay, so now let's see another approach read. Context Manager. Severe use. You use it Context manager. In this kind of approach, you do not required to explicitly close to fight because you're already in a context off file. So once you're out off the context, so fight immediately by ton will take automatically for us. So let's start with the let's start with Vicky where so quick and then we're going to use the open function same. We are going to pass this new start the extra fight, and we are going to open it in tow are reading more and as we are going to assign it, put off file video. So in this way we are in a context manager off this fit afterwards, let's just retreat the line. One violent so file is one we're even in violent because thesis, let's keep it thus filed one instead of fight, we already use one variable for even. Okay, now we are out off in this context manager, so it will automatically close despite Okay, So the very first line off that particular tax pile it has. Really. Now let's say we want to read another line because we are in a context manager, so it will lead the total to line. So first line indicates states will leave the first line and a second plane will indicate it will lead the second flight. Now, if you see here, there is no absolute requirement to close the file or to check file whether this file is closed or not. So that is the major difference between wit, context, manager on every doubt, context manager, how you can lead a very simple plain tax pile in a fight on programming. So that's all about this video friends. So I hope you enjoy listening this video and see you into next video 43. Reading .csv file: Hello, friends. Welcome back to the section on importing Dina Pytorch. So in this video, we're going to see how we can rip are not CS Refine. Nazism is a common groaning for the comma separated value where each room has been represented by the individual records and individual column has been represented as a individual feature. And this is one of the most suitable format for the further analysis off data plus machine learning algorithm implementation. So in this video, we are going to see how we can lead this Nazi s we file. So we are going to read this to different way. One is numb. Pie is providing a to functionality like a low tax function. And under one is a gen from tax function and another being we're going to use the bind us. Okay. I have already created a to file which I hope taken from the standard database like a titanic Nazis. We file which file indicates that what is the record? So individual passenger while they were on a CPA off tightening and under one is amnesty. Tasek. So this is one of the very famous need as it embellished. It's a handwritten character, our database. So individual accordingly, present which character that particular image is. So first we are going to read this amnesty CSE and afterwards we're going to see this titanic dot cc. So if you see this AMN it's not seriously, Individual records has been represented in one single line soul. It is in the moment the next record will start. It comes in tow. The secondly, every number here is been separated by coma and this all number is nothing but the pixel off individual images. So let's see how we can really, you know, numb by library. So let's just make necessary important for us so inside. And we were going to use the function like a lower tax, and here we'll pass it amnesty. Now all those data if you see their new American nature. So we need to pipes what kind of data you want. Input. So we want to put all those they got that is a numeric. So, rather than putting all indigent valley will put it as a float afterwards. What? The other argument. So there are some records which has been commented with value like a hash. So what happens that this record needs to be discarded, so we need to explicitly inform the system that how you can discard it. So that is one option, Like a comments. You can pass it and you can put it like a hash that discard dys record. Now, all those value has been separated by the comma, so you can use the Dell emitter like Kama. So, deserter, I guess the minimum sufficient argument to redefine and let us assign it to them and miss later. So we have successfully read it. Now, let's just see how the data looks like. Okay, so you can see that all those data has been represented by the two dimensional are forming . So first record individual has been represented by one particular image of one particular character. The second rule is another character. Okay, so that is all about the amnesty data. But if you're observing all those data available in a numeric form, let's say if you want to read some categorical data, some features are in a form off non American nature, like in a string for me. So how we can do it. So numpty library will keep another functionality rather than using this Lord text we can use Jen from takes its agenda function to read that fight. Now we're going to read this titanic dot CSE so you can see that in Titanic Nazis we The first ever role is not a record, but it's a hider. That's the one thing. After that, individual records will start like a serial number. The class in visual individual passenger has said, What is the sex and what is the age? Whether this is survival, survived or not, and that is an last 20 frequency. So let's see how we can read distinct Shane from tax. And here we'll pass by training about seriously, Same way De Lemaitre will use the comma and a D type. What kind of data you want to really hear? The data is a make sure in a farming, some are enough large sum arena, non numeric nature. So we'll just provide this like a nun. Let's just assign it So we have successfully ready. Let's just so we got all those records, like in the jail accord, Okay, But the very first roars so it has taken into account. We just need to skip that first. True, because that's had it. We don't want to include this as a part off record. So there is a one argument like a skip harder one. Okay, so let's just displayed again. Tightening underscore data. Let me just run it. Okay, so the first record has been discarded. So I hold. Just skip this for the very first recorded, which indicates that it's had so don't include it as a part off. So we have seen Dr Videl pop number reliability. We'll use the to function like a low tax and a gin from Exxon via use to lead the data, not CSC, For my file into our fight on programming environment and in the boat took is it has returned the n dimensional. Now, by now, the next house generally in a data analyst, this process, once you read the data, you need to go for the analysis purpose. So if you want to do all those kind of mathematical operation, you can straight away. Go apply all those operations on this reminisce Daytona tightening data. But we have one more other options unavailable through which you can process your data very easy. So we have seen a leader section that how to analyze data, tell Pop find us lately. And this is one of the very important and a famous library, like a core lively off the scientific computing inner By Tony Environment It is nothing but the finest. So let's see how we can get the same thing in a finder's likely. Once the data is available in a form off the cuff em in a pond us, you can easily do all those kind of further processing on the top of it. So in panda were going toe read this titanic Nazis, we find. So we have a function like a relentless coast. Yes, me. And we will just supply here. Instead of passing this day, limiter is equal to comma. Here. The argument name is little different, so it's a separate. So how you want to separate your data? So that's all. Now, let's just, uh, I just assigning to the Titanic variable. Let's see, Data looks like, so it has given a very nice, beautiful structure for us in a nice tabular form it. So all those data has been league individual records columns, and they are all separated, very nice. So being successfully lead to three different way in this video. Now, once the data is available in a decaf, inform A we tell quote responders like you can do all those kind of operation like joining , merging whatever mathematical or logical obsession you want toe applied once your data is really in Passaic for your machine learning. Okay, so that's all about this Radio France. I hope you enjoy listening this video and see you into next studio. 44. Reading Excel and .m Matlab file: Hello, friends. Welcome back to the section on Importing Dina Piketon. So in this video, we're going to see how we can read an important file like Excel File in Under Vanessa Matte lip dot and fight. So let's start with how to read the axle if I So for that I have created no dummy file like of one is a Excel taste. Filing under one is a Mac Leptis fight both of the file. We're going to see how we can read in tow icon programming environment. So if you open this Excel file, I hope created total two seats. One is as soon as toe and in both of the sit there is a tabular form of soft datas are available, both of them data doesn't make sense. But here our purpose is to just read those filing toe by Tony Environment. So let's start Soto. Really. We're going to use this pandas lighting, so we'll just use the pretty before that has just import. Find us as a clearly and we will use the function like Accel fight. An argument will pass like excel taste and let's just assign it to the very simple five valuable So we have successfully opened this file and we have file Pointer will be available in our file variable so file. However, one argument, like a six knee it will give us. There are two states are available when is asked one and under. Vanessa has now let's parts individual businessmen and I stoop So file has, like, parse function and we will supply s one and it will return this in the form off later frame object behind us. So we just assigned to the idea Let's do the same thing again. It has so the year fun Andy after has been created. Let's just displayed is the F one, so you can see this is exactly the same rate of it was available in our Saito s one and under one is a six esto. So this will be just now. Read the excess it. Okay, so let's move towards the another file mac lib dot m five MK Libby's metrics elaborate one up The enterprise level scientific computing and vitamin, which is having a huge bunch of packages, are available for doing many of the engineering task and one daughter and file, which is nothing but a Mac lip proprietary daughter and filed. It isn't binary file. So as a human, we cannot really. But sometimes the data's are available in daughter. I'm fine. And if you want to read those data into your fight on environment, we're going to use decide by library a science fight on library, and this daughter and file is nothing but, ah, collection off all those my club variables. So for that, I hope created one dummy dot and fight. And we are going to read that daughter and fight now. Okay, so let's import this from the side by library. What other functions and a model we're going to use. So for on site by, we are going to use the Are you Maria and really important Lower Matt class, The same Laura met class. We can just call it and we will supply a Mac Leptis dot might find not met. File is nothing but pick you all into the Mac lip collections off variable unless just assign to the variable X. So we holy despite Lex. Now let's just bring this X variable so you can see that there number of variables like Global's had a rituals and bite on this file has been created and one other one very important variable, which is nothing but their data available. So all those variables which is available in a later so we just we can just grab it like X later. So here we have a very well Sarah available. Von is a another Vanessa be. He's having a value one and bees having a value. So this way we can grab the data from my clip dot matt Fine and Excel daughter satisfying. So that's all about this radio offense. I hope you enjoy the sending this video and see you do next with you. 45. Read Sqlite Database: Hello, friends. Welcome back to the section on importantly, Dina Pie Tone. So in this video of year going to see how we can lead our relational database file liker escalate database file. So for the sake off that I have created a sample escalate database inside that I helped kept just the one single table like a employee stable. And in this video, we will see. Ha! We can read those table into pytorch environment. So with by turned, there is a people libraries available escalate three. Okay, so from the escalate three Well, first connect to relate abyss and we will supply that. Which particular file we want toe connect. So I have created business service, sq light taste db dot baby. You can keep any other name also, but as I o created the file with his name, I'm just calling it. It will create a very first connection object which will connect our pipe on environment directly to the database. No, off the dick. Let's just get the cursor out off it. So connection, not cars. Let's assign it to the car variable. So now we got the closer and on the top of because there we can execute all those kind off already. So let's say we want to execute, like, very simple select star from employees which will return all those records which are available in the employ stable. Okay, so we have successfully executed. Now we got all those data and took car variable, which is nothing, but the cars are object. So from the castle object, we need to fetch all those record. And let's just assign it to the pros. Now, inside this rose, all those record has come. So legis trade through for looking so local variable real going to use row and really straight through, but was okay, And there's this spring in the three General. OK, so you can see that all those records are available. So this is the first black or available in early Thomas is the second record. So there are a total he tried course out there in a database. So this way we can leave proof. Ask you, like three library into pytorch environment. Another way, also, through pine does library the same thing. So inside the find us, we have one functions available like a SQL clearing in the same Gary, we can pass it here like a select star from them blue. And apart from that, the second argument will pass, which is nothing but the connection object which we have already created. See you and and let's just assign it to dirty if which is nothing but don't leave tougher him. So what is the advantage of using this approach is that once you read this data into tabular, which is available in a tabular form, it you can just directly grab it into data frame object for the further analysis. So here that a lot of things you need to do like a creating connection object is a minimum thing you need to do. But after that, creating closers and from on the top, of course. So you applied those query than you face all those daytime your guinea trip through all those thing. Now, in this case, just supply this selection are from employees over query and a connection object. And it will just written our data every object. So he says, successfully legis, print this data. Fremont, check. Okay, so you can see that it has given a very nice tabular form it all those data in a data frame object. Okay, so we have seen the two way how we can grab the data from escalated a bit. The same technique you can use to grab the data from another relational database system. Like a my bicycle. Even article my school database server. So that's all about this video France. So I hope you enjoy listening this media and see you in the next video. 46. Fetch Data from Remote file: Hello, friends. Welcome back to section on importing data in a bite on. So in this video, we're going to learn about how to fit. The file of it is available at a remote location. So for that, I hope kept this three different you are in and three different you are ill is pointing toe . Three different types of file one is pointing to diets Tamil, you are in when is pointing to the CSP file at a remote location. And under one is the geese on file at the remote location. So we're going to see how we can fetch all those 35 into bite on programming environment. We will grab this data from all different sources and we will try to snatch some information out top post it. Okay, so let's start with ex Tamil. You are so far. That's Tamil. You are. We are going toe used the request model available in a fight on. So let's just importantly, Crist. Okay, now, inside the request we haven't to my tourists are available like a you can send a get request or either upholstery Chris here, we'll just pointing to the particular HTML page So we'll just use the gut request in his argument, will pass leg. It's demon you are in. Let's just static to the arias. The response. Okay, so we have successfully grabbed it. Now let's say we want to grab whatever the data has been French. So that is a one tribute like a rtc dot packs. And let's just set it toe html later. Okay, like me this spring, This HTML later so you can see that whole webpage which contains all those values like html JavaScript cord off www dot google dot compete It has next. Now let's say we want to find a particular title of that particular page. So how we can do it? We have grabbed its data from right Stammel pitch. Now we want to extract particular title of that Pete. So for that will use the library like a beautiful soap. So we have a marty leg bs for and building put beautiful so out of it inside this beautiful soap we will pass these HTML data and we know that the title of that pace That's Tamil. You are adaptable dot google dot Commies Nothing but Google only so said Google. Okay, the second argument We need to pass. Like which parts? So you want to use it? So we are going to use this each time and or part, sir. Okay, so let's assign it to the soap object. So now we got all those HTML output. What? We gravel from this particular HTML, You are really into solve object. It is nothing but available in a soap up check for me Now inside the soap object for May we can apply a lot of matter like finds some back or find particular. Italy will kind of battery. So we want to find, like, a title and legis assign it to the title or less is directly pregnant. Okay, so it has grabbed this thing, the whole title stuff. Now if he applied this just a string so you can see it has came like a Google. So Google is nothing but the title of the peach google dot com. So in this way we can grab the data from HTML. You are ill. We have used this request modern and from the request murder. We craft first date our full HTML pit and then we can find a particular element or particular tag off visits Tamil element. Now let's see another. You are like Jesus on your the same thing we're going to use for Jason. Also, we are going toe. Use this request model only, and we will send the get graded instead of this Extremely rural will send this just on your own unless assigning toe. The arias took little time, but we got the response. Now let's see this ponds. So for the response, exactly build off this heartiest subject so you can see that in Jason Output there is a one feel like a first name, which is nothing but the John last name is Smith. He's allowed that is the number off pigs at it. There are some of the things like the response we didn't respond. So Jason object with energies on objects are also available. Let's grab some of the so far that we need to apply the maker like a Jason so it will form this Jason object, and after that we can extract some particular tag out off it. We'll just put like a chase on deck. So J son data, let's say we want to grab the very 1st 1st name. Okay? So it came like a John. Let's say we won't do crap this anything like a push. Tell cold so you can see the postal. Cory's actually inside that dress. So first we need to fetch Dad dress and as the second argument will pass it like a postal core. So first, his address. She's on data. It was probably so. It has nights this city, postal, Gold State state address. Now you want to fade. Just the postal quarter profit we can like like a focus on the postal code has been snapped . So this will begins night to data from this u R L b. She's pointing to some Jason file, which is available later. Remote location. All these three you are in for the practice for was I will keep it in tow. Resource folder. Now let's goto a last one like a CS with You are So you know our earlier video. We have seen that how we leave the file Nazism. We use the number off technique liquor We use gen from X function or low txt function of number reliably. And then we tried with the Pandas library and we used that you see a sweet function on the same thing and the same function of despond as you can lose Whether you want to fight the your local machine dot cc file ordered remote machine Nazis, we pride. So let's use this partner. So So input, beauty and well, we're going to use this. Really? Lucius will be passed. This lius me, you are not just assign it to the dear. It's a type of order of the state station. Okay, it has snatched his data. OK, so you can see that all data has been gravel. Even if the U. R l is a remote, you are in or it is Ah, local machine. You're you are. So we got those data in tow. Data from object DEA. So in this video, we have seen that how to fetch three different file like a GIs on file CS we file and HTML files, which is available at the remote location, and you can extract some data out off it. So that's all about this video prints. I hope you enjoy listening this video and see you into next week. You 47. Fetch Data from Facebook API: Hello, friends. Welcome back to section on importing data in a pytorch. So in this video, we're going to see how we can fights the data from Facebook. Very Pierre. So the information from the Facebook very p a. Like whether it's a personal information like me, or whether your friends information or you want to sort some object or some post or some like or some particular user on a Facebook. How we can do it with the help of Facebook. Very p A. So let's start it. So I helped to different you are. And it says that what are the documentation associated with this Facebook? So while searching for this documentation, you should be logging then only you can get this particular access token to get as an authenticate user while throwing the request to the Facebook's over so you can see that Facebook full set off documentations are available, like which hosted we need to use. So let's grab this information when Byron so we need to send the Gatwick wrist and service and point you are really is nothing but this last we two point find me now This V 2.5 is nothing but the service wasn't. Now, once the Facebook upgraded, Daly Ph will go toe 2.62 point seven in a Facebook AP exploded. We will see that. What is the latest wasn't up there, but Hillary P I they are serving right now. So this is the host and we need to send the data to the X TTP as you are. So let's go table. This individual data like this is the host. This is our service and point source said we sent point will nothing but upended here after our host. And we need to send data to the https. This way we will form the you are in now. This is the minimum thing. We have snatched it. Now the last thing is that we need to send the access token along with checked Now how to grab this access token. So we need to goto this graph ap I explored. So once you are logged in, you can get this access token from here. So if you just put it like get token, you can get this access Tokcan just copied and put it here like another. Very well. Okay. And we will just upended here like access token as a one off the get better metal. Really Say access token and hold this token upended Adang of the You are Okay. So this way we help grab this data. Now we can use either of those. Service wasn't like to point for 22.10 any of envy and we can use it. So we are just simply was 2.5. Let's say we'll use the 2.9 anything we can use but operated when you will get maybe more information or or some of the rules and policy. Maybe Facebook has updated. You will get it now. This is all about the personal information. Like a mean, Let's say we want to find information like of friends. So what is the end point? So in point will be something like It's less your was a number slash mix less friends. So this is the other end point. So we put it like this way and our every single decrease unit toe upended. Sexist. Okay, so this is here. Access token. Let's say we want to search something. So let's see. What is there any point product? So for searching purpose. We have our endpoint likes lass search, and then you need to put the query and you need to put object like a type. So type has a lot of different things. All those documentations that available here. So what? The type? Let's say you want to search for particular user all we want to search for a particular place. So here the denim number of types are there user page or particularly even. Do you want to say Let's say we want to search for a particular user. So how to farm this? You are in. So are this three year old. We're going to use it every time. We need to write like a wasn two point here as a vehicle to use this to White Knight along with the Q. We will put the very like, let's say we want to search the our hero Mark Zarqa book on a Facebook itself and its type . He's one of the get perimeter will send like user and along with then every single leakers we need to send access token on. So first we help created this you are ill and reform this you are like we want toe access. This personal information me then slash me. Friends. We want to get the France related information. We want to serve something on Facebook. So let's grab this thing and put it in tow of a Jupiter notebook and found is all those you are in. So let's create first access Tokcan valuable. So this become our broken so I'll just make it one variable like inside the talk and I hope so token we have created. Now let's create or let's from the three different you are in. Okay, so the very 1st 1 we're going to use it's nothing but me. So my own information along with Dick just apprentice Another one is like a France informations off friends. This part will remain common throughout the three. You are in on the same host. We need to send the data. And along with Dick No, it is the access took really to send. Okay, The last one is there's such So let's use this search host will remain the same query. We want to search like a Mark Zuckerberg and the type is used and along with dead will stand this nexus token. So we found this three different You are like me. You are one is a friends who are on the door of Eliza search. You are so with Delle Pop or Dysart. We can find a particular place or even like that. So here we have three different his argument as input parameter toe this gap liquids like a which credit you want to search what type off things you want to surgeon allotment that exist open before to request me and friends. We just grab this thing directly. Slash me and slash me slash friends on along with that, we just sent the access token. So we formed this three. You are now in the last video. We lose this request model so we just send this three. You are a request directly as a got request to the Facebook's over, and we will grab the information. So let's say we we want who will just first important this request Modern. Okay. And inside this request we grab this unless assign it to any one. Whatever value titles like Sam Li will snatch for another toe rewarded like variance information or such information, and we'll make it. Friend took France one. Okay, so Let's ready to hit this thing to do your Facebook server. The first you are with snatch data like personal information, the snack and will fetch the data. Like, what are your Flames are dead and 3rd 1 it will actually search related to quality. Mark Zuckerberg. Okay, so it took a little time, but we guard the data now let's see what are the day tight has grabbed so and me one text so you can see my name is and get mystery and my profile ID's 14145 somewhere. Not so. I grabbed this daytime This first you are on Lee on Kid. Misty may be a lot of other information. I have make it here. And so I'm not ableto get it. Let's see for the flames. One, not texts. What other response rates? Because sent so deservedly stop my friends like the inside the date object a lot off my friends. Information's out there. So those friends which helped make their profile public that only might be visible toe. Let's see, what are the response came in such one regular. So search one. Not the next. Okay, so almost we got the zero variable might be some thing. Let's let's change this just toe the mark. Maybe the string is very solicitous such whether we got some response. Okay, the Dell pockmarked there. A lot of people said that and all those people whose names start sweet or Marca Facebook is very donors like Mark here. Also, when Mark Soccer Sager's Mark Zuckerberg is also there, it may be ordered. No magical Got me may not be, but in this way we can grab data from the third party. P. I like a Facebook. You can save big. Grab this data from other Silicon Valley's big company like a Google, Facebook or apple or even Microsoft rest a p A. So in this video, you send it how you can grab this data from Facebook of the mortar V B P. A. And I hope you enjoy the sending this radio and see you in Don't next video 48. Introduction - Data preprocessing: Hello, friends. Welcome back to the new Saxon on data Pre processing. So what is data pre processing Before diving into a machine learning algorithm part or before fretting data to the machine learning algorithm, we should have a data which machine learning algorithms can process. So sometimes what happened? The data is not available in a suitable for mitt. So in that case, we need to transform this data which machine learning algorithm can understand. So in this section, we are going to do all those pre processing required for the data in this section. We are going to see how to import this all machine learning and fight unrelated library which will be useful for the data pre processing. And we'll see about very simple data reading. If there is any missing data are available. How to deal with that missing data? What are the strategies are involved and how machine learning library, like a psychic learn as skillon will be helpful for feeling those mission data. If there is any categorical data, machine learning algorithm cannot cope off it. So how to deal with this categorical data and transform from category into a numeric value so that we can fit all those data with a machine learning algorithm. Machine learning system is kind off training plus testing system. So how we can use the ask Ellen library building functionality off train and test split functionality with Dell Pop spreading data. Sometimes it may happen that some features off data are available in one drink and other features are available in some of the village. In that case, if you fit all those that are directly to a machine learning algorithm, one feature have a higher dominance than other feature. So to avoid this thing, we need to normalize this data. So all those topic we are going to see in this section so stay pure and from the next video on us, we're going to start about individual topic one by one and see you in tow. Next radio 49. Reading Data: Hello, friends. Welcome back to the section on data Pre processing. So in this video, we're going to see how to import the all pytorch library needed for the data pre processing and how to read the data. So let's just first important, necessary library. So we're going to use a number library and we will assign it on Leo's name and be for the number and another important library we are going to use, which is nothing like a find us and we'll give the last name like pretty. Okay, So we have successfully imported this numpty library and a find us lady. No. Next us, we'll do read the data. So we're going toe use this Banda's library and read CSB function from Rick Pandas living. So I have created fund a tough I like they are not CIA speak. It's a synthetically and manually generated data by me to explain the different concept like a missing data, categorical data splitting later and how to normalize data so you can download this thing from the resource folder in this section. So let's just read this fight. We are going to use this really CSC and we will supply data docs, visa input, argument and legis Assign it to the later looking. So we hear successfully Reedus Later. Now let's see what this data is all about and splitting into input data and output data for the further processing so well, So it has displayed this NATO. So their totals 10 records up there 1st 4 records, other input values like individual personal accord, which city it belongs to. What is its age? Whether that person is smoke or not, What is its happiness? Index out off 1000. And based on all this perimeter, it is tryingto product output variable. But whether it that particular person is healthy or not, yes or no is a common output. So it's a categorical data in output. So this four is input data and we have read it in tow data from object variable name data. So let's split it into input data and output data. So for the input data will use the common convincing in a machine learning it is nothing but the X and for the output, we're going to go small life. So let's possibly hex. So they've got we're going to use I Look, I look is for locating are extracting particular column and particular room. All those row we need to extract. So very first argument will put Plaza Colon, which indicates that all wrong, we need to expect so Second argument will provide, like each other column we need to extract. So we need to expect this is the first column. This is second column third column in the fourth quarter, but here the next thing will start from zero Fund 23 most of the time in a fight on programming language. Upper bound is excluded, so we'll start with zero and we'll go up to fight so it will include, like 0123 We need to goto only four, not five. So X will become zero to fourth column, so x become over this much. Data from the 1st 4 column City eight Smoke and Happiness Index take this values. Why this values? Because whatever it will output this data dot I look, it is nothing, what, another date off. But we do not need to detain a form of data, for we need data in a numb pyre. That's why I helped put the dot values at triple afterwards Now let's extinct the output Radium. So like a lot I know we need to extract all gross, but only Last column. So last column is indexed by the number 0123 and it's a four. So it's absolute for where number will put it on value. Okay, so let's just execute. No, let's just see how we are extracted This daytime took input and output. Okay, so you can see that all values which contains the next feature nothing but the feature and input value. So it's ah, city belongs to age, whether he smokes or not. And this is a happiness index. Let's just bring a small by the output video output variable contains whether but developers and it's happy are not so. Yes, No, Red is happy or not. So we have seen in this video how to import despite on library and how to read those data and the U. S. Protected stay dyinto input and output variable. So that's all about this video plans. I hope you enjoy listening this video and see you in the next video 50. Handling Missing Data: Hello, friends. Welcome back to the section on data Pre processing. So in this video, we are going to see how toe handle this missing data. So in the last video we have seen about, there are two featured in which we found some missing data. So let's just see and analyze it. So we have a second column, which is nothing but a a Joffe individual person where we found one n n So which is nothing but the northern number. And another one is the happiness index where we found and then So there is a two column where we found there is a missing off data. So as a machine learning system designer arms the machine learning engineers what action we should take. I don't We can describe this whole feature. Are either we can discuss that particular record, So it depends on the context that battle particular record is adding any value while Ningde a machine learning system or not. So most of the time, we do not discuss this particular record. So in that case, we should have some strategy toe Philip. This data and that strategy might be any kind of statistical measuring technique Let's take very simple one, like averaging off all number. So we are not dealing with this particular number based on the road, but based on the column, Because if you see particular column, it has a particular change. So age has particular leans like it varies from somewhere around 22 to around 18 so we can take the average of all those number, and we can fill it up with here and here in the happiness index. Also, we can take a bridge off all number and we can replace this an N B average number. So let's see how we can do it with a scale on library. So psychic land labor early provides one pre processing model so you can just import this a skillon thought pre processing model and inside it really important. This imputed in pewter is nothing but how to deal with missing data. So you are going to impure some computer value in place off some empty data or in place off some not a number kind off later. So let's just import it. No, The next task is create this impute er object and fixed all those input data so that imputed object. So for that, we need to create imputed off. So in pewter object request certain amount of argument So we need to check its help. So for checking this help, just praise question mark afterwards executed so it will come up with the whole signature For this in pewter, so impure requires this much amount off data as input argument while creating the imputed object the first wizard missing value. So how system will recognize that particular will use a missing. So default strategy is it's an end. So here also be, however, nn so we can just write missing with musical Brennan Let's say you have ah date of it where the missing value has been described by the blank value so somewhere that you guys blank so injectors you can supply here the plank, the next us is the strategy. So if you see this strategy how you want to impute your data, you want to compute the mean or 1,000,000 order most frequent, which is nothing, but the more the next one is, the Axis axis is equal to zero by default. So they're saying that you want to compute a longer Haro or along the column. So if your computing along the column you just need to put called axis is equal to zero so deserved the bare minimum features are there. So let's just start creating this imputed object. So 1st 1 we're great. Like missing values inside the missing. Really? Well, put any next one is this strategy. So strategy will use Laker mean afterwards we have access. Exists become zero as we want to computer breeding along the column and legis Assign it to the computer object. Okay, so we have successfully created this beautiful object from this imputed class. Now let's fit this in pewter object to our input Data X now, Benito, we do not need to fit toe every single column. We just need toe fit the second column and afford color which is indexed by this number one and number three. So let's just put in pewter God Afterwards. There is a to matter. One is a fit, another one is transformed and they hope combined This matter also freed and transport So you can do both the US fit and transform it a one time. So it is something like you just for fit and then you just transform it. I tell you, the meaningful what is happening in feet and transport both. And then they come up with the matter like freakin class bomb. So I did. You can apply this feet and transform individually one by one. Or you can just transform for use. Dissidents freed underscore transport out of one sort. Now what this fruit will do, it will actually take your input data where they need to compute. This means so it will just compute the mean like a training process off the machine learning system and what transformed will do. It will actually a pen that computed value in place of your missing value. So they are just transforming. Get up on one forming toe under the phone. That's why it's a transform. It fitting mints. Whatever the necessary, competition is required before transformation. So either you can use feet and transport when violent, or you can use feta on this score. Transport might have one shot, so we're going to use this pretender school transport in what later you need to supply. So we have our argument like X all rose me to supply, so we'll put the colon in comma. So first our human indicates Rose, and the second argument indicates what other columns took column baby indicated by first column and all the way up to We do not need toe. Give this one. So we'll just like toe so it will include only column, which is in explaining number one. So let's just assign it to the X one the same way we can write for the last column. So last column will be next. 012334 Well, like like a 3 to 4 here on something like like 32 for cooking. So before executing, let's just manually compute mean value off this particular column. So we'll take like, 24 so their total nine values are there. We'll verify for the age number and there's this executed. Okay, so we got the number like 35.44 for now. Let's just execute this one. We can do with this happiness Index also, but I'm not going to do that thing for happiness Index. So I'm guessing it might be correct, Tony. So let's just VD very five if The Age column So let me just execute despite and let's see whether it has updated this age. Not a number will do with this number or not. So I have successfully executed now registering this ax. Okay, so you can see that age value where there is a not a number or blank really was there. It has been updated with new Age, and it is nothing them but amino. Their average off all numbers in that particular column. So it's a 35.444 here. Also, it has uploaded, retracted by point for knife. You see, it is not a number Has a record like London Dan just send 665 So yet also not London instead of name. It has replacement average. Well, yes. And a 665 Okay, so we got absolutely correct. And for another column, like happiness on index ALS, it will follow. So this way we can use this in pewter class, inner psychic land library, and we can handle them missing later. So in this city of years seen it that how to handle this missing they come. And so that's all about this video plan. So I hope you enjoyed the sending this video and Syrian to next video with some more strategy for the pre processing 51. Categorical Data: Hello, friends. Welcome back to the section Honor data pre processing. So in this tutorial, we are going to see how to deal with the categorical data. So why categorical data? We need to deliver it. What is categorical later? So let's take our earlier example first. So I'm just going toe print the sex. So if you see here all those data it is not new American nature. It is called the categorical data. So if you see here the first feature on a tired feature which is having a value like a Mumbai, London, New York? Yes. No. Now all over machine learning algorithm can take on Lee new medical data into account. So we just cannot throw all this move by in London in New York. Just no kind of data directly to machine learning algorithm. We need to house some strategy toe, transform this non numeric data into numeric later and then only we can persist for the machine learning system design. Now, if you see by and output, will you that also contains yes. No. Yes. No. So that is also non numeric Indonesia. So we should have some mechanism to transform this from one form into another form. So what are the strategies are there to transform into new medicals? So one thing we can do whenever there is a pool will use that available or a three levels are available. So if you'll see first feature, that is, only three features are three different levels are available. I very same member. It's London or New York in a this feature. Whether particular person is smoking or not, it is only two possible values that there yes or no and output also is it to you so you can transform like yes to let's say, funny and no toe. Let's say zero. So that is a one way you can go. And in case off this move by London and New York, you can transform like a Mumbai will transform it to zero Langlands, real transforming one. So New York, the class monitor toe. So whenever there is a two features are available, your problem is almost gets all because either it's in a one direction or it's in the other direction, specially in output value. So, yes, we will categorize stood of one, and we will supply to the machine learning algorithm and nor will be categorized toe the zero. But here there is a one problem of any where you have a more than two features or more than a two labels are available in a single feature. So you here we have a total freely with. Now. If you supply this Mumbai's at zero. London is the one in New York. As a group, it may happen that machine learning algorithms tried to capture the relationship like London is greater than moment on. New York is greater than London, but our intention is not to provide. The daytime and saturated algorithm will understand that, like that is getting then Mumbai because of years supplying one for the London and zero for the Mumbai on New York. His gait of a London because in New York is two and a London is one. So how to solve this problem? So first problem is just a very simple mapping. We will solve Victor label and quarter, so we are going to use label and quarter. But for the another this first feature non numeric numeric unversity. We just cannot operate this kind of strategy. First, we need to convert into a label and Garden. Then we should have some other strategy, like instead off transforming toe. Just 01 and two. What we will do, we will just create the three more features like first feature is like a Mumbai. Another one is a London in last ones. New York. No, let's take all example one by one. So first feature is saying first equity saying its present belongs to Mumbai. So, like that becomes you don't become one and you become zero. The 2nd 1 is in London. Sanou Mbaye becomes you, London become one and your become zero. So in this way we can transform three different level in a single feature into three different features. So once we supply, just in case of, let's say, Mumbai, we are going to supply this a musical, the one instead of we are going to supply 100 So in case of London, we're going to supply zero in case of new, we're going to supplies you and for move away for that particular legal. We're going to supply money so it want captured. The interdependency, like a Mumbai is greater than London or London is greater than or New York is placed in a Mumbai. So those kind off interdependency want take into consideration. If we put it into this carnage and this form, it is nothing but one hard inquiry. For every single level in your feature, you are considering one extra feature that is nothing but one heart and corner. So we how to mechanism bonus transformation. One is the heart and quarter mechanism is a real you. Now let's see how we can do it in a psychic land library. So we are going to use the pre processing Marty s killing, not pre processing. And from that we are going to import the two classes. One is the label and Corden on Inter belisa one hard included. So first we will transform all these three features the city where the person is smoke or not, an output variable to relabel encoder. And after that we will just pass our first feature. So the one heart and Korda and we will see that all those course which is having a non numerical value as our input data output data has been converted into but regular numeric value. So let's create one object for label and quarter, so label and Gordon's. We're going to use it for input and output boat so heavily and the school exile create one variable. Another one isn't really in the school. Why? Okay, so I created a to object alley in the school accidentally in the school? No. Okay, So we who created the coup object based on the label and court a class. So now let's this fit this data over input data and we will assign it toe the same particular feature. So I just This car wrecks and we'll use the free transform it. This free transport matter is a very common and general matter across all machine learning algorithm in a psychic land. Maybe so first fit, and it will transform. So we'll just supply all grow. No. Zero column because he wrote column is nothing, but we need to transform this move by London column in under my Elizabeth smoke or No. So that is a tar call. I mean, next, my number two. Just assign it to the same column because we need a tight that particular Let's only further Father was seen. So same way we will pass Tommy to this second column number for reference by the number legis assign why you're so so here. We are going to use Ellie in the school by valuable, and we'll apply the FIC transforming the top off. Okay, so we have successfully executed. Now let's see whether it has transform correctly or not. Okay, so it has transformed this value. Now, if you see our earlier Very so. Mumbai, Mumbai has bean. Mumbai has been transformed. Toe number one London has been transported to zero. And in New York has been transformed toe. But that is not sufficient. We are going to use one heart and garden on the top off it. And the next one is a yes. No. Yes. No. Yes. No. Yes, Yes, yes, yes. So you concede it. Yes. No. So yes is represented by one and noise re present by zero. Then again, three years. So Oh, yes has been transformed. No one and no has been transported. You. Now let's see. Output over. Okay. Output are searching from 1001 Okay, so with Delle Bob, label and good of you transform over data to some new medical value. Now, let's use the one heart and quarter on the very first feature, but it says that which city person is belongs to So we're going to use this one heart in order will create a one object off this one heart and couldn't. And here we need to pass one arguing It is nothing but a categorical feature. So which feature you think that is a categorical features of our first row First column only. So first feature is nothing but a category feature whose indexes and Jiro So this is the one object will create and we will assign it to the one heart and golden available Ohe. Okay, now let's just freed this weighty to our input Data X and Vee vee no days to the story and let's assign it again to the X. Okay, so we have successfully executed over hacks. Now let's see whether X has been transformed or not. Okay, so you can see that now in our X the input very with a Lear, it has a total four features. But now if you see here, there is a total six features are there. So 1st 3 vigil belongs to the city belongs to then immediately disease. The age 24 1 is whether particular was in its smoking or not, and to 41 is a happiness index. So first record belongs to, I guess ours Mugabe. Second record is London in 10 is in New York. So if you see this movie has been represented by the second column, then it's a London. London is represented by the First Column and in New York has been represented by the card column. So in this way we'll transform over all those data from non numeric form into numeric form , which is suitable for the machine learning algorithm to get process. So in this video of your scene, the two important encoding technique which will transform your non numerically dyinto numerically time. So that's all about this video evidence. I hope you enjoy listening this video and see you in the next video. 52. Splitting Data in Training and Testing Set: Hello, friends. Welcome to the section on our data Pre processing. So in this video, we are going to see how to split the data into training Set up later on a tasting set of or why we need to do this separating dis training later and a testing data in case of machine learning system design. So we know that machine learning is he called toe training plus testing system. So it's you can say, like once you have a train set of data you supplied to the machine learning algorithm and whatever model has been created afterwards you apply. Tasted on it the railyard that how could your particular model is so in all machine learning algorithm, it's always recommended specially supervise way off, learning to evaluate your model, divide your all toasting data into training set and tasting sick. So how we can do it. So psychic Land library as Skillon will provide one model, pretty processing and inside a plea processing really important, not replacing its cross relegation cross validation and we'll import training taste split class. So really already ah, separated out over data in tow, input variable and output variables. So after training on tracing spread off data. It will separate this data into chunks. One is a train set off. The dining it is set of didn't even inside the train set of data that is input data extreme and output data. Valtteri. In case off tasting data, it will be access and answer Vitus. So there is a standard convincing off writing input diner output data in case off this by tone based machine learning. So let's just fuss. Split it inside. The spirit will pass. The two are given like a one is input very well, and it's output with him now. The next argument is this size. How much amount of data you want to do, what toe the distinct phrase. Let's say we want to put these things I A total 3 30% of your data, so we like a 0.3. And let's just assign it to the extreme and then X tests who I drilled. Lightest. Okay, so we have successfully executed. Now let's see in our original data site like X X contains total, then records and why are so convinced? And the Cosby So because it's a corresponding how put off X to the Y. Now let's see whether how many guitar records in this particular extreme next days contains . So if you see this X stream, it has total seven records on you. So 1st 2nd 3rd Port 67 and same with tasted quitter. White win. So why Pain also has a daughter salmon, because so it s pleated this data into sound deeper Sunday Tinto training records and 30% in toe tasting records. Let's check expressed also. So expenses totalled a three that causes total six feature and rightist will be total to read a course. So this way we can us plate you're our input data in tow to different juncture 70% agent training The diner 30% of testing says you can even change this size to the 0.25 0.2. Normally, people will keep 0.2 or 0.25. So that's all about splitting their time to training. Setting a tasting said I hope you enjoy listening this video and saving to next for you 53. Normalize Data: Hello, friends. Welcome to the section on a data pre processing. So we have gone through a number of different techniques in a data pre processing step and make later. So table for the machine learning algorithm. Now let's take another step like a normalization of data. Now what does the normalization mean and why we need to normalize data? So let's says, based on our X training data, which were expected in the last video. So if you see this training data, there are some featured. It is available in the form of zero in one, so it lies between the range zero and one. But there's features like ages, which is having a very small value, compared toa the happiness index. So if you see all those feature, it is not lying in some standard in between minus one and one, or let says you going fi what? Maybe between zero and one. So what problem? It will create that most of the machine learning algorithm depends on the Euclidean distance measurement. So if one record wants to get compared with another record, normally Euclidean distance will be taken to account. So do you do this Euclidean distance why it is such a kind of problem vehicle. So let's try to understand. Let's say we have a first records like Lisbon and a second record will be something like this one. Now let's just concentrate on just hate feature. And under one is a happiness index feature. So forget about all those because it is not significantly contributing much. And in the other one is 45 70 Okay, so we have a new data point like which is having a month feature to define six. Explain 45 707 Now, if you substrate visting or if you see the formula for this Euclidean distance What that formula is suggesting that you substrate corresponding, coordinate like extra minus excellent. And put this into square and again, Why? To minus raven, then scrap and take a square root up. Hold those number. But if you just substrate tart if I minus 45 it will be somewhere around 10. So this components become just 100 on is the first component will become a security somewhere around 100 but the second components it it's obstructing 707 minus 665 so it will be somewhere around defy, plus fights somewhere around 40 and 40. We need to scratch, so it will be somewhere around 1600 and now we'll take a square root off. This 17 hundreds was so you see that the second feature, which is nothing but the happiness index which is having a higher dominance always because it's all values, are arranging in a very high range of tea. The range is also very high. The numbers are also very high, So when you put any substance in between high range value, it's square become even much more higher. So we should not do that thing. And, like machine learning algorithm, do not understand that this happiness index is a higher value. So it said, the higher dominance and eight has a lower value satar. It has a lower dominance, even other features like utilizing a zero and one only. So if you substrate from one and zero, it's square become born only, so you start off even one. Sometimes it became just the one on the so that feature is a human lower dominance. So to avoid this kind of situation, we need to put all those features hit one. Particularly either will be 18 a minus 1 to 1 or it will be in a 0 to 1. So Psychic Land Library provides number off day to transform this data and to normalize this data into particularly so we how scaler Czar standards Killer one classes available in a psychic land library. So let's just apply distinct scaling but pre processing model and inside that we are called the Strangler scaler. Now just just create this kinder scaler object and will create a variable name Xcx. He want to apply in tow and put it on. So that is all about now what this time the scaler will do It will take the whole bunch off range off the value in one particular features like Oh, let's say, age 35 45 24 29 36 eighties. All those value it with sub sects. There are a lot of different ways to do it, actually, to put it into standard that better you take any individual value in sub strict a minimum value off that particular feature and you just divided by old range, so range is nothing but maximum value of that particular feature minus minimum. Well, so this is a one way you can do it. I saw you one more adobe that is a statistical measuring. Technically, you can go ahead. So instead of substituting this mean you just stop Trickett from Aboriginal You are mean value up that particular feature and divided by the standard deviation. So there are a number of ways you can transform your all range of daytime toe One particular friend once you divide by the max minus minimum order right by standard deviation. And now you're subjecting here the x mean and your years of taking x x minimum. It's a it's x, the average value In that case, the whole range of value received from one range to the UN ordering and yet another range become almost similar for all those So that terrific and transform from and we control a normalized this later. Okay, so let's just fit this later. Whoever Input, training data and input Tasting later. Savio, Creative on object A season discotheques when they're well freaked the transform Send the data XT eight and let us assign to the extreme Okay, same review gold and we'll do it X taste awesome. The next s case We do not need to fit and transform. But as we have already freaked it this later, we just need to transform. So just transform it Now what this individual feet and transformed will go for It will actually calculate this value, and it will convert this or regional over feature in tow. This particular feature, based on the standards, killer expression and transform will actually transform this data and put it in tow This output video. So that way, extreme and excess. Okay, so let me just executed. Okay, so we have successfully executive. Now, let's see, How do you guys be? Comes home. So if you see that all those data which were our earlier in the range of somewhere around 10 25 6 65 all this data now lives between the range between somewhere around plus two one on minus two. So all that has been transformed toe one particular. Let's see about extension. Same happens. We text. It's also so it has transformed toe same way one form into another home. So all data we got into one particularly so in this video of you seen about how you can normalize data to make the data so table for machine learning algorithm. So that's all about this really offline cell. I hope you enjoy the sending this video and see you next video. 54. EDA of pima indian diabetes dataset: Hi, friends. Welcome to the section on exploratory data analysis. So in this video, we are going to analyze one of the sample data set which I have taken from you see, a machine learning repository. The link off this data said I will put it into a description. So let's just open this data set this website. You see, I machine learning reports three contains a lot of different kind of data set for the learning and research purpose. I hope they can this be my Indian diabetes state, a sect. So this doesn't disassociated with classifications related algorithm. And there are total 768 instances off this data sets are available and associate id at the roots of eight. Let's just goto this data folder this data for look and think too important fighting. When is the dot data fine. And one is the dot name file. So let's just open this not named file. This not named file contains all meta information associated with data set, so title off the data. The source from when this data has been originated was donated. This database vented. State eyes been resealed. There is somebody living information associated with this data set. Number of instances are available for this, does it? Number of attributes are available in this data's it information related each attribute plus output class variable. So if you see here, there are total 93 words are available out Talk This nine a tribute Who one is the class valuable visits The last attributes. Apart from that, there are eight attributes are available, which is nothing but the independent variable for our and it does it. The 1st 1 is like a number of times. The patient got pregnant pregnancy related information, then the plasma glucose concentration. Blood pressure level, those skin thickness level by degree function of diabetes and is information so desired? Er, some of the small description associated with heated tribute. Apart from that, it will do whether this data contents missing attributes or not, how your class distribution are there. How many total number of record lies in a Castillo and how many total number of records lies in a class move on so deserved that some of the matter information associated with the state asset Let's go to this name not Datafile not data file. Contain your actual data set. So every single row re present one instance off record and every single attribute are separated by the comma value. The last value is corresponding toe output, will you? It is nothing but one or 01 corresponding toe basement has a diabetes and zero corresponding took place and do not have a knife. It is apart from Red Daryl totally that we will saddle the first attributes sixties corresponding toe the number of times pregnancy. The second is 148 which is corresponding to plasma glucose concentration. This way for the first present information, this is the second patient information. So let's just download it on your local machine or you can just straight away point this particular you are in and you even download inside your finder's laboring I have already taken on my local machine. If you want to take it into local Massie, just please control us and it will download it to your local machine. So for the analysis purpose and we're going to use a bind us like really? So let's make some necessary put. So for Panda, we're going to use import find us. Has BD and Fourth of his delegation purpose. We are going to use Matt brought lit So inside the mat plot like we are going to use the pie plot model and we will do the yes tip PLP. And as we've owned all our graph off my plot like to get displayed inside the Jupiter notebook. So we'll just write Matt bluntly space in line what it will do. It will display all those crap inside the Jupiter, not book only, so it won't create another window for the graph. Civilization now legislate this Be mined in diabetes. Later zip file through one of the functions available in a fine dust library. Really see us and we will give you the final beam. Are Indian diabetes Lord data, not CSP, and will give the 100 name. So I have already created Winery, which is corresponding toe, the sort of crony named for individual active So first twenties or practice prayer is nothing but the pregnancy. Then applauses. Plasma glucose level. Last one is a class eight pedigree function. Skin thickness. Let's give it another argument like headers, physical do name names, physical name and legis. Assigning Toto later. For now, let's say we want to see first. Few record up to state as it so we can just later we use this hair fusion. No need to pass any argument. It will by default display first fire record for you yet how this fight a court looks like instead of fight. We want to see, let's say, only seven record or 10 record. We can just posit argument like seven to the head function in Italy. Display for seven record rather than a reading data from the first. If you want to see the later from the last, we can use the tail function and it will display last fire, of course. Instead, off Fighter called Let's Say we want procedures to record. We can just pass it as argument toe the tail function, and it will display two records from the last. Now let's say we want to see how many total number off records are available in over it and how many collectibles sat associated with our day testing so we can just use, say, Patrick on the top off NATO Fremont. So it will you want topple. It is having a total value. So first values corresponding to number off records on number off those in a second value corresponding to number off column, which is nothing but I traveled, so their total nine attributes are there black plus toe class out off them. Heat are being variable and the last class is the output variables. Now let's say we want to see what is the data types associated with individual attribute so we can just use me type function, and it will give us the what is the data types off individual attribute. So apart from Mars and a pretty all I tributes are endangered. 64 bit attribute. Marcin Perry belongs to float 64 bit Attribute. Now let's say we want to see some of the statistical measurement barometer. Associate it with our data set. We can just use describe function on this data frame, and it will immediately display the count mean value of that particular feature standard division off the feature minimum a maximum value and a court trial value corresponding to each and every feature of the data set for the corresponding feature. Like brick, the Countess sounded in 68 the mean value in a this particular feature practice 3.84 the standard Davison is 3.36. So this type of analysis will be very helpful when you want toe do some sort of standard additional on your data it how much your data has been already standardize. How much are how many our place exist or not? So how much percentage of your data lies in first quarter second quarter? Tyler told Cordray. So that you want to remove theirs. Don't sometimes sole value. Now let's say we want to see how much Cory listen. Is this between individual pairs off feature so we can just lose core function and it will give us The Cornelison exists between each individual attribute with another, sets off all attributes. So every row and column contains same set off attributes, so pregnancies break is always one plus. For this practice. 0.12 sits a positively correlated but not very highly correlated. But instead of plus was this black place was his back drag his little bit higher correlation. So zero point competitor by 12 0.14 little bit higher, very skin was his back is negatively correlated. Let's say instead off visualizing legis, assigning toe the core and core become the frame and out of this data frame. Let's say we want to see Onley both pairs off. Correlation where the Cornelison is greater than zero point fight. So it will only display those pairs off Correlation where the correlation is zero point in greater danger. Avoid fight. So between the agent pregnancy, the correlation is very much high. 0.3 people, apart from all pairs. Okay, tribute. We don't have any kind off correlation or very less correlation, which is less than 0.5. Let's make it a little bit less injured a point for so we'll get somewhat more sets up activity. So we got another two sets of pets like class Was this plus Hannah Taste forces skin has a high correlation exists Now let's say we want to find the Souness off data how much your data has been skilled, whether it's a positively skewed or negatively skewed. So we can just use this huge function and it will give us the stillness off. Individual feature like Chris and a Mars is negatively skewed. Apart from it, all of our features corresponding to the state a state are positively skewed. Let's say we want to see the class distribution of this data set. I mean, the last column associated with status. So it is idle zero or one. There are some value associated with one in some, values are geo. Now let's say we want to see how many values lies in zero and how many lies in a one. So we can just use this group by function and we can pass on class, attribute, name, and we'll just apply the size matter. And it will give us their total finders, that course corresponding to class zero. And there are only one records which is corresponding toe class one, that there is no uniform distribution among the classes because there is a very high number of people who is not infected by the diabetes. So it is lying in a class Jiro, where is in a classman that is only half off the people. Chris compared toa What is available in a class here? Kenjiro, Let's say instead of class, we want toe group by each in a decreasing order it has almost given. So each 21 has been appeared 63 times. He's 22. Working appears 72 times so people who engage 2122 20 t has a maximum number. Off records are available. Retailer data set there is if you see at that extreme office 70 70 to 69 has only one tour , not more than two even presents. The records are available because so that is all about the exploded three data analysis. You can do a lot of other kind off analysis with this data before applying any kind of machine learning algorithm on the top of the status. Said so in this video, we have seen how to do expertly 33 tie analysis in the next video. We'll see how to visualize this NATO sacked. So we are going to use this Mac plot lib library, toe his allies this data set and before applying this machine learning algorithm, when should have understanding up how your data should look like so thank you guys for the watching and see you in the next few 55. Visualize pima indian diabetes dataset: Hi friends. Welcome back. So in the last video, we have seen how to do experience Lee Data analysis On the top of the mind in diabetes data set in this video, we're going to see how to visualize the data. So let's just display our first few accord So later frame object. What were imported in the last video. So we hope total eight feature Sudden and Vaughn is the class volume of output feature. Now let's say we want to see the scatter plot associated with individual pass off feature like pregnancies plus prognosis. Place bag was the skin. How working week. So we can use the building function of the spine that slavery. So let's make some necessary input. So from finders, we are going to import, locking and from plotting really important us cattle metrics and legis plot this cattle metrics and we will just supply that GIF. Let's makes figures eyes little with decent. So let's make it 10 7 so it will be 10 in a proportion and height will be selling. It needs to calculator, scatter plot for every single person records so it will take okay has display scared of legis make it here semi colon, so it won't display all those areas Stuff. Okay, so we got some nice looking scatter plot individual parents off feature like a bag. Was his back here? This so proud was his prayers become a kind of history is a plasterer suspect How did has been distributed class forces? Let's say it is a party. So this is a press. What's this place? So it becomes a history because you didn't have any other feature to get compared. Now let's say if you want to see in just a historical for individual attribute so we can just use later not hissed and let the supply figure size. So let's make it 10 7 We need to put Sami column TV very tight. One display all our information so you can see very nice visualization for his diagram. For all individual feature like age mostly concentrated here, it Lord digitally ignored because this party is also lowering. Whereas in case off this press, they're mostly value lies between this 50 or 60 to 80 whereas in case off this class distribution, either it will be a zero or one so zero corresponding toe. This many numbers hope records are available for volume is having a little less number of records. You see the very last one taste. Mostly, the date has been concentrated in the lower area. The high radio that is a very lace amount off record said there. Now let's utilize this data with some other kind of block. So let's make some line chart, so we'll just use later, not block the F dot plot. Very well supplied. The kind is equal to Let's make a line check. So what it will do it will try to display for all individual feature. It will display it a line as it is, a total nine attributes are available. We want to put all those lying charging toe one single grab so we'll just make it. Subplot is equal to and let's make a layout. So Leo greatly three. My tree So their total nine features. So it will be organized in a pre rose and three columns. And let's make last one figure size. Then Comma seven looks decent for me, so I'm just keeping it. 10 comma. Seven. Okay, so it has displayed so but it looks little clumsy, so let's just read it and displaying all those things. Let just display for one our particular feature, like a pride pregnancy so you can see has displayed it. So in that case, the legislative modus thing, because we don't want this applauding, all toasty. Okay, so it has nicely visualized all those pregnancy related Leica for 768. Let's make under the charge like a box plot so you can just use the f word block kind marks black in the same day. Supplied will make it through Leo. Let's make it three common crease envy. And the figure says so. 10. Come seven. Okay, so it has displayed total nine charge for us For each individual feature, it has calculated this median value. Oh, extreme Maxima, Our curse. And what is the 25% Cartwright around this 1,000,000 value? It has capability for us. So for each individual pieces a year, you can see the number of feature records lies outside here, off the 1,000,000 value here also now, in the last video, we have seen how to calculate the correlation between individual feature and we try to display where the correlation Waas creator, NGO 0.4 or 0.5. Now let's visualize with some sort off color. So for that we're going to use the Seaborne library. So let's just import the Seaborne as a Sinus and let's say it again. Calculate its leader court so it will. No, the correlation. It will calculate the correlation between individuals, pass or feature. And now we'll use the hig map has sent its Lord heat map, and we'll supply here the core so you can see it has calculated value. What is worth a calculated coalition value was there for each individual value. It has mapped to some sort of color. So very darker color are really sculler belongs to very high correlation, so you can see the diagonal values are all very dark. It really is color. So it is associative victor. Very high correlation because it's associating same feature with itself only. Apart from that, the blues color is moving towards the negatively correlated and release color he's towards positively correlated. So in the last year, we'll see in the year. Age in pregnancy has a very high Cordless, and so it looks like a little bit more than this compared to other parts of the church, this one and this one. It's exactly same because Greg, what's his age? And yet so it's horses pregnancy. So this part. So this one thinks class was this. Glass also has a little high correlation. They stores his skin thickness is also very high. Correlation Little hired coalition. Now let's say we want to see that guitar distribution. Acosta are different classes, like How many are values lies in a class you're in class one. So in the last video views in African group by you can use this group. I function off the finders library, weaken group this data into class zero in class one, and we can get to know about that. Tommy and I course lies in a class even class one. Now, with the help of bar chart, how we can visualize it so we can use this mark block now across the horizontal axis. We want to use class. And for the vertical axis, we are going to use the Greg. Let's see any one particular feature, and as a data, we are going to supply that. Give the full data frame object. So what did we do? It will try to display the bar chart for Class zero in Class one. And for classier, it will calculate that what is the average value associated with this pregnancy related, attributing for class one. Also, it will calculate the average value associated with this pregnancy related returning to displace Bartek so you can see that for class zero, the operating mean value of this pregnancy is three point somewhere. For Class one, it is somewhere around the four point it looks like 4.8 or 4.92. So how did value has been arrived? Ah, we can. We can just grabbed a little like the F from the year. Where will just find where classes equal do? Let's say zero. So what it will do? It will try to capture only those data from the early years later Frame object where the class is equal to zero and out off that. Also, we want Onley pregnancy related feature, and we'll just apply the 1,000,000. Or we'll find out gradually up that particular sex of pitch so you get 3.29 it, so it is somewhere around three point to 90. So for Plaza has calculated listing and for let's make it. Let's find for Class one. Also 4.86 of cities around somewhere around 4.86 Let's make let's draw the chart for a donor sexual feature like pregnancy. Let's search for ages so everyday it for class zero is somewhere around 30 to 31 somewhere . But for class one, it is cacti seven or somewhere around 27. So this way begin our orderly between individual go feature that we respect over output were here. Well, how much data has been are distributed for classier and how much it has been distributed for classman with respect individual feature. And we can use this functionality like a bar plot to better visualize it. So that is all about this video we have seen number off ways to charge your be my Indian diabetes that has it. You can visualize more, more and more charting related functionality based on your object you after machine learning problem statement. So that is all about this with you See you into next video with some more machine learning stuff 56. Introduction - Data transformation and Scaling Data: hi friends who welcome to the new section on data transfer Mission and scaling for the Machine Learning. So in this section we are going to see by B requires such a kind of pre processing. We'll see about what other different techniques for three scaling and a transformation off later, like a re scaling up later. How to standardize data. How to normalize data, how to buy analyzed it. So why we require such a kind of pre processing step. The day does are available in such a huge where idea of different ranges and every single algorithm later required in some particular range. Somewhere good item required. There tryingto standardize form like a normal distribution, but it does are not available like a normally distributed way. So we need to transform their timeto standardise for so decide to some of the common reason why be required to do a pre processing under top off. So for the pre processing part, we're going to use the Psychic Loan library and say Mazar Lewis section, we're going to use this. Be more Indian diabetes later sect. So deserter, some of the features drag. Plus I just swat form. I have given and I have kept this be my Indian database file into same folder where I'm running this Jupiter. Not so in this video will just import this later. And from the next video invites, we'll start about individual technical police killing standardization, all data. So let's just run it for importing purpose. We're going to use find as like it. So let me just make necessary import and from PD will use the really CSC file as it is a all the does are available in the CIA's. We find it pile name and as argument for the need will provide lives so it will become a header. Let's just assign it to them. So it becomes the data from now from the data from ledges extract value. So whatever the outcome will come, it will be like no umpire and from the numb pirie less this extract all those inputs and outputs output value is this last variable class and input features are from this 012345670288 He's been excluded. So let's just assign input as exe ETA for we need to take all grow So first that women will be calling, and the second argument will be 0 to 8. X has been included, and output y is equal do data again, although, but only eight column. So you're successfully important Legis printed. So this is the first feature. Second feature card feature is a little wife. All those classes has been the same. So this is all about Bybee require a plea processing and this Pema Indian Day diabetes they dissect. We are goingto use it for our fatherly scaling up the data in the next video. So stay tuned from the next video. We'll start how toe transform energy scaling off this does it. I hope you enjoy the sending this video and see you. 57. Rescale data - Standardize data: Hi, friends. Welcome back. So in this video we will see how to scale the data and how to standardize data. So let's start with the scaling off later. So we have seen in the last video one day Dublin object. We have imported Indian eight others present related information for all night participation. So deserter, some of the feature. Now let's apply this killing part. So this input data set so by B requires such a kind of scaling because absurd all those data all those datas are available in different different means. The very first feature which is black pregnancies nothing but the number of times woman has been pregnant. So it varies from 618 Another one is immediately goes toe 1 48 85 So the details are not available in a some particular. So this scaling of data is nothing but putting those they timeto one particular thing. And that thing we'll do with the help of this formula. This formula, taken from a standard a psychic learn document is and it's a very straightforward in in to do that you subtract the minimum value from that feature and do right by the range. And now, whatever the portion you have got, you just appended to the portion assigned to the max minus min level. So let's just manually force calculate that how we will get so legis. Take example of this 61 that and let just transform it. So the 0 to 1 ridge. So let me speak of 61. So for that we require first, actually what is the minimum value of that feature Number one and maximum value of the feature number, which is take a maximum begin. Just grab it like all row is a scholar. Zero column for The Guardian and Maxwell, you know, gets closer. Capital X So 17 is the maximum value, and let's see the minimum value. So minimum value zero says haven't seen is the maximum will. Now, let's apply this to this formula. So first, let's calculator X standard. So X is nothing but year. In our case, it's a six minus. What is the minimum values? Jiro. Be right back. The range range is nothing but 17 minus zero sound in minus you. So we'll get something like 0.35 29 now disease. Our next anger X resting. Now let's just applied this thing toe The Eskil was x skill Multiply by. We need to transform it to toe. Gee wrote wondering so maximum value can go toe one and minimum and you will be zero plus z . Okay, so we transform over data six to the 0.30 fight. Same way we can use this formula to transform all those. Well, you. Now let's use the Psychic Land Library built in model of Min Max scaler. And let's see whether we will verify whether we'll get this particular answer or not creepy transforming through the range 0 to 1. So from psychic Lynn s Kaylan 1,000,000,000 pork the pre processing modern and from pre processing while it will use the Min Max Scaler class. So legis please create a limb Max Keylor object. So mm killed I'll just assigned to mm Skilling and one argument. We need toe supply like a feature leads off each other and we want to put it labour between zero to one. The next task is we need to fit over input data. So we just used and then transport are fit and transfer, but at the same time, so just take freight and transport boat and we'll apply the whole X matrix, which is nothing but input. Because every single feature we want to scale from its original way to the 0 to 1 leg and like this, assign it to Billy Skill X. Okay, let's just print the scale axe so you can see that very first value we got it is 0.35935 to 9 inches. And from this formal also, we got exactly similar job 90.3 fight. So this way we can scale there now from any range. So the other desire teach our desired, which here, in this case, it's a between 01 So that is all about the scaling off data. Now let's start with the standardisation off. Now let's see about standardisation of data, how to standardize data and why we required to do this standardization. So let's first off their our input metrics. So if you observe this data, there are many. The Indies are available and reach we have scaled to the one particular range in earlier example. No, all those data are not normally distributed. Not question is to protectively everywhere the ranges are different. The meanings different for particular feature standard abuse and is also very. But we want to standardize these data toe one particular range of mean value and one particular range of standard divisional Now by be required this thing because many off the algorithm books based when you standardize this later. So we required this pre processing step, depending on which kind of machine learning algorithm you want. So let's apply this standard addition first on very first feature off pregnant. So before they let just calculate the mean value for first feature so we'll just take a first column and we'll apply. The main function so means 3.84 so it is not in a range of caution. Distributed way. We should be a zero and standard deviation Trevena between the range 0 to 1 same religious Catholic starting their division. So just use a study so standard everything is also three point practice practice six. But we want to convert the standard division, so the range 0 to 1. So let's convert it with the pre processing model over Skillon, so we'll use SK Land, but three processing and really important stand. That's killer so standards killer will make objective standards skill. And so I just made a skill. Er, it's a standard skill. And on the top of as killer Villa, play fit class for over input data set and let us assign it to the risque lacks. Let us print the scale so you can see the data has been transformed. The very first value. We got 0.63 instead of a clearly held value, which is six. Now let's just verify the mean and standard division off this very first feature, So that is skilled. So we'll grab the very first picture and we'll find me So you can see the mean is very close to the zero because it is saying that then Risteau. It is two minus 17 so it's already much close to zero. Same way. Let's just grab the standard division, protect one particular feature so we can verify. Did our data Iseman transport so you can see the standard revisionist. Once over data has been transformed from the range or mean value, which is having a clear for the very first feature 3.84 3 point artistic. So now it has been transformed Toe, close toe, Jiro mean and standard. It isn't exactly one. So that is a way we can transform the data in a standard days for me. So that is all about this video. I hope you enjoy the setting this video and see you in the next video. 58. Normalize Data - Binarize Data: hi place. Welcome back in this video we will see how to normalize the data and how to finalize the date. So let's start with the normalization off date and viral required those kind of normalization. So let's see. First, our input day does it. We'll continue dealing with our diabetes patient data sick. So if you see all those datas are available in a different different ranger, we hope holiday transform those data into some particular range. Well applied the caution distribution v off transformation. Now what does the normalization mean? Both of the technique which we have seen earlier that was applied on a column Lee, but one of feature level. So it will be affected across the feature. But this normalization of data will effect across the road. So that is the one fundamental difference with normalization with any other technique. So when you apply this normalization to your input data, it will a polite across the individual room. So why be require a normalization and what does the normalization mean? So normal edges and is nothing but individual record When you project it in tow, high dimensional space, we are trying to normalize it toe unit factor link. So if you take a very first track or so, this is a very first record, and if you normalize it, it will have a unique unit laying off one unit. Land will, no Y. It is helpful because if you see majority of machine learning algorithms with depends on Euclidean distance calculation. So when you want to compare one record with another record one instance off record with another instance optical. We always compared with Euclidean distance offering and when you have a normalization off post later with unit length. In that case, it will be a very bad beneficial. While comparing that day does it. So let's see how we can do it with some of the normalization model off psychic land. Skilling pre processing William Put normalization same way will make a class off normal. I just I just assign it better non non class and let's just supply. No, we can do Fritton transform individual your street entrance for both at the same time and we'll apply ex so you can see that the very first record almost it is Eunice unique link. If you want to calculate how it has become a uniquely and you can just make a square up individual component. You summit over it, and again you take us quero topic. It will be a one first squared, then some off all square. Then again, take up square root, so that will become a uniquely so you are normalising all your individuals according to the unique land. And that will be helpful when you're dealing with Euclidean distance kind off calculation. Let's see the another one like a finalization of leader now buying rice Data Mineiro's is in itself indicated you are trying to divide the daytime toe So different Chunk, Let's see the input data for us. Let's say all those data we want toe divide it into some poor, different buckets. So if let's say value is less than five, we want to put it it to like one bucket. And if the value is greater than five, you want to put it to some other bucket. So why this kind of things will be helpful? Let's say you have some broader, that's and you want to calculate some more advanced feature, or you want to re calculate some feature, which is not available. So based on let's say very first feature. So all those ranges we have seen in the last video that in just are available from zero to somewhere around 70 the very first feature, but you don't want to believe it. All those new medical value from 0123 and most of the all values between zero toe are 70. Instead of that, you want to divide all your first features off data into two differential. Let's say the number of times the human got pregnant, which is less than five, and with this in greater than five. So that means you're pocketing your data into two different chunks and your d calculating our feature, because for you, it is not very much a useful that how much, exactly. Sometimes it's not very much useful are based on your machine learning algorithm, which you want to apply. You don't want toe. Give the exact value up that particular feature. Instead of that, you are just trying to give that weather data is less then sometimes so hard it is great against them to so so how we can at you with the pre processing, More off the Ice Talents or Alexis Report scaling 0.3. Processing important. Buying a riser. Let's make a object of buying arises and here we need to supply the castle. Well, let's make a trestle value. Let's say the fight for the first column only it will will apply on the whole Set up later said only, but you can apply on toe very particular. The 1st 1 record only our foster feature only, Or the second feature on Let's Just Make a Free Transform and we'll apply for transformer input. X. So we got all those. 101010 So lets you see the very 1st 16 because excess gated invite becomes well again, 11 become zero because it is less than five on a date. It is great that invites it become one. So this way you can pocketing the date up between two different buckets, like somewhere loser less than some to sell it somewhere. User Great that in some cases, based on your machine learning, I'll go. Don't teach machine learning algorithm you want. So that is part up. Basic pre processing related toe transformation. Anna, the scaling related officer in this section We have seen that this four. Fundamental, technically, how to re skill data. How to standardize data hold to normalize the time how to finalize the leader. I hope you enjoy listening this video and Syrian tunics video with some more machine learning stuff. 59. Introduction to feature selection: Hello, friends. Welcome to a section on feature selection. So in this section we are going to see what is feature selection. Bodies feature engine in what are some of the benefits off peaches selection and what techniques were going to use for the feature selection? So what does it mean? Get feature selected. So before throwing all your data to the machine learning algorithm, it is one of the very important pre processing step when it's toe. One needs to analyze the data that have your future. Looks like read a particular sets off. Features are important for predicting your output or not. Sometimes there are some cases where some features are important in some features are not important, not important in the sense that they are not very good are contributing well to predict your outcomes. So in that case, you need to select the best feature out of that and throw all those transform best feature . They talk to your machine learning algorithms. Daniel Machine Learning algorithm will perform better now feature selection. You can see from other direction or other. We also like, from other perspective that it's like a real using your dimension. Let's say you have tens of thousands off features are really one. Let's take one example of image. How? What does the features election mean for this particular image? Let's take a me job, dog. Let's say this image now I want to detect whether particular image has a cat dog or any other animal. How will I detective? So rather than putting the pixel level intensity for this particular image directly into a machine learning algorithm, what I will do, I will try toe, predict some sort of feature. Let's say what is the average color intensity off this particular image? What is the history Graham of this particular image? What is the eyes color looks like for this particular sets of families off cat or dog? So disaster. Some of the you can see extracted feature out off your original data. So better than feeding all your original data directly to a machine learning algorithm, you can extract some of the important feature, which will immediately become one of the very important feature for predicting your output . Based on your object. You don't machine learning system design, so that is nothing but the feature selection that extracts important feature out off your morning now. What are the benefits off features selection Due to feature selection, it will definitely bro accuracy because you are removing unimportant Feature on top deck. As we are removing some unimportant feature, that is, a less amount of features are available, so there is a less training time also required as all off your unimportant Is that also over freaked to go Martin. So he as you remove your unimportant data that will reduce the possibility off over fitting . Also so disaster. Some of the benefits off feature selection in this section we're going to use some of the statistical base technique like uni variant selection matter features Election Matter videos is completely statistical based technique recursive feature elimination technique. Our principal component analysis, which is one of the very deterministic technique and statistical base technique Leavis selection we are going to use for all of this implementation. We are going to use the psychic land like so from the next video, we'll take one by one individual matter like a select Gables, then rickerson feature elimination and we will apply on our standard lied about sex, which we are going to use in this course a peanut Indian diabetes data set. So same data said we are going to use. So I had just based gold for that reading off this data set. So I have just read this data. Second toe data from object. The starting are splitting this day timeto input, values and output. Williams The 1st 8 feature from 0 to 7. It is input value in the last picture is class. Very well. That is a output video. So I just played it this later. Accessing put an adviser are from the next video. We will apply this individual matter like a select Gabor's records so featured elimination onto this date. So that is all about this video saving to next video. 60. Univariate feature selection: Hi, friends. Welcome to the video on uni Variant features selection matter. So it is the first feature selection matter which we are going to use to select the best gay feature. So let's say KZ called for and we want to select out off. Eight feature What were important in the last video. But input Data set contains a different feature for the be mined in diabetes data set and out of this eight feature, we want to select four best feature. So for that we are going to use this select cables class from the feature selection model. So let's just make some necessary important from S killer feature selection. We are going to import Select Game is class Now what does select Kate this class will do? It will try to grab the 1st 4 or first K best feature or best predictor for protecting your are good variable. So we helped open hate feature and out of eight feature. We want to select four based feature, so let's make object of this game is class. So as the first argument, we need to supply the score from that, how you want to score between your input features and output feature. So based on different scoring function, it will try to calculate the measurement parameter between your individual input feature and output feature. And it will give the school so that'll number off scoring functions are available like half underscore class. If, like a child square matter f underscore regulation, let's take anyone off them. So for that we need to put the necessary class so from a skill and seem, let's import, have regulation and wills up supply this F regulation based scoring function. So the input argument off this selective this class and under argument like how many total number of feature you want to select? Let's see out off eight. We want to select a for best feature and just assign Celek que missed. Now let's just fit this over input data sick. Why? So we had just putting data over fitting this modern off select Gabe is class for input and output data. So it will try to calculate between all individual features and our feature. The importance of individual featured the scoring based under different scoring function you away. So aspect laissez Les has assigned us freak. Okay, we have a smaller right, so ask for it. Now let's see. What is this poor underscore EC adults so it will return us. The individual scoring level off individual feature like the second feature, has a high yester well, so legis horrendous. All features name, So glasses are escorting value like a 213 next one immediately the 71 So it's 1/6 features her. The masses are under the second highest and then 46 corresponding to age. Actually, the last one has not been included because there is a total 88 as were killed in the last 20 output were able in the fourth basis 39. Now legis transponders later said, Well, just transform and will supply the input data set so you can see that let displayed over Original Data Center. Let's compared it after transforming water. The feature. It has everything the very first, which which is having a highest important which is a 2nd 1 toward it and 30 so which is corresponding to plus so plus is corresponding to 148 in the first record. So first record 1 48 has been preserved. Let's take a 2nd 1 like a 39 so 39 6 has been result, then a 71. Do do this for printing stuff. We were not able to see what this 40 six is the age ages corresponding to 50. So 50 has me, including 33.6 months available, so that is corresponding 6 to 1. Practice it 3.6. So this way it has selected the four best feature and vile trading time, which is nothing. But while fitting your input function, it has calculated this best under scoring function, which video provided the schools value with respect to all individual feature and after transformation, it has given us the four based feature based on this scoring starting order. So this pregnancy, plasma glucose living mass and it is the four based feature toe predicting your are put class radio. So in this section are in this video you seen this air figuration functioned in a number of other functions, like a giant square matter or select percentile. Let's select down a drone. Let's say we want to select the child's school and we'll just supply here the Czeisler and let's just again execute this select cables class and they're again fit this data. And again, let's just calculate the scoring so I'll just make it as it is. And I'll again Eddie exude this class so that we can compare bitter legal matter so you can see that schooling is different. But still, it looks like that ordering has been paid so that the 1st 1 here is same 1 200 Antarctica responding to 1411 ah didn't 100 anyone and 1 27 and when it even so, legis transform agency Whether boat has been given the same result or not same sex off four feature has selected. So so this first feature, 1 48 has been selected. Then there is another feature like zero has been selected in case off half regulation matter and then 33.6 Entropy Motor Simms. Three features is comin along Teoh the stool matter or pistols courting function. The one is different, so you can select a number of other scoring mechanism and you can try it out. That part of the best feature out off total features are really willing water. The feature you can eliminate. So because features election is equal into the feature rejects on also feature elimination Also. So that is all about the uni variant features election matter In this video we have seen in the next video, we'll see Donatella matter later. Recursive feature elimination. So that's all about this video. See you in the next video. 61. Recursive feature elimination: Hi friends. Welcome to the recursive feature elimination matter. So in the last video videos in uni, variant features selects and matter. So we selected a game based feature out Off total. Eight features were available. So instead of there in this video, we're going to see how we can eliminate features. So rather than selecting feature how we can eliminate the feature, So let's start it. How does it work? Let's see about first input data said What? It will go this recursive feature elimination. It's a record Su in a nature, so it will take all the 1st 8 feature it will supply to some particular selector. Modern art, a machine learning system. And it will calculate the score. Once it calculate the school. It will try eliminating some couple of feature in the next state prison. It will compare that particular school with our earlier depression. If it finds Stack, the accuracy has been improved, it will take the disease and raised under accuracy. So let's say it happened. Accuracy has been improved. It will eliminate those feature. If Chris is not improving that minutes, that future is still important. So let's see. Same from the scale on feature selection model. We are going to import records to feature A limb isn't so arctic. Let's make a class off Our has a first argument. We need to supply which model we want to give. So based on which modern, it will keep calculating the scoring mechanism or accuracy. Opta model. So let's make a largest dick regulation. So all this my country having discuss in this course yet, But from the next six and onwards will take all those pictures. So I am just taking as a frantic any particular modern. So let's take a largest degrees. And for the sixth largest integration is available in a still learn about linear model section. And from that you are going to put largest regulation. So the first argument which we are going to supply nothing. What estimator? Just the largest in regulation that can argument, which will supply what other total number off each other you want to select, so end features, facility and best picture to select. Let's say here also, you want to select the four best picture and let us assign it to the RFP project. Let's fix this later. Frick. Our input and output there will be no successful effected this data. Now let's find some scoring mechanism. So it has a ranking. Patrick. So this is the ranking Les Medias execute, and then we'll understand. There's another one they support. Okay, so you can see that it has given the true and false associative individual feature. Let's just bring this name So individual feature. So the very 1st 1 it is saying that true, keep it the plus. It is improved. Keeping press false skin falls baseball's must through its required because the important feature Freddie required age false it is not very much important. And the basement ranking it has been so 1st 4 feature it has given a value one so associated with frag class ready Marce and ready. It has been one remaining all four feature it has given that the argument I have another starting or the light crude people fight. The first sets off four feature has been given a value one. So this four features are important based on Dover in court, arguing like a logistic regression criteria, and we want to select number Ford this Egypt out off no legis transforming our wills. The supplied input argument. That's so it has returned. Four. Feature. Spring The Foster We'll just bring the first record only first through in all column. So 61 48 So first and 2nd 1 Is it true that this car is wanting to sit? Send 1 48 Let her decree and 30.6 0.6 Stoll selling. So we just got responding to bury and h very any ages. No, no, not ready in H, its corresponding to Mars and a pretty so Mars and the last one isn't 50. You just responding to age is not important. Mars and pretty easy is important. So Mars is technically 0.6 and are ready 0.27 We just corresponding to this way. So it has selected a four based feature. So Rick Unseemly. It has calculated the number off our Chris, and it has made after every prison. It has a calculated accuracy off the mortar that, after rejecting or eliminating some sort of creature, negated better reason. So that means that particular sets of features are not really much important. So based on that criteria, ethnicity pursue future elimination matter. So instead of this largest dick regulation you can give any other Martin Also Lakers. This is entry model Order Support Vector Machine. So no more will be hope discussing this course yet. So I'm not going to put any other modern because this video is about the introduction of this, how to eliminate the energy conceivably the feature elimination. So in this video of you have seen rickerson feature elimination matter in the next video, we'll see some more my personal animated feature selection and feature a village. So that's all about this video Syrian do next. 62. Principal component analysis: Hi friends. Welcome to the Wiljan. But in civil component analysis, send the last video we have seen to matter. One of them is that recursive featured elimination and under one is a university it feature selection man. So when is the features? Selection matter? And under one is a feature elimination matter, but it is not feature selection or elimination matter. It is like a feature transformation. Metal featured decomposition kind off matter. So the last two matter what we have seen in the last two matter, that is a month invidious common that it has selected, based on individual feature, that which featured is important, which not important and whatever particular feature is important it has, captain. But in this case, we are not going to select one particular feature, and we are not going to re eliminate in one particular feature. Instead of that, we are just transforming the sets up later into some other sets off axis. So the criteria based on what we're going toe transformed this daytime toe. Another sets off axis is that X is along with the alliance's maximum, through which we can just immediately classified a timeto number off different groups So that is what? The basic criteria that based on the sorting order off variants, it will take the maximum amount of energy which preserved in that particular component. So let's start it. So this is one of the featured decomposes and matter. And in one of the separate section I will produce you that how this matter walks in a lay mentum how we can implement what is the exit? Mathematics is running behind it. So in this section, in this video, we'll just use the building functions is available inside this s Cale on library, so from a still not decomposition will import BC. We have input data. Let's just displayed is input data Soto Total there are more than eight features are available. Now let's make a class off an inside that we want. We need to supply that. How many particular sets of components you want to take a stake? Let's take a three components and let us assign it to the PC. So let's freak input data, so we'll just supply extent. Fine. So we're successful if we did over input data does Now let's try to understand the gravity , say the components. So you can see that it has calculated the components in a different direction because out off eight different feature representation for individual particular one instance off, like off our input data. We want to re present now all individual or instance off record with just three diamond sent. So this is the one direction which is characterized wider. Eight different numbers or eight different dimension. Another direction is this one and the tower direction is this one. Now, once we found that in this particular direction, the variances maximum weaken just project are all input data toe this particular our direction, then another is in tow this particular direction and the third feature to get into this particular by accident. So this is the principal component is the second most variance principal component. And this is the Torah most so here, if you would just project all your input data to this particular direction In eight diamonds and space, you have a maximum. And as you will be result then little with men Egypt minimum and land little bit lower energy will be prism. So we want to take 1st 3 components. So that's why it has given us the metrics off total 33 components factor, and every component has been characterized by the eight different Davidson. Let's soc the explained variance so you can see that in the first direction they're variances maximum So 4 15,056 10 932 So variance has been given in a sorting order in decreasing order. Now legis transformed this daytime compared between put data and whatever the transform data record in our legal matter unit rated selection Macarena Recursive featured elimination matter BC just transformed. So after transforming this data, every single echo should be characterized by the three numbers are three diamonds. And because we have taken a three best component, we want to get select. So as argument while creating the instructor for the class we are given. Number of components is evil, so you can see that every single record is the first record. There's a second record is a total record, so every single lock or has been characterized by the three different numbers. So first, his minus 75 minus testifying minus seven for the first cycle. Now, if you compare with a Lear matter all along both off, this matter had just map this data one same mapping. So either eliminate particular feature are keep that particular feature. So if you are going toe, keep that particular feature while transformation, it will retain that particular feature. But in this case, you can see the numbers not looking like exactly same. So this 61 48 72 to judo point toe 6 to 7 has been completely transformed toe three different sets off number because in this case, it is not just like a linear mapping deck. We want to give one picture and we want to keep one particular if you want to eliminate for in particular feature in this case of yours, just transforming the different sets of access. So that teas that self different axis on which we helped completely mapped over data. So that's why the date output is completely different. But if you see the very first component this minus 75 minus 80 toe, this the first time is an invasion. A maximum energy off your data will be put, is own, and it is having a maximum of alliances. Associative Retek. We'll see in a ready graphical manner. And now when we learn about the exact this feature decomposition matter like a principal component analysis and linear discriminate analysis. You know, one particular section. So that's all about the principal component analysis. I'll see you into next video with some more features. Election matter. 63. Remove feature with low variance: hi friends in this video, we're going to see one another matter. Enough features. Lexan. Actually, it's a feature selection, not creatures section. So this is the one of the alliance mistress. Old Matter. It's in the last video. We have seen principal component analysis based technique. They're also the variance Trestle was a important feature. So wherever the your maximum energy and maximum reliance was exist, that particular component it will take into account. So how that matter? Off principal component analysis is the friend and off very stressful matter. So in case or principal component analysis technique, we were transforming data from one sex of access to the another sets of access. And after that, we're trying toe see in which direction the maximum energy is preserving. So in that case, we were transforming data toe another sex of excess. But in case off line straight, so it will try to calculate where Ryan's for each individual feature and wherever the Reliance trestle doesn't meet the criteria or where I trestle want make. In that case, that will just discard that particular feature. It won't transform feature toe another sets off dimension. So let's see over first thing put it as it so we haven't put like total eight features are available for a range test. So we are going to use s killer feature selection and inside did. We're going to use this very stressful class. So let's make her object of this violence, treason, and as a input argument, v need to supply. What is the trestle? Lay well beyond reach. Keep your later. You don't you just want toe retain your data beyond some threshold where I'm so let's make a trestle Anything. Now, One more thing here Need to consider that all the guys are available in a different different rage. So it may have funded it looks like Take this particular second feature the plasma glucose concentration level. He does a very high value, but still it may be possible that it is having a low Beuys. Where is instead of that? Here in 1/3 column. It may have a lower value, but it can have a higher variance value. So what will you do instead of just directly applying this very stressful Miller trestle matter? It is always best recommended practice that you convert all set So plate up, pretend off the mid Max scaler, which we have seen in the last section you know, featured pretty processing step. But what your daytime one particular range, like a 0 to 1 or minus one toe in this video year not going toe are protecting. But we can just apply to resolve and only like let's make a crystal like 200. And, like, just as I need to live free reading, let's feet over input and output later. So just freaked. Oh, like this assigned toe. We're different. Where are interest free function? So now we have created our input and output data toe alliance. This whole matter, it will calculate of Orion circles each individual feature, and it will. He was letting which particular sets off feature need to discard our It'll retain particular sex off feature, but it won't last for many, let's say will kill toga rise. Okay, so it has calculated Brian circles each and every individual column or attribute well corresponding to very 1st 1 The second it looks like better ears toe, then a tree. So that minister, in a very second feature on Lee, it has a very high variance value. So it is always best practice toe or recommend and put or keep this particular feature compared to other feature and those who okay hurt this featured also have even higher right ? So this is correspondingly lettuce Brenda Attribute Name. So this is the 5th 2015 will be test so test has even hired their eyes. Now this may rise higher new do we have Ah different different range of datas are available Once you convert this data in tow some particulars that's operate with Min Max Scaler class . You may not get the exactly same result. You'll get to something different. Listen, but the basic idea behind it you put your all day timeto one particular range and apply. Nder applied of variance trestle in the top off in V in those feature If you find that variance trestle meet some criteria are beyond some Krystle value. That means it exist with a lot of different energy for predicting your output intact. Particular timing said so keep it remaining. This started So it is all our about the machine learning or the design system designed it with Krystle point you want toe, choose it. So that is all about the very stressful matter, and in next video, we'll see some more matter related toe. Three Biz matter for the feature selection Syrian do next video.