Learn Data Science with Python - Part 3: Functions, Iterators & Generators | Tony Staunton | Skillshare

Learn Data Science with Python - Part 3: Functions, Iterators & Generators

Tony Staunton, Reading, writing and teaching.

Play Speed
  • 0.5x
  • 1x (Normal)
  • 1.25x
  • 1.5x
  • 2x
11 Lessons (1h 17m)
    • 1. Class Introduction

      3:13
    • 2. How to get the most from this class: Skillshare 101

      1:39
    • 3. Class Frequently Asked Questions

      0:54
    • 4. How to set-up your development environment

      5:08
    • 5. Jupyter Notebook Introduction

      4:27
    • 6. Python Functions

      12:34
    • 7. Default arguments, variable-length arguments & scope

      19:23
    • 8. Lambda functions & error-handling

      8:55
    • 9. Python Iterators

      12:25
    • 10. List comprehensions & generators

      6:22
    • 11. Practice Lesson

      1:38

About This Class

78a1c205

Welcome to this class where you are going to learn how to take your data science skills to the next level and develop a deeper understanding of the Python programming language.

This class kicks off with Python Functions. Being a Data Scientist means that you will be constantly writing your own functions to solve problems with data. Function writing is part science part art form and you will learn both in the first lesson.

You'll start by learning how to write simple functions, then move on to writing functions that accept multiple arguments and return multiple values. Throughout the lesson, you’ll learn how to apply your new skills to various data science scenarios.


In lesson 2, you'll learn to write functions with default arguments, which means the user won’t always need to specify them, and variable-length arguments, so that they can pass to your functions an arbitrary number of arguments. As a data scientist, these are both incredibly useful tools.


Lesson 3 teaches you Lambda functions which allow you to write functions quickly and on-the-go. You'll also learn how to handle errors that your functions will generate from time-to-time throw.


You’ve worked with iterators before when you learned how to write **for loops**. In this lesson, you’ll go deeper. You’ll look at how iterators can help you deal with large amounts of data - in this case, a large set of employee data that you will learn how to load in chunks using iterators helping to save computer memory.


Using what you learned in lesson 3 you'll build on your knowledge of iterators and learn about list comprehensions, which allow you to create complicated lists and lists of lists in one line of code.

List comprehensions can dramatically simplify your code and make it more efficient and will become a vital part of your data science skillset. You'll also learn about generators, which are extremely helpful when working with large sequences of data that you may not want to store in memory but instead generate on the fly.


We close this class with a practice lesson which will give you the opportunity to practice your newly learned data science skills. This lesson is done at your own pace so you'll have the chance to write your own functions and list comprehensions as you work with iterators and generators and take your data science skills to the next lesson.

Each lesson in this class is created using Jupyter Notebooks which means that you can download the Python code, experiment and improve upon it. You also get to keep the class notes for future learning and reference.

At the end, of the lesson is a final project to apply what you've learned.

What are you waiting for? Enroll now and take the next step!

Transcripts

1. Class Introduction: Hi, everybody. And welcome to this class Land data signs with heightened part tree functions, it aerators and generators. In this class, you're going to learn how to take your data science skills to the next level on develop a deeper understanding off the pipe and programming language. This class kicks off with piping functions. Becoming a data scientist means that you will be constantly writing your own functions to solve problems with data function. Writing is part science, part art form on your land boat. In this forced lesson, piping functions. You'll start by learning how to write simple functions, then moving on to writing functions. That, except multiple arguments and return multiple values throat a lesson. You learn how to apply your new skills to various data science scenarios. Lesson to look. The default arguments. Very real ink arguments and scope in this lesson, your land to write functions with default arguments, which means that the usual always need to specify them on variable length arguments so that they can pass to your functions an arbitrary number of arguments. As a data scientist, both of these tools are incredibly useful in lesson tree. You learn how to use Lambda Functions and error handling Lambda Functions allow you to write functions quickly and undergo. You'll also learn how the handle errors that your functions will generate from time to time . In less and four, you'll explore piping it. Raiders. You've already worked with operators before. When you learn how to write four loops in this lesson, you go deeper. You look at how it Raiders gonna help us deal with large amounts of data in this case, a large set of employee data that you learn how to load in chunks using it. Aerators helping to save computer memory less and five teaches you list comprehension on generators using what you've learned in the previous lesson, you build on your knowledge of it aerators and learn about Liz Comprehension, which allow you to create complicated lists and lists, would enlist all within one line of code. This comprehension can dramatically simplify your coat and make it more efficient and will become a vital part of your data science skill set. You'll then learned about generators, which are extremely helpful when working with large sequences of data that you may not want the store on memory but instead generate on the fly. We close this class with a practice lesson which will give you the opportunity to practice your newly learnt data science skills. This lesson is done at your own pace so you'll have the chance to write your own functions and list comprehension as you work with it aerators and generators and take your data science skills to the next level. In this class is practice project. You learn how to create a pipe calculator which draws on areas that you've learned in this class, such as creating functions, passing arguments and so one. Give the project to try uploaded to the class and let other students see your progress and give you feedback. Now, as you can see in this class introduction video, all the lessons are written with Jupiter notebooks. On my ghetto page, you have the opportunity to download all the code files for this class. So here we are here, learning data science of piping part tree functions, it aerators and generators. And there's all the code files 1 to 6. The class project on some F accuse. You also get access to every day to set used within this class, so you have everything that you need to take this class offline, unlearn at your own pace and keep practicing and practicing until you get it right. Piping used in this class is piping 3.7, and you can easily access that via the Anaconda distribution, which is free to download first Couple lessons show you have to set up your anaconda distribution your pipe environment on your super notebooks so you can hit the ground running when it comes time to code within the lessons. Thanks for listening, and I hope to see you in the class. 2. How to get the most from this class: Skillshare 101: Hi, everyone. Now, in this short video, I'm gonna show you how to make the most out of this skill share class on have to enjoy our journey together, learning data, science with fightin. So as you can see, I have a draft of my course open here, and you can see my classes on the right hand side, there are more to come. But looking here now, focusing in on the video player window you can see on the left hand side the bottom lower, left hand side speed button so you can increase the speed. If you find Trudy Trudy classes I'm speaking to slowly so you can go all the way up to double a speed. You can wind it back by 15 seconds. If you want to relearn something or re listen to something over on the right hand side, you can view the notes, so that's an important part of people Sometimes. Miss, I often add notes to certain classes if people have feedback or questions, or something might have changed in the technology between myself teaching this class and you taking this class and then on the right hand side, you can switch it into full screen. And don't forget you can always higher and lower the volume. Suit your own needs. If we just scroll down a little bit, you'll see reviews community projects and resource is so in. The community section is the best place to go and ask for help should you need it in the projects. And resource is tab is where I add in the class project. Any resource is of files that you might need to have your learning journey up the top. You'll see the follow button, and they encourage you to follow me because I often release updates to my students, such as new competitions, new projects, new challenges, updates to the course on much more. I have a very active instagram feed as well. So check that out to stay up to date on the course on new piping, data science techniques and everything related to piping on this course. Now, finally, if you've taken and enjoy this class, please do leave me review with some feedback on what you liked or what you'd like to see improved about the class. That's it for this short video. Thanks for listening, and I'll see you in the next class 3. Class Frequently Asked Questions: Hi, everyone In this short video, I just want to show you where to go to get some additional help with the class on with uber notebook. So if you head over to get hope dot com for its last T Staunton, that's my get home page on my profile. See a number of repositories click on Learning data Signs were fightin. Then click on Part one. Introduction to Fightin and you'll see here Class F A. Q. So click into that and let's just have a brief look what kind of FAA cues we have. So do I need to have super notebooks and Anaconda installed? I'm not gonna go through each of these. I'm gonna let you read these at your own pace. Where do we get the resource for this class? Well, that's on. Get hope. How do I know where my notebooks are being saved and so on? And the last question is particularly useful. How do I get help if I'm stuck on something? So again, just a short video to let you know that I do have an f A Q page, and I add that all the time with common questions that come from students such as yourself to go to get hope and check that out. If you have any problems or don't forget, you can always drop me a message in the community section off the class. Thanks for listening. I'll see you in the next class. 4. How to set-up your development environment: Hi, everybody. And welcome to this class in this class. We're to go. We're going to discuss the environment set up. So exactly what those air a development environment looked like for coding pipe. So this classes tree main objectives that is to install piping would under conduct on, um on the home page here that you can see next downloaded zip files of all the class super notebooks and finally open and explore our Jupiter notebooks. For those of you who may not know what Anaconda is, it is a distribution of piping which includes not only piping but many libraries that will be used to wrote the upcoming classes. Anaconda is an all in one installed that is used depopulate data signs on machine learning . When you download and install Anaconda, the Jupiter notebook development environment is also installed. As I mentioned in the introduction, Jupiter is a development environment where you can write cold display images and make notes . When it comes to data science and machine learning. It is the most popular I d for exploring and analysing data. Before we go, any foreigner I'd like to point out that of you are an experienced piping user already have a development environment set up that you were happy with. Please feel free to continue using your set up. You do not have to use Anaconda or Jupiter to be able to follow along with the classes to come. The piping code that we're going to use can be used in any I. D. So here we are on the Anaconda home page and you can find it out anaconda dot com Now the Anaconda home page may look different when you come to visit due to updates and things like that. The main thing that we're interested in is the download button in the top right hand corner kick on that. Now you get a bit of its summary of what Anaconda is on. If you scroll down, you see a couple of options. So the first big one that jumps out at you is that Anaconda is available for piping tree and piping to We're interesting piping tree. Keep in mind that when it comes to this page, diversion number may have changed instead of 3.7 and might be treat 0.8 or 3.9. But that's OK. Piping tree is what this course is all about Before you select your pipe inversion, make sure that you are downloading for your correct operating system. So, as you can see here you have windows. You have Mac OS, which I'm aren't. You have. Lennox, if you were wondering whether you should download the graphic installer are the command line installer. The graphic installer can be a lot simpler to run on install as it runs. It gives you help. A helpful step by step guide. Truly installation process. She can see here Graphical installer. So click on that to download it for your operating system. Once the download is finished, go ahead and open up the file. If you're on a Windows machine, it will be a dot exit. Or if you're on a Mac, it will be a DMG ID away. Select your file now the Mac installers pretty seamless. But if you're on a Windows install, you need to pay attention to some screens that pop up in particular this screen here. So as I don't have a Windows machine, I've taken a screenshot of the insulation prompt. So on windows. When you get to this screen, advanced installation options, the forest box here will be on ticked. Make sure you select it even though it says not recommended. The reason it says not not recommended is because if you have already installed evasion of piping, checking this box will make Anaconda your d felt version of piping. But again, that's OK. Make sure you check this box before we proceed, then continue with the remainder of the installation. Now, when the installation is complete on a Mac, you will see Anaconda Navigator added to your list of applications. So I have a look in my application list. Here. There we go. Top right hand side Anaconda Navigator didn't windows in the lower left hand side. Besides start button. If you run Syria for Anaconda, you will see the anaconda options pop up. If I click on Anaconda Navigator, give it a moment for the interface toe. Open up. Okay, so here we are in an Anaconda Navigator home page. This is how you can lunch your Jupiter notebook on many other applications that comedy Anaconda Navigator on whether you're on a Mac or a Windows machine, this interface is going to be the same. This is essentially as I just said how you will access your development environment. What you need to do next is select lunch on the Jupiter notebook, and you should see a browser window open up with your computer files in the directory. Something like this. When you have Jupiter open running, just a word of advice. Make sure that you have a modern Web browser selected as your default Web browser. Something like Internet Explorer is not gonna work very well with Jupiter. You're better off using Chrome Edge Safari or Firefox something. Mother. Another thing to point out here is that when you're using Dupin notebooks, even though we're inside the Web browser, we're not actually connected to the Internet. Jupiter's just using your Web browser as an interface. Now we have Jupiter notebooks installed on pipe and hopefully, successfully. If you've any problems, let me know in the community section. Let's head on over to get hope and see how we can download all the class files that you are going to need now here, here on my get home page, And that's at get hope dot com ford slash t stardom and once again select a repository that you're interested in which is learnt data science with fightin where you can do here on the screen is clone or download the files. What you want to do is you want to download them, so download the zip file. Okay, so once your file downloads, you'll need to unzip it. So, on Mac, that sometimes happens automatically or on Windows. You'll need to use your default on zip file utility. So here we have learned land data signs with titans. That's a massive file on, as you can see, what in that we have part one introduction to heighten and all the classes that come with it. Let's jump back into Jupiter. As I said, here it is. Here would in my downloads folder you might need to browse to this folder in your Jupiter Explorer. Now, if you don't want to work into downloads folder safety on zip file to a location where you do want to work from. So if I click here introduction of fightin, there we go. All my Jupiter files ready to be opened on explored, and that's exactly what we're going to do in the next class. Thanks for listening, and I'll see you there 5. Jupyter Notebook Introduction: Hi, guys. Welcome back now. In the last class, we looked at installing an opening up Jupiter notebooks. We also downloaded the class files that you're gonna need going forward to kick this class off. I'm going to show you an alternative way to open up Cooper notebooks, which is from the command or the terminal window, depending on your operating system. So let's do that right now. So as you see here, I have my terminal window open on. I simply type in Jupiter notebooks. There we go. Give that a moment on. What that's going to do is open up a browser window with your file directory. Something similar should happen on your system. So as you can see, my default Web browser opens up with my computers file directory. I never gave in here toe where my notebook file they're saved on. Then I click on the one I want open. And here we are. If you have already downloaded Dupin notebook that we did in the previous class, you will have the exact same source code that I have in front of me Now. Now, in this class, I'm going to give you a brief very high level overview of Jupiter on its interface. So here we are, in a file structure with the all the files that we downloaded relevant to this class. But over here on the right hand side, you can see new we click on new weaken selector environment. So piping tree you might have several environment. If you have previous versions of piping installed on that's okay, select the one that you were working with or the most up to date one. So select piping training. In my case, on I get a new cell. First thing you should do is retired to yourself. So I'm just gonna call this example. There we go. Perfect rename. As you can see here we have a salad bar on This allows us to enter in piping code or just general Smackdown text. And look at that in a moment. But in the cell, if I typed print brackets quotation marks Hello. Hello. World is not how the old start Hello world You can see here This is a complete pipe and print statement. I click shifted, enter and I get the output Hallow world. I could have done just the exact same Think without the print statement by saying Hello, World again shifting. Enter and we get our output truck. This class, you'll see me used to print statement to generate output. There will be times when I don't use it. The output is the same with one difference here in that line. One. When I use the print statement, I don't get to the left hand side and out statement or an out market within line number. But as you can see down here in line to into, out to, that's the main difference. So I use it interchangeably throughout the classes. So have you seen the using print and not using print? Don't worry. You can wrap it all in the print statement or without the print statement, whichever makes you feel better. If you would like to add additional cells, you can do so with the following matters. So as you can see here, my cell is active and you know it's active because it's surrounded by a green box on the courses inside. If you press escape, it turns blue. This means that the sale is not active. We can do here now, a select A, which inside of two Sal above or select B, which insert to sell below. If you're working in a cell, so print hello, You can also insert a new cell below by instead of pressing, shift and enter to run yourself, press off and enter there we go automatically into a new cell. Now, at least out of the beginning, you can also add normal text into your Jupiter file. So again, press escape to make sure our cell is not active. You condemn press M or up here the drop down. You can select code mark down or whatever else you want to use. So I've selected M, so I click, escape and click Enter to reactivate that sell on. Now I can just type hello. Let's have a look at the output. Nowhere put because that's just normal text. So that's how you would add notes to your Jupiter file. So if you're writing a long or follow along with the classes, you want to make a note. You want to say I need to revisit this example. You could make a little note here for yourself. Or indeed, you'll see. As we go, you can produce massively big reports that tell us, Tell a story about the data that you are examining that you can use to print, save and export to share with other people and all sorts of way to export. And to do that, you just go up to the top parents. They file download as, and you can see all the options that you have here to export a Jupiter notebook. Some other things are important to know that if you're in Jupiter on your running your code away, sometimes you might want to restart. So up here you have Colonel, you can restart. You can just restart the entire notebook. You can restart in, clear the output and he can restart on ruin all the cells again. So that's restarting. Clear the output. So as you can see, all the output is gone except for the plain text. So again, I can click, shift and enter shift and enter on my output is there. That's just something to know if sometimes your programs get stuck or to taking too long to run. Now, if you would like to see the keyboard shortcuts for order options that you have in Jupiter , you can go help and keyboard shortcuts on here is an entire list off what you can do within Jupiter using the keys. I won't go through every one of these. I leave that to yourself. Okay. Thanks for listening. That's a brief introduction into Jupiter on. We'll be learning a lot more as we move to the classes. Thanks for listening. And I see you in the next class. 6. Python Functions: hi, everybody. And you're very welcome to this lesson where we're going to be discussing on learning how to use and right piping functions. The previous two classes leading data signs with piping parts one and two have been stepping stones to reach this point as it is now time to learn how to write your own functions on how to apply them to the world of data signs. This lesson represents a major milestone on your journey to becoming a data scientist, as knowing how to create your own functions is fundamental in industry. In the examples ahead, you will learn how to define your own functions without parameters to find functions but one parameter and the fine functions that return a value. If you have completed the 1st 2 classes off this learning Siri's, then I don't need to tell you how important pie tins built in functions are on. As your knowledge of piping grows, so too will you. Reliance on them to progress is the data signs or to perform data analysis task. You'll need the ability to write your own functions that have functionality specific to your programming needs. This class and the lessons that follow will give you that ability. Let's now take a look at how we go about the finding our own functions. So let's jump into an example right now. So as you can see here in this code example, I start that with a common to create a new function called Square. So to create a function, we used a keyword D e f. We then give it a function name square. In this example, follow what brackets on the colon. This line of coal is called a function header. As you can see, there were no parameters within the brackets. So nothing here within our two brackets to complete the function, we add code beneath the header. So take another look again that cold. In this example, it consists of two simple lines of code one creating a new variable called square value, followed by the print statement. These two lines of code form what is called the function body. It doesn't have to be two lines. It can be as many as required for your function. Also note that the body of our function is indented from the function header. So to use this function, we have just created. We do what is called calling the function. Calling a function will execute code within the function body in our example above, this means assigning the value of four square to to the variable square value and then printing it out. You already don't have to call functions. It's done in the same way as we do a pie tins pre built functions. So let's have a look at how we call the function we just created. We simply type in square brackets. If I hit, shift and enter, I get an error because I haven't shifted. Enter on our new function. So shift and enter and there we go Square 16 which is fourth of the power of to you might be thinking that the function of of is pretty limited. I mean, how many times do you want to square four by two? What about squaring and number ordered and four add dysfunctionality. We must add what is called a parameter to the function definition which is in between the brackets in the function header. So here we go again, we've simply added in a parameter in between the brackets of our function header here on that parameter is called value. Now you'll see now when we create a new very well here in new value instead of having four , we have value so value to the power of two, which could be any arbitrary value as well. See, now in a moment, shift in and around that on no. Again we're calling a function square, but this time we're inserting a value. So shift in Internet. There we go. So in the example above, you can see that I placed a word value in the brackets of the function header. When you call the function, you can do so by passing arguments in it. The function square now except a single parameter and prints out its squared value. But what if we do not want the printer value directly, but instead return to value to a variable and just a point out? When you defined a function, you write parameters inside the function header. When you call a function, you pass arguments into the function. There will be times when we don't want the print out the value, but rather have it returned. Saved to a variable which continues elsewhere in your program. We do this by using the return key word. Let's take a look at an example below. So again we have our square function on. Everything is the same, except we get to hear on. We have returned new value. So the square function that returns the new value variable by using the return key word. Now, instead of calling square and passing it an argument, as we did previously, we can assign the result of square to a variable. So here we have square here. We're passing in the argument four, but we're assigning the result off this call to the variable known and then we're printing out known. And there we go 16. As your functions become more advanced and complicated, you want away other than the normal option off commenting to describe what your function does, we can do this with the use of doc strings. Doc strings are used to describe what function does. Not only will they help you remember what your function does when you come back to it after some time away. They also help on you the coders who might be reading your code Doc strings are placed immediately after the function header and place between triple double quotes like this. Let's add Doc strings now toe are square function. So here we are with our square function, and we simply started off with triple double quotes and closed with triple double quote on Insert the description. We just explored the basics of defining functions. Let's look at some more examples. So in a first example here we're defining a new function called Shout. We're Putting in a dock string, which says, Print out a string. And then what we're doing is we're creating a new variable called Magic Underscore Word onto dot vary, but we're assigning fightin plus exclamation mark. We then print out the magic word to call her function. We simply called Shout with no parameters in between the brackets. Let's print this out fightin exclamation mark. Pretty simple. Let's now update our shout function by inserting the parameter word. So as you can see here, it's the exact same. But this time when we called shout were inserting an argument Monte piping. So let's print that out. Shout with an exclamation mark, which in this case, is Monte piping with an exclamation mark. Let's take a look at another example, so again we define our function, shout we insert the prominent word again. We have our doctoring. But this time we're returning the very what magic word. Writing and printing it straight out. So in this line of code, now we're passing the phrase monthly pipe to the shout very but which is called named after . So here we are here what are named after variable on the left hand side, named Underscore after equals Shout Monty Python we print out are named after variable and there we go month eep iten. So if you didn't know it already, just to be clear Fightin is named after the TV comedy Siri's Monte. Fightin Let's take a look now at multiple parameters on returning values. So in previous examples are functions only accepted one value as examples above word or the square root. But what if we wanted pass in more than one? What if we didn't just want to raise a value to the power of two, but a different number? We can do this by allowing our functions to accept two values instead of one on how do we do that or very simply here within a parameter section, we add in a second value, so we have value one value to doctoring. Tells us what this functions gonna do. Raise value one to the power of value to so we could have our new variable square value equals value one to the power of value to we. Then return square value on in this line of code were passing to our function. The arguments tree and 200 m printing out the result in the next line of code. So let's have a look at what that looks like. Nine. Perfect to laboratory arguments into our function on the correct result. Output it if we want our function to return multiple values, we need to create objects known as two poles. A tupelo is similar to a list in data can contain multiple values. However, a tupelo is immutable, which means its value cannot be modified once it has been constructed. Two pills are creating using normal brackets. Let's take a look. So here, as you can see, our common simply, says a tupelo with four elements. So we're creating the variable numbers equals opening brackets. 12 tree foreclosing brackets. Let's print out what a type of the variable numbers is, as you can see, it's a tube. We can unpack it Tupelo into several variables. So again we call our to pull variable, which is numbers on what we do next is we create a syriza variable separated by commas. A B C D equals numbers, so we're assigning numbers to these variables here. Let's print out the forced variable Siri's a Let's print out be. And so what? We can also access the individual elements of a tube like we do it list, using zero indexing. So again, very simply, we creator to pull here. Let's access the second element off the tube, which is, too. Let's access the fort for a go. So as you know, zero indexing means that our account started zero, as we've seen in the code example above when passing multiple arguments to a function the order matters. Let's expand the proves example to demonstrate this. So here again, we're defining function raised. Two values were passing a two parameters value one and value to on what we're doing in the function body, as you can see by the dark string, is we're raising value one to the power of value to and vice versa. So, very simply, we have our first variable square underscore value one and two that we're assigning value one times value to our next variable called square underscore value two equals on. We're assigning to that value to to the power of value one in this line of code here, we're creating a new to pull onto data. But we're assigning the variable square value one square value to we returned a new tube all so that it can abuse elsewhere within our code. Next, we're creating a new, very well call result. And to that, we are assigning the function raised two values on passing in the arguments tree. And two, we don't print out the result. Let's have a look. Nine and eight. There we go. So as you can see, the order does matter because we have two separate results. The first tree to the power of to is nine on, then two to the power of tree is eight. Okay, let's take a look at another example passing in some more parameters. So we're creating a new function called shout so d e f shout on. Then it were passing into prominence word one on war too. So what's this function going to do? But it's gonna contaminate our strings with three exclamation marks. So how do we do that? First we define a new very recalled Shout one onto it up. Very well. We assign word one plus a tree Exclamation marks. Next we create a new variable called show too. Onto it up variable were assigned it war two plus tree exclamation marks. Now what kind of help? What do we want? What we define a new very well. Call New Shout onto it out Very well. We're assigning Shout one plus show too. So we're going cap needing to two words that a usable in put together we diameter a new shout and two new shout We pass in the words Congratulations on you And we assigned that output to yell Then we print out yell congratulations you not the cleanest of output but you get the point Here is the same function again. But this time we have constructed a new tube with shout worn and show two on assigned to the variable shout words So as you can see then what we can do is on the same line. Create two new variables. Yell one yell too. And pass it. Shadow. There you go. Congratulations. You to two new lines. Congratulations. And you. Let's now take everything that we have just learned and put it together. We'll start by importing panders on a simple data set that I have downloaded from IBM, which you can also find at this link here the file is saved and available as a resource from a ghetto page. The link is here. The forcing that we're going to do is create a dictionary containing data about the number of staff in each department. Well, then expand this code to become part of a function. So let's have a look. Import Pandas PD. Nothing new. Let's create a new data frame and we do that with the read See SV function on. We don't pass in as an argument the location of her data file. Next, we initialize an empty dictionary so that dictionaries called department underscore Count equals curly brackets for an empty dictionary. We then extract the column that we're interested in, so we're creating a new very recall department onto that. Very well. We're assigning column department from our data frame. We do that as we know from previous lessons with square brackets. Now let's iterating over department column in the day, the frame and we do that with the Line four injury in department. And then what we're doing is we're using if statement to say if an entry is in department underscore, Count darkies add one. So department underscore account plus one. So that's gonna is rate over. And every time it finds the department, it'll add one to our account else. Department count entry equals one, which means if it sees a replica of department name, leave it as it is. Let's print out a department camp now from our data frame, and there we go. So what? Energy departments? We have 446 staff in sales, 961 Research and development 60. Tree in Human Resource is excellent. So what we have just done is created the functionality for iterating over entries in a column on building a dictionary with keys as the names of departments on values as a number of staff in the given department. So as we know, dictionaries have formed using key value pairs. The key is on the left, separated by a colon value on the right. Next, let's define a function with the functionality we just develop. Returned the resulting dictionary from within the function on Call the function with the appropriate arguments. Let's give this a go. OK, so we start off with same two lines. Now we're creating a function here on this function is called Count Underscore. Entries were passing that two parameters. One is our data frame. On one is a column name again. We create an empty dictionary and then we extract the column that we're interested in. From the data frame, we have the same four loop here, but down here now, in the result, what we've done is we've called their function on. We've passed in our data frame on the column name that we're interested in so we could change this column named anything within our data set on, we will get the result. So there we go. It's not adds a lot more functional to our code and that we're not limited were not hard coding in a department name as we did here. Instead, what we're doing is when we call our function, we can insert a new column name here, which gives us much more flexibility. Okay, that's it for this introduction to piping functions. Thanks for listening. And I see you in the next lesson. 7. Default arguments, variable-length arguments & scope: Hi, everybody. And welcome back in this lesson, we're gonna be looking at Pipkins default arguments on variable Lent arguments. Now, in previous lessons, you've learned how to create your own functions, right functions with multiple parameters and functions that can return several values using two poles. In all of these examples, we've bean defining variables and using them without any issues. Well, at least I hope you've been used them without any issues. If you're having problems that make sure to drop me a question in the Q and A section or in the community for him now, you are going to learn that not all objects you create are going to be accessible everywhere within your scripts. This is where the idea of scope and user defined functions, which are functions written by you, comes in scope, tells us in which part for program an object may be accessible from now for the context of this lesson and those to come, names come affair to a variable name or the names of functions. As we have already seen. Very boats have names as two functions themselves, their tree types of scope that you need to know about global which is defined in the main body of a program, local, which is defined inside of function. Names cannot be access outside the function definition or built in names and pie tins pre defined, built in modules such as print and some which we've seen a lot off. To make things a little clearer, let's take a look at some examples. So here's our first function here, and it's called Square and we've seen this in the previous lecture. I'm not going to call this function on passage. The argument. 10. So let's see what happens when we shifted under. There we go 110 to the power of 10 inner function square. We have a variable here called New Val, So new value. Let's try and call that very well. Outside the function after has being executed, we get an error and, as you can see here, new violence not to find so politeness not recognizing this variable, even though we have called it music above. So as you can see, we cannot access the very one new value after the function execute. This is because it was defined only in the local scope of the function, the name new value was not defined globally. So local scope, as we just summarized above is, would end the function definition. So what if we define the name globally before defining and call and function on? How do we do that? Well, if you look at a quick let someone here, global variables are defined in the main body of the program. So here we are here, create the global variable. New value equals 10. So we're creating it before a function definition. Now we have our function here written the exact same. We call it passing at the argument five shift and enter 25. Now let's see if we can access the very but new value. 10. No, Anytime we call the name in the global scope, it will access the name from the global scope. But why is the value of new value 10 on not 25? Any time we call the name in the local scope of the function, it will look first in the local scope, which is why calling square five results in 25 on not 10. If pipe and cannot find the name of the local scope, it will then look in the global scope. In the example below, we access new value, which is to find globally within the square function. The global value access is the value at the time the function is called, not the value when the function is to find If we were to reassign the value of new value on call the Function Square, we see that the new value of new value is accessed. So our names are a little bit confusing there, but as always, wouldn't example and makes it likelier. So we're creating a global very well here in new value. In here we have a local variable new value to Let's have a look. What do you think is gonna be the output? 100? OK, so why is that so simply we have a variable appeared new value, which is 10. Let's square that and assign it to the very about new value to their ego. Return new value to Let's call it here on power to tree. Today you go 100. Ok, now let's look at another example New value 20. So we've reset a new value birth variable. Let's square that passing at the AGM entry on we got 400. Okay, Perfect. Now, whatever you want to change, the value of a global name would in a function call This is where we can make use of the keyword global. So we have everything the same as before. Except this line of code here we're using the keyword global were saying the access new value. So here, within the function definition, we use the keyword global, followed by the name of the global variable We wish to access and change Caller function square passing at the AGM entry New value 100 when we call new value, we see from the output that the global value has been squared by running the function. Let's take a look at another example this time using strings. So the first time we do here is create a new global variable book. Underscore Name equals my book. That's the final function called change book. So what does this function do? Well, as we can see in the dark string, change the value of the global variable called book on the score name. So to do that, we first have to access the global variable within the function and we do that by calling up the keyword global on calling out the variable name that we want to access. We don't change the value of this variable. So we say Book, underscore, Name equals on. We give it the new value. Now the first thing we want to do is print of the book name. Then call her function Change book as we know our function is going to change the book name to the book about me and then printed out. Okay, so this is pretty simple. Once you start using it and start practicing it on, I have to say one of the things to learning pipe and getting over the learning curve is practice practice practice. So again, with this piece of code here, the first thing we did was create a variable a global variable called book Underscore Name my book. We then printed that one out. We then called our function, which changes the variable book names value, and it's going to change it to the string. The book about me and we printed out. So we have our to book names there. As we know, there are an awful lot off functions built into piping such a some print on much more we've seen low to them. So how do you view all of those? Well in Juba notebooks, you can import built ins, which is obviously Pipkins built in functions, and then list out all of them on. There it is. Which of familiarity are functions growing? It's time to look at nested functions. So for this section, let's consider a couple of functions one called F underscore inside and that is gonna be placed within a function called F underscore outside. So a function within and without the syntax of dysfunction would look something like the following. So here, just code owned, actually run. It's just a bit of pseudo code. So, as you can see here, I'm defining a new function called F Underscore. Outside, the purpose of dysfunction is that demonstrate nested functions. We would then have a variable which would be signed an average value. What in dysfunction? We now have a new function D e f f underscore inside same thing again we return value in the example above, we have X reference inside function f underscore. Inside, we have also reference X In the elder function, f underscore outside. How does piping handle this piping for searches within the local scope of F underscore inside. If fighting does not find X, it searches the scope of F outside, so it works on an inside out approach. The function F outside is what is known as an enclosing function as it encloses the function f inside to not that hard to understand or grasp. If piping cannot find X in the scope of the enclosing function denser, it is global and finally built in scope. Why might you want to use nested functions? Imagine. We want to use a process a number of times within the function, a function that takes several numbers as parameters and performs the same operation on each of them. One way you could handle this is the right of the process several times. But this matter would not be able to scale. What if, instead, we need to perform this process 100 or 1000 times? Well, let's look at an example. Scroll down to put this at the top of screen. So here we are now, creating a new function called adding on. We passed it four parameters x one to the x four. So what is this function gonna do? And that description there is not right. Let me just take that out. It's going to return four values the result of four values plus seven. And as you can see here, I'm actually missing one. So, addition, four, you can tap the auto Complete X underscore four plus seven. There we go. So return addition, more in addition to Edition Tree Edition. For as you can see, this is not very scalable. Alternatively, what we can do is to find an inner function would in our function definition and call it when necessary. Okay, so here we have our both function re factored. So add four numbers with seven. But now we have an inter function called inner and we assign not the parameter X. So return X plus seven. Okay. And what is exe? While X is going to be whatever we assigned as an argument when we call our function adding on and you can see +12 tree for shift and enter and up. There we go. +89 10 11. Much easier. Much more scalable than what we tried in the example above it. On what is the type of our function, adding it's a function, as we would expect, excellent. What we have just done is created a nested function. The syntax for a nested function is the same as it is for annual function. When using the scopes, remember that they will search in the following order local scope in closing scope, global scope and finally, build in scope. This order is known as the L E G B roll. When using nested functions, there was a term known as closure. Closure means that the nested, or inter function, remembers estate of its enclosing scope when called anything to find locally, Indian closing scope is available to the inter function even when the error function has finished execution. Now this is a little bit hard to understand, I grant you, but you'll get the hang of it when we start looking at some more examples. For now, it's good to know the existence of closure. You never know somebody might talk about it and work today. That's how I look at another example. So we're creating a new function called my Underscore Echo, and we're signing that the parameter and So Return and Inter function. That is the purpose of the function. My underscore echo within that function were defining a new one called Inner Echo. Once dysfunction going to do we have a variable onto that. Very. We're assigning word one times n. So whatever is in my echo as end, it's gonna multiply our word one. Then we're going to return echo under school word. Next we return inner echo, which makes it available to the outer function. And then we have exactly what we want a function to do. So we create a new variable twice on to that variable were assigned my echo on we pass in the argument to which represents the parameter in next we called eco tree times. And we do that with a new variable calling her function my echo tree. Now let's print out what we have just created. There we go. Hello. Hello on. Hello. Hello. Hello. So this example here will take a little bit of practice on a little bit of effort to understand. But believe you me wanted clicks. It is well worth it. Just keep at it and keep out. Don't forget you can always ask me questions Now, at this point, you might be wondering, Is it possible to alter the value of a variable defined Indian closing scope While it is on , We can do it with the keyword, none local used within a necid function. You can take a look here, so my code here is the exact same. But as you can see within my inner function, I have to keyword none local and have called out the variable that I want to change. So if we shift and enter and up Hello, hello, hello, hello with exclamation marks. So our changes implemented. Let's look a default and flexible arguments in the next example, let's work with a function that accepts multiple parameters. So let's assume that one of these parameters has a common value. A common value could be anything, such as days in a week or a year. In the example to comb, we're going to provide one of our parameters with a default argument, unless otherwise specified. Applying a default argument or parameter, it's relatively straightforward. You need only add in an equal symbol after the parameter, followed by the value in the function header. So what exactly does that look like? Well, here we are with a new function squared and we're passing it to parameters. Number on P O. W equals two. So what we just don't now is assigned a common value. Which is too, too The parameter p o w. We have our doc string. We have a very big crater called Squared Underscore. Value equals number times P O. W. Which in this example is too. Then we return are variable. Next, let's call squared on passing the argument 99 to the power to 81. Next we have nine times. One. Why is the output of that? Just one? Because I said in the opening summary, default argument works as long as nowhere argument is defined or specified. But in this example, we've passed in one, so nine to the power of one is just nine. Let's go again. Squared 99 to power 5 59,000 Let's look at another example. So in this example we've modified are my aka function from above and we've passed in the parameters. Echo equals one and scream equals faults we start off. It are variable, echo underscore. Word equals ward one times echo. So the number of times We want to repeat the word we have an if statement with their function which capitalizes the echo Underscore Ward variable If scream is true So what? That What is that? If statement look like? Well, if scream is true capitalized and concoct innate with exclamation marks The very about Echo underscore. Ward knew we forced create the variable which is here to that. Very. But we assigned Echo Underscore war dot upper closer exclamation marks. So if the parameter scream is true, we're gonna put our word in the upper case else echo. Underscore Word else. Echo. Underscore. Word on the score. New equal. Simply eco underscore Word, pleasure, exclamation marks. So, in other words, no capitalization, we then return are variable. So what do we do next? When next We're gonna call, shout, underscore Echo with the arguments. Hey, Echo equals five and scream equals true And I think by now you know how to do that. We create our variable wit underscore big on the score Echo equals r function Name my underscore echo with our arguments And then we're calling my echo again this time with the parameters Hey! And scream equals true again So Let's have a look. Hey, hey, hey. Let's change this one here to false see what we get. There we go. So, as you can see, it is no, it is not all in uppercase for a second call. So far we've looked at fixed and default arguments. But we won't always know how many arguments that we may want the past to a function to deal with this. We use flexible arguments. For example, What if we want the song? Not just two, but several numbers, much like you calculated us. Here is where we can use flexible arguments. So let's have a look at how we would do it, Ash. So turn all arguments passed to our eggs into a tube. So we're defining our function. Add underscore. All were passing in the parameter star and then the keyword arcs. So what is this function gonna do? It's going to some all values in acts together. So we initialize the variable. Some underscore all, and we set that zero next for Nome in our eggs loop over the Tupelo Arabs and add each element together. How do we do that? Very simply. Some underscore all plus equals known so add. And then we return to variable some. Oh, so let's have a look. Let's call her function. Passing the argument one. Now let's pass into arguments. +1846 on the second argument tree 8404 So we're gonna add all these together. We get that. Now let's see if it works for floats. There we go. In the function definition of both, we use the parameter star arcs to turn all the arguments passed to the function call into a tube called our eggs in the function body. As you can see in the function body, we loop over the tupelo eggs on, add each element to some all. Let me just take out a bit of duplication there at each element. We then returned variable some all on already the caller function. We now pass any number of floater, indigenes, toe, add on the score all and have them summed up so very much like a calculator. If a calculator only had one button, which was the addition button. We've just seen how the pass and arbitrary number floats are into a function. How about an arbitrary number of keywords to do this we use a double star followed by K, W A or G s. Quirks. Choirs are arguments proceeded by identifies like this. So in this line of would be code we have print underscore all employees equals Tony stoned and so employee named position equals piping developer. So arguments proceed by identifiers. What we're doing in this line of code is calling a function called print all passing to it . Identify us on parameters. Let's right the function print all on To begin, we use the parameter choir eggs preceded by a double star. This turns you identify key word pairs into a dictionary with in the function body. Then in the function body, we simply need to print all the key value pairs stored in dictionary quarks. It's important to point out that is not the name our eggs and quarks that important when using flexible arguments, but rather that they are preceded by a single and double star. So here's our function. Print. Oh, pausing at the parameter choir eggs proceeded by double stars. As you can see here, we're printing out our employees report now for the key value inquired start items, print key plus value and the employees report on How do we call our function? Print on the score All while we simply print all pass in our arguments that shift in entering that There we go. Begin employees report and employees report began on end. Okay, let's look at a couple more in depth examples In an exercise above, we created a function that compiles the different types of work department in a sample. Hate your data Set download from IBM. The output of dysfunction was a dictionary that had the department has the keys on the counter staff in that department as the value in this exercise, we will generalize the department analysis that we did in the previous lesson. To do this, we will include a default argument that takes a column name. Okay, so you seen all of this code pretty much in the last lesson here, importing pandas and creating a data frame. Now we're creating our function count on the score. Entries onto their function were passing in the arguments. The data frame, which we created just here on cull underscore name equals department. So the apartment that we want to use within this example Rudin initializing an empty dictionary. We're extracting the column that we're interested in from the data frame and we're iterating over the department column in the data frame on again were combining accounts to get a total count at the very end of a program next here. A lot. The last few lines were creating two variables. Result one result to and to them we're assigning from her data frame. The column name department on the column named Job Role said accounts would in each of those columns, that's printout of results. Let's see what we get Gift at a moment to run. There we go, two counts for each one so within our department's, we've seen this in a previous lesson. In the previous example, we have two total counts. But now what we're doing in our job role were able to add in because we've extended our function, were able to add an additional columns. So sales executive, we have 326 salespeople. What else we got down here? We've 80 research directors. Sounds like a lot on a human resource will be 52 so on and so on to route the example. So, as you can see by modifying a parameter what in our function here, we have department, but we can also add in extra ones here. Job role. Because, as we know, if we specify a new argument in a parameter, we can implement that you're now going to generalize dysfunction one step foreigner by allowing users to pass it a flexible argument, which is in this case, as melon color, as many column names as a user would like. Okay, so as you can see, we've modified the parameters. We didn't account on the score. Entries function to include star arcs. What does that mean? What? We're not passing in a column name at this stage, as we did in the previous example. Instead, for down here we're passing in the column names result count underscore. Entries are data frame on the columns that we're interested in. So we're interested in department on education field so much more flexible, arbitrary arguments or random arguments while not necessarily random, because you're gonna want these arguments from your output off your data analysis so that shift and enter in that and there we go. Our output includes that apartment, but this time we have education field the technical degrees 132. Perfect. Okay, that's it for this lesson. It was an awful lot to cover in this lesson. And I encourage you greatly to download the source file for this code, which is located in my get home account. On practice, practice, practice on Always. If you get stuck, feel free to ask me your any other students in this course a question on We'll do our best to get back to you. Thanks for listening, and I'll see you in the next lesson. 8. Lambda functions & error-handling: Hi, everybody, and welcome this lesson where we're gonna be looking at the wonderful world of Lambda functions and error handling. So let's get started. In previous lessons, you've probably noticed that functions can become big and complex very quickly. There is a quicker, short way to write functions called Landy functioned, so called because to create them, you use the keyword lander. Let's look at a few examples in the code below were writing a previous function as a land the function after the keyword lander. We specify the names of the arguments DeNicola followed by the expression that we want the function to return. So here we go here we create a new variable raise underscore to underscore power equals. So we have our keyword Lambda number specifying the names of the arguments. So two arguments x on why separated by a comma. Then we insert a colon, followed by the expression that we want our function to return. So in this example, we want our expression to return X to the power of why. Then we call our Lambda Function by passing in the argument to entry that shift in entering that There we go eight. That's perfect because I'm the functions can be written so quickly they can tend to be a bit dirty. In the example above, there is no doc string. So Lambda functions and not for use all the time. Don't go off and close this lesson and replace all your functions with Landers. Some function definitions are simple enough so they could be converted into a land. The function and this means using class code. Let's take a look at Notre example from a previous lesson and convert it to a lambda function. Here we're creating a new function. Echo, Underscore. Word on we have lambda word one echo colon word one times echo. So exactly what we want this function to do You want to multiply word one by the echo value that we insert So here we are here So we're inserting Hey, the string Hey, Times five has print out a result Hey, hey, hey, hey, hey, There we go. In the real world land, the functions are best used when you want simple functionalities to be anonymously embedded within large expressions. What that means is that the functionality is not stored in the environment. Unlike a function defined with D E F. Toe. Understand this idea better. You will use a lambda function in the context of map function. Map applies a function over an object such as a list. Here, you can use Lambda Functions to defined a function that map will use the process. The object. Let's take a look at what we mean in this example. The first thing we're doing is creating a list of employees, so employees equals. And then we have our list with our names were going to use the map function toe. Apply a lambda over employees. So let's create a new very well called employees on the score bonus onto that we before us used a map function on. But inside the map function, we have our Lambda item colon item Plus, and we're adding the string 5% to employees. Now let's convert our employees bonus to a list, and we simply do that by creating a new variable called Employees Bonus List and then using the function list where employees bonus as the argument. Let's now print out our new variable. No, you go. As you can see, we have Tony plus 5% Jame plus 5% and so one. So all the employees and what bonus targetting the example above? We used the Lambda functions to anonymously embed an operation put in map. We will take a look at this again, and Notre example by using a lambda function would filter the function Filter offers a way to filter out elements from a list that don't satisfy a certain criteria. What exactly do I mean by that? Let's take a look at this example again. We have our employees name list. We're using filter to apply a lambda function over our employees names. How do we do that? Well, for us again, be credit variable, so results equal are filter function. We have lambda so length of our members, any employee whose name has more than four characters again, we convert the result to a list when we printed out Harry. So within their employees, the only person with a name greater than four characters is Harry. A very simple example. But just to show you the power, let's now apply to filter function to our employees. Data set in the land. The function below. We're checking if the Argument X is yes within the column over time. So let's find out if anybody has worked overtime. The Lambda Function Dennis signs the resulting filled the object to result. The code below returns all employees that worked overtime. So the first thing we need to do is import pandas. We then create a data frame using our sample employee data set, which reviews plenty times before. Then let's right the function that will find our employees who have worked overtime. So we're using filter cause up. So we're going to filter out employees who did not work overtime. So we're using Lambda X Colon. Andan X is yes. So any time within the data frame in the column over time where X is yes, so that's what we're filled during. Four were filtering the data data frame column over time, and we're looking for yes. Then we create a list from the filter object results. So Result list equals list results, so we're just simply converting the variable result from above into a list. Then, from that list, we reiterate over and find the overtime. Let's have a look like that ruin for a moment, and there we go. All the employees who worked overtime another useful function is reduced, which can be used for performing computation on the list. However, unlike map and filter, it returns single value as a result. To use reduce, you must forced imported from the funk tilled module. And how do you do that from funk pills? Import Reduce Perfect. Next, That's creator list. There we go. Result equals reduce. So we're calling our function on What are we doing? What we're saying Lambda Item one item to So we're passing in the arguments. And then what do we want to do with those arguments? We want to add them together, and we want to do it on the employees list. Pretty simple again. Like everything with fightin, Lambda is really comes with practice and practice. Then we print out a result. Tony Jane, Mary Harry. Okay, let's look at some error handling in any programming language, not just fightin. It's always go to know when an error occurs within a function. Ideally, era should be thrown when we use a function incorrectly. In previous lessons, we've already seen several errors. Thes errors have been related to Piper's built in functions, such as when we try to pass a string when an integer was expected. Now that we know have the writer on functions, we also need to know how to catch specific errors on right corresponding error messages. So let's see how we would do that. That's first create a simple function. So square root function. What kind of error are we gonna work on, where we're gonna work on the type error? So in the code below, we tried to pass the argument, Tony, to the function sq or T. The function is expecting interject put instead received a string and so on. Error is drone. So here we are, calling a function with strain Tony and we should get an error. There we go. As we can see, it's a type error so unsupportive upper and is not telling us much. Kobe labor clear. So piping tells us that this is a type error. And as I just said, the message could be a little bit better. Errors caught during executioner called exceptions. We can catch exceptions would try, except clause. Piping will run the code following try and if the cold run successfully, that's great. If piping cannot run the code, or if there is an exception. Pipe what run the code following, except let's make this a little clear with example. So here again we're defining a function sq warty, and we're passing it. The parameter X. So dysfunction, according to the doctoring, will return to square root for number. Then try return X to the power of 0.5. If that doesn't work, print exe. Must be manager or afloat. So again, let's run this. And as you can see, we now get amore Informative error. Message X must be an integer or afloat. We can also specify the error that we want to catch on. Allow other errors to pass undetected. So our next code example we've added in type error tour except close Again X must be an editor of Float Within Piping. There are lots of other errors that can be cut, and you can check out the documentation here for more information. More often than not, Instead of printing an error message, you will want to raise an error by using the keyword rays, and we scroll down here and moved us to the top of screen. Okay, so back to our previous example, let's imagine that we do not want our square root function toe work for negative numbers. How would we do that? So here we are here we traded our function. So if X is less than zero so obviously a negative number, we raise the value error. X must not be a negative number. Then we tried to execute a function return X to the power of five. And again, as before, we raise a type error. If X is not an integer that shift on that shift and enter again on there we go in our value error. X must not be a negative number so we could change that toe one. Perfect. Let's no go back to employee data set from previous lessons. What happens? Every user passes in the name of a column that does not exist. Ideally, you would like to inform them of the error. So this code here is something that we used in a previous lesson. And actually, it's just a lesson gun by on inner function. We're passing the parameter column. Underscore name equals apartment. But what happens if down here in the result somebody want to column that is not within our data frame? The data frame does not have an ABC column on the result is known. Let's close offer one more example of raising an error. So as you can see here, in addition to the code above, I've added in the line raise value error. I'll let you work away without when yourself and in your own time. In this lesson, we've looked at Lambda Functions on error handling again, all of which will come easier and better with practice. As always. If you have any questions, please feel free to ask me, Thanks for listening, and I'll see you in the next lecture. 9. Python Iterators: Hi, everybody. And welcome to this lesson where we're gonna be talking about fightin it. Traitors. So we've already seen a lot of it. Aerators used in the four loop. In previous court examples, we use the for loop to it rate over a list and print out all the elements within that list . Four loops can also be used to it right over the characters of a string or a sequence of numbers when used with the range function. The reason we can loop over these objects is because they're called it trebles lists, strings, dictionaries, fire connections on range. Objects are all. If Robles Inp Iten If an object hasn't inter method than this, isn't it terrible? 50 iter mattered is applied to an irritable on it aerator object is created. So you might want to read that line again after I feel finished saying it because it's quite a mouthful on a little bit to take in. This is what a four loop is doing. Whatever is really knowing about it before loop takes he it terrible creates the associate id it aerator object and then it rates over it to be in it aerator an object must have an associated next method. This method would produce the consecutive values. An iterated is created from ineffable using the function. Either after we have the iterated to find, we pass it to the function next which returns to force value calling. The next method again returns the next value and so on and so on, as we have seen in other lessons the concept of over hard to understand without example So let's get started. So as I just mentioned, a string is in a terrible the interval name is passed to enter the it which is the name we're going to call. Our trouble is now what is greater. So what does that mean? Okay, so for us we create our variable name equals Tony, We're now creating our either using the method, so I t. And again, that could be any names. So I t equals either. So we're using the tormented and were passing it the variable name were then calling the next method on passing at the accurate argument I t So we hit shift enter, we get t. So we're starting the iterating process of moving through the variable name next. Oh, and white. There we go. Okay, so you get the idea. Once something is created as inevitable and passed into the next method you can, then it right over it. What happens if I press next again? When we've gone through all the characters in the string, we got an error. Sounds a lot more complicated is but what you see in an example, it's not too bad. If we wanted, we could also print out all values. Often it aerator and won't go with a star operator or, as it's also called the splatter operator. So again, same code is above. But instead, now, in a print statement, we have print splatter operator I t. There we go, Tony, So type it. So what is the type of it? Hope? Word? Hopefully it's an integrator, and it is. It's a string it aerator unpacking, and it aerator in this way can be done only once is after it is complete. There are no more values to it, Ray. True as we have seen, we would have to redefine our traitor to do so. At the beginning of this lesson, I mentioned that dictionaries are also trebles to reiterate over the key value pairs of a dictionary. We first need to unpack them by applying the items method. So here we go here, in the next code example, we're creating our dictionary. We have Tony developer Jane Finance. So four key value in name that items print the key value pairs. There we go. Simple enough. I mean, a scroll down moved us to the top of screen. Let's now use the Internet ID on the next matter to it. Rate over file. So here I have data sets. Enter that txt. So again, that's in the resource is file available and get hope file equals open. So I've created a new file. Variable on. There we go. You can see we've iterated true to content of that file with only has two lines in it now, as mentioned list are also it trebles. So now we're creating an employee list. As you can see here, employee equals and we have our list. Next, let's print each item in the list using a for Loop four person, an employee print person, our next line of code create an iterated for employees called My Underscore employees. And we do that again by using the term it'd on passing it. Our list employees you can see here, I've added in a print statement, which just separates the two outputs and then finally print each item in the it aerator on . We do that as mentioned with the next method. Let's run this and there we go, printing it out as a dictionary on printing it out as an iterated. In a previous lesson, we had a brief look at Peyton's range function. Range doesn't actually create a list. Instead, it creates a range object with an iterated that produces the values until it reaches the limit. Let's take a look at an example here we're using the range function would in the hermetic were signing that to the variable value. Next, we're printing out the range would in our variable. Finally, we loop over range and print values. There we go. 012012 so you can see you're getting the same output with both methods. So the examples above we've been using the teeter function to get an iterated object as well as the next function to retrieve the values one by one from the iterated object, there are also functions that take it. Aerators arguments, for example, list and some functions return a list and a sum of elements. So what we mean by that? Well, let's as always, take a look at an example here. The first thing we're doing is we're creating a range object called values were giving that range 0 to 38. Then we're printing out the range object. So print values next we're creating and list of integers. So values underscore lest equals list values. So we're taking the variable that we created just two lines ago. Values were inserting that into the list function. Next, we're printing out values underscore list. So we're printing out the list we just create in this line of code here. Values underscore. Some equals some values. So assuming the values within the list finally print out the variable value, some let's have a look. Okay, so print the range you can see. We got that their number printing our list 0 to 37 then finally were pending out to some of her list. In the example above, we just passed a nit aerator from range to a function and then printed the range it's list , and it's, um, not to battle. Next. Let's take a look of numerator, which is another built in piping function. It takes a miserable for any argument, enumerate function returns and the numerator object, which consists of pairs that contained the elements of the original miserable as well as the index would in that terrible again, as always. Let's look at an example to help us here in this example. The first thing we're doing is we're creating a list of employees. Then we're using the enumerated function to numerator over that list. So, as always, we create a variable E in this case on reciting to E. R Employees List, which is in the numerator function, then print of the type of e. So hopefully this says enumerate, which it is. Now let's use a list to convert or enumerate. So enumerated. Underscore Lester E n Underscore List equals list E. So we're converting enumerated list that now print out that list. There we go. Our list with an index zero Tony worn to tree and so one you numerator object is also unutterable on. We can loop over a shown below, so we start off for a list of employees again and then four index comic value in the Numerator Employees Print Index and the value. So again, we're in numerator over a list by default. Enumerate begins indexing at zero, which is fine, but it does look a bit strange when printed out. We can alter this with a second argument start. So as you can see here in a for loop, we have employees as one argument. A second argument start equals one. So we're telling enumerated, Start indexing at one. There we go. 12 tree for looks, liver better. Let's now discuss another of Piper's built in functions. Does it function? Zip accepts an R between number of intervals on returns, and it's aerator of two boats. As always, Let's take a look. An example below. Two lists are familiar list of employees on the second, which is their assigned roles. If we zip them together, we create a zip object, which is an it aerator of two bulls. So we have two lists employees on roles. How do we sit together? What we passed them to the zip function, both as arguments. So said Eagles IP employees and roll. Now let's print out the type of said, and as you can see, it's class zip. Now we can convert this it aerator of two balls into a list imprint of the list on how we do that. That's created new variable called Zipped Underscore List equals. And again we're calling the list function on Passing It Dead, which is the zip off boat list together from the example above. Now let's print out at Variable Tony Developer Jane Finance, Mark Design, Alice Marketing. The first element is a two, but which contains the first element of each list. That was it. The same goes for a second and so on and so on. So as you can see, what that means is a Tupelo, as we learned in the last lesson is created in brackets. So we have Tony and developer here. Surround by brackets on Tony is from our list of employees on Developer is from our list of roles, so we have the two lists zipped or combined together. If we wanted, we could use a for loop to it right over the zip, object and print after two bulls similar to the example above again we have are two lists. Now we have a four loop for, said one Is that, too, and ZIP employees and roll. So we're passing the to list. As arguments print said one and said to There we go libre cleaner. We could also views the splatter operator to print out all the elements. The only difference. Here we have our two lists. Zad equals it. Employees and roll print splats. Zed. There you go a couple different ways to achieve the same end results. Now in your life is a data scientist. You're going to have to deal with files that contained large amounts of data on a regular basis. But have you thought of how your computer will be able to hold a large file of memory? There will be times when you were working on a data from a database or ganging on a P I that there will be too much data for you to hold in memory. One way to handle this scenario would be to load the data in chunks, perform whatever operation is needed and then removed the chunk from memory and begin the process. The next junk. Does any of this sound familiar? Well, I really hope that those because to me it sounds like an iterated er on. We've just learned how to do that. The pandas read. See SV function provides us with the ability to load in on read large data sets by Chunk. Appropriately, this argument is named chunk size. Once more, Dear friends, our employee data set. Let's start by computing the numbers of hours of overtime worked. Imagine that the file is too big to store in memory, so we're gonna do it by chunk size. So first thing we're doing importing Pandit's PD Next, we're creating an empty list of store each iteration. Now let's use the chunk size argument and read Underscore CSP. So four chunk in read underscore CSP. Then we pass in the location of our data set. Chunk size equals 500 that's 500 rose off the data set. Results depend some chunk standard hours. So what does this mean? Some the numbers would understand it. Hours column beat in the 500 damn row chunks on appended to the result total equals the sum of the result Print total give down a moment. There we go. Overtime hours worked 117,000. So just again we're iterating over our data set here, and we're doing it every 500 rose, when we have our hours are standard hours counted up where? Something them together on a pending them onto the result. So every time we loop through the data set we saw in the hours on a panda to the result So we're combining each loop true to file the object created by read on the score. See, SV is an intra boat which can be iterated over using a for loop on each chunk. Created is a data frame. As you can see in the four loop, each iteration computes the some of the column standard hours. The result is the independent analyst created area. We don't print out the result, which is the number of hours of overtime worked by employees. Here's something very similar this time We're initializing an empty dictionary on We're iterating over the file chunk by chunk, this time in chunk size of 10. Then we're iterating over the gender column. Finally, we're printing out the counts on the score dictionary Female 588 male 882. So the gender breakdown. Other employees. It's good to know how to process a file in smaller, more manageable chunks, but it can become very tedious having to write and rewrite the same code for the same task each time in the exercise below, you'll be making your code more reusable by putting your work in the last exercise in a function definition. So let's have a look at that. So here we are. Now we're wrapping the cold from the last example in the function definition called Count Underscore Siri's so are passing in tree parameters. The C S V file the column size on the column name all this code next is all the same here We're returning accounts dictionary and then we're calling it, so we're giving it the result. The name results underscore counts count entries for department next line accounting entries in the business travel. So let's print out the results. There we go total and sales total in business travel. So how many of her staff travel? Some rarely Some frequently and some don't travel at all. Okay, thanks for listening. And in this lesson, we explored Pipkins It aerators on once again. This class is very technical and very behind the scenes. Look at how piping works. So I don't expect you to just watch these lessons and know it off by heart. It takes a lot of practice every day on as always, please do free free to ask me questions in the community section. Thanks for listening, and I'll see you in the next lesson. 10. List comprehensions & generators: Hi, everybody. And welcome to this lesson where we're looking at Peyton's list comprehension now. In previous lessons, we've used a lot of four loops, but have you noticed that they can be very inefficient when working with lists? There is another way to work with lists that's more efficient and takes only one line of code. And that's done by using what's called list comprehension. The syntax use of square brackets where you place the values you wish to create. Also called the output Expression. This is followed by a four class that refers to the original list. As always, Let's take a look at an example. So here we're creating a list of numbers 1 to 6. Now we're going to create our list comprehension called new numbers, So new numbers equals square brackets gnome plus one for Nome in numbers. So we're iterating over our elements in the list numbers. Now let's print out the variable. 123456 Perfect. See how much more efficient that is when compared to a four loop. Let's take a look at Notre example again. We have numbers 1 to 6, then we're initializing an empty list for Nome, and numbers upend each number +12 new numbers list Print that out to tree +4567 Excellent. This comprehension can be used over any interval. Here's an example using the range object to this time known for Nome in Range 11. So let's print that out. 0 to 10. Excellent. Now, in a previous lesson, we looked up nested for loops. This comprehension can be used to replace these as well. So here we're initializing a new list, recalling that Combs CEO MBS short for combination. So four x in 12 tree nest it then for why in tree 14 So if X is not on, why upend X and y that's print out dust And there we go. We got a set of tipos for when X is not in. Why now? Do you think that the same could be done with this comprehension? But in the square brackets place the desired output expression followed by the two required for Luke clauses. Let's have a look. Combines two equals. So here we have our X and Y four X in. And then what? The values of X are on four. Why in what? The values of wire when X is not equal to, Why print out the result And there we go. We get the same result. Now, if you're struggling to read this comprehension, then you are not alone. Andy. Both cold examples readability has been sacrificed on. That is a common problem with this comprehension. Liz Comprehension are something that only get easier practice in cases like above, it will be your decision whether or not to use this comprehension. We can also use condition ALS incomprehension. So in the example below, we've squared the values in range 10 and output at the results on Lee under conditions that the result is even. We do this by using the modular operator, which is the percentage symbol which outputs the remainder of the division. In the case below, we're dividing the squared values in range 10 by two. And if the remainder zero meaning that is uneven number. The result is out. Put it to a list and how do we do that? So power are gnomes by two for Norman range 10 on If the resulting division by two of Nome is zero, we add them. There we go. Zero for 16 36 64. We can also use conditional operators within the list comprehension. Andi output expression in the example below for only and even integer we output. It's square in another case signified but he else condition for odd integers. We output zero So as you can see in the output where we have a zero, that meant that we had an odd number using dictionary com pretensions, we can create new dictionaries. The syntax is similar to list comprehension. There are, however, two main differences. Firstly, the dictionary is formed or curly braces, and secondly, the key and values are separated by the colon in the output expression. Here's the syntax for that numb for Nome in range 10 printed out on There we go, our key value pairs. In this example we have created dictionary were keys of positive integers and corresponding values with respect of negative integers, An interesting mechanism and Liz comprehension is that you can also create lists with values that meet on a certain condition. One way of doing this is by using conditional is on its radar variables. So in this example, we've created a salaries list. We then creator list comprehension called large salaries, and that large salary is only gonna print out if our salaries are greater than or equal to 45,000. And there we go. Now let's take a look at generators. Generators are related comprehension, which is why we're talking about them here on No, take a look below at the line of code, which is a list comprehension. So nothing new here in this list comprehension. But what we're doing in this line of code is we've wrapped the line of code within round brackets instead of square brackets, so straight away you'll notice the difference of boat outputs. A list comprehension produces a list as output. A generator produces a generator object. So what is a generator? It's similar to a list comprehension, except for the fact that it does not store the list in memory, which is crucial. But it is an object that we can iterating over to produce elements of a list as required. And how do we do that when this piece of code were looping over a generator expression producing the elements of a list and there you go output, but in the range 10 0 to 9, like any other, is greater. We can pass a generator through the function next so that we can it rate over its elements . There we go. Range 012 You can put in print next result print. Next result and so on. I have to stop the two. This is what's known as lazy evaluation, meaning that the evaluation of the expression is delayed until its value is needed. This is good. I'm working with Larry sequences, and you do not want to store the entire list in memory, which is what lists do. Instead, you can access elements as and when needed. It's worth pointing out. Anything that can be done with this comprehension can also be don't with generators as an example. There we go print a list of even gnomes from a generator expression. Let's take a quick look at generator functions. We've already seen that generators at our own type, so generator functions are functions that went called produce generator objects. They are created with same syntax use when creating an iota function. But instead of returning values using the return key word, the yield sequences of values using the EOE keyword. So here we go, let's take a look So we create our function gnome underscore sequence We pass it the parameter end generate values zero to end So I equals zero So we're initializing a new variable there I setting at zero. So while I is less than end so whatever value we pass in yield I and add one to it each time. Now let's call the function result equals known Underscore sequence 10 So we're passing the argument 10 into the function that's printed out. There we go. 129 Output here stops when I is less than end. So in this example we've passed the argument 10 in which replaces end on when I is less than 10. It stops so it stops at nine. Pretty good stuff. OK, that's it. Thanks for listening. As always, if you have any questions, feel free to reach out and ask, and I'll see you in the next lesson. 11. Practice Lesson: Hi, everybody, and welcome to this lesson. Now this lesson is a little bit different from the ones that have become before because this is a practice lesson on the reason I've included this in the classes because the last few lessons have been very heavy. Very knowledge base. They're going to require a lot of practice. I think this lesson will help to solidify what you've learned so far. So in this lesson, we're going to challenge yourselves to bring everything together that we have learned to this point and apply it to the world of data science. We're going to be working with sample employee data set that we've seen in previous lessons . Whoever this time, we're going to dive much deeper. The first part of this practice lesson is to create a zip object by calling the zip function I'm passing to. It had our names and roll data, as you can see in the code example below, I have gone ahead and generate a list of heather names and row data, both taken from the sample employee data set. Next, we don't create a dictionary from the zipped lists, object by calling dicked on passing it zipped lists. The resulting dictionary is assigned to the Variable Employees Dictionaries. Now you can see the sample code here, and I'm not going to go through the whole thing because this is a practice lesson to let you go away and practice on your own time on at your own pace. You can see here removed down through it. I have all the cold here. I just haven't done the output so you can take it away and study it at your own pace. It's a pretty big file, but it's really good and really helpful, I think, to help solidify what you've just learned, as everything that we've done so far has been pretty deep level piped knowledge. Now there won't be anything in this practice lesson that you haven't done. It's just that I have brought everything together in one Joubert notebook file to help you with your learning. So go ahead and give the practice lesson to run true. And as always, if you have any questions, please don't hesitate to give me showed in the community section. Thanks for listening, and I look forward to seeing you in the next class