Emotion & Sentiment Analysis with/without NLTK using Python | Attreya Bhatt | Skillshare

Emotion & Sentiment Analysis with/without NLTK using Python

Attreya Bhatt, Developer

Emotion & Sentiment Analysis with/without NLTK using Python

Attreya Bhatt, Developer

Play Speed
  • 0.5x
  • 1x (Normal)
  • 1.25x
  • 1.5x
  • 2x
10 Lessons (1h 12m)
    • 1. Introduction to Emotion Analysis (NLP)

      2:42
    • 2. Creating Project and Installing Python

      2:10
    • 3. Cleaning Text for Natural Language Processing

      8:13
    • 4. Tokenization and Stop Words (NLP)

      7:47
    • 5. Algorithm for Emotion and Text Analysis

      11:19
    • 6. Emotion analysis

      6:35
    • 7. Emotions in a Graph using Matplotlib

      6:51
    • 8. Twitter Emotion Analysis using GetOldTweets3

      13:38
    • 9. Installing NLTK | Tokenization and Stop words

      6:41
    • 10. Positive or Negative Sentiments | NLTK

      6:19
  • --
  • Beginner level
  • Intermediate level
  • Advanced level
  • All levels
  • Beg/Int level
  • Int/Adv level

Community Generated

The level is determined by a majority opinion of students who have reviewed this class. The teacher's recommendation is shown until at least 5 student responses are collected.

34

Students

--

Projects

About This Class

Welcome to this course on Sentiment and Emotion/Mood analysis using Python

Have you ever thought about how Politicians use Sentiment Analysis? They use to find which topics to talk about in public.A topic can have different sentiments (positive or negative) and varying emotions associated with it. Politicians analyze tweets/internet content to find out these topics and use them to find holes in the opposition.

How Google Maps classifies millions of locations like Restaurants by analyzing the Reviews

How Amazon shows products which evoke Positive Sentiments/Emotions for the buyers

How KFC use it to do Market Research and Competitor Analysis

If you want to know Technology running behind, this is the Sentiment Analysis/Mood Analysis course which is going to use Natural Language Processing ( NLP ) and Text Mining to analyze different moods in a text ( example - Sadness, Excitement, Loneliness etc)

51f7d499

Meet Your Teacher

Teacher Profile Image

Attreya Bhatt

Developer

Teacher

When I was a kid I saw this YouTube video on how to make a folder invisible on Windows. I have never looked back since then. My love for technology has only grown.

I started with security since that was one of the areas that fascinated me. Then i went on to win the award for designing using Photoshop at Cofas '2012. On the destructive side, I always made scripts that used to mess up the systems at my school. I was almost suspended. I learned my lesson and vowed to do only constructive things. To make people aware of security issues, along with 2 friends started a Facebook page and group called YAP

To make things more interesting Web development came into my life which helped me get into the most prestigious chapter at my college IEEE. This helped me master different things... See full profile

Class Ratings

Expectations Met?
  • Exceeded!
    0%
  • Yes
    0%
  • Somewhat
    0%
  • Not really
    0%
Reviews Archive

In October 2018, we updated our review system to improve the way we collect feedback. Below are the reviews written before that update.

Your creative journey starts here.

  • Unlimited access to every class
  • Supportive online creative community
  • Learn offline with Skillshare’s app

Why Join Skillshare?

Take award-winning Skillshare Original Classes

Each class has short lessons, hands-on projects

Your membership supports Skillshare teachers

Learn From Anywhere

Take classes on the go with the Skillshare app. Stream or download to watch on the plane, the subway, or wherever you learn best.

phone

Transcripts

1. Introduction to Emotion Analysis (NLP): Oh, right, you're beautiful. Bite on developers. Welcome to this new media Siris in which will be using national language processing, which is the art off, extracting information from text on it's also known as nlb. In Shot on, we'll be using this national language processing to analyse emotions and sentiments are forgiven text or an essay. So if it's a side of saving, people tell whether the emotion present and that they say it's sad or not, or whatever different emotions are in that text on. After completing this video, Siri's you'll be able to analyze different emotions present in an essay or text like sadness, happiness, jealousy on other emotions on, you'll also be able to find out the dominant emotion in the text. For example, if a text is about how you met your long lost brother, then it's a happy emotion and you'll be able to tell that the dominant emotion, obviously, there'll be other emotions inside. That s a or text, but the dominant emotion will be happiness. So you'll also be able to find out this emotion. And then thirdly, you'll be able to plot these emotions on a draft so particular essay or they bake amount off test contains a lot of emotions on. You'll be able to plot all of these emotions on a graph according to how much these emotions are occurring. And, fourthly, you'll be able to tell that the whole text is a positive or a negative emotion for text contains too much of sadness. Mostly, it's a negative emotion on. If it contains a lot of good words, then it's a happy emotion or a positive sentiment. So this positive or negative thing is known as the sentiment and emotions are like sad sadness, happiness and jealousy. Except lastly, you'll also be able to scrap tweets with the HASHTAG and find out the public opinion on that hashtag, for example, you can search for hashtag Donald Trump and find out whether that emotion is associated with the positive or negative sentiment. It doesn't matter, like whether it's positive or negated, but you'll be able to find out on given any hashtag, for example, you can even so sure hashtag New York and see whether the people are talking about positive things about New York or negative things about New York. So firstly, we are going to be doing all the natural language processing and sentiment analysis on our own without using any off the external libraries or packages so that you guys properly understand the concepts off NLP on. Then we can go on to use the N. L. P K library to shorten our works. If you don't know what analytical every basically are very famous library for natural language processing which political in the future videos. So from the next video, we'll started installing python and buy some. If you already have it in star, you can just going to the next video and you don't need to use by time. You can use a little off your own choice, but I used my charm and if you want a fall along, you can also install the same. But any of these guys are seeing in the next video where we will start our natural language processing journey. So I see you, what there 2. Creating Project and Installing Python : All right, guys will come back in this video. We're going to learn how to install pie tone and by charm, which is the i d e for writing code into our computer. If you have already installed by tone and you only have a place to write godly, sublime tex or any other idea that you like to use Phil Fee to go with that If you don't know what ideas idea is basically a place where you write code and by chance somewhat looks like this. Now I prefer by charm, but you can go with any other i e. So let's get started. So first thing you need to do is installed by taunt. I'm just gonna not in start fight on 3.8, because I already have by 20.7 and stored on my computer. And you can also do that if you follow along. But feel free to download any version of bite on. That's about three. You can even download the leaders abortion and then just go to the by charm website Over there. You just need to go to Jetbrains Starcom Pie Charms and then download on download the community wash in the community. Washington's three, obviously al recommend downloading a professional version which had a 30 day trial. But if you don't want to get in and download the community version, which works pretty well for our needs, So after he had downloaded it, it was somewhat look like this on after you have opened it. Make sure you create a new project and you can call your project whatever you want. I've Corlett sentiment analysis so that you guys can understand what this project means on if you wanna fall along, you can also name your project the same. After that, I'm just gonna create a new file over here via gonna right all of a court. So there's gonna call this mean by 25 on inside this mean fight on file. We're gonna right. Let's don't focus on this. This is basically might get hub repository. So if this partisan pop up, don't worry about it. I'm just gonna cancel it and cross it over here. And I was gonna create this means or by foul altered organ confused with the straight color and green color. Whatever happens over here, let's focus on the court that I'm gonna write on. Just follow along the inside this mean dark by file. We're gonna be writing all of our court from the next video on there. Pretty much ended with a pretty sharp video. Just toe. Make sure that you guys have everything said I probably doesn't have a lot of requirements this natural language processing and sentiment analysis, because we're going to be writing our code from scratch on. That's pretty much it guys and see you in the next video. 3. Cleaning Text for Natural Language Processing: All right, guys, welcome back. And this week, we are going to learn how to clean the text before we can apply our own natural language processing concepts on it. And now cleaning is done in two main ways. First and making sure everything is in lower case and secondly, remove all the unwanted characters from it like fluctuations. But even before that, we need to read text in our fight on program. So for that, we're going to create a new file over here. I'm gonna call it dot next, and this is going to contain all of our text. Even if it's an essay or anything copied from the web, I'm just gonna give it the text off. I love Bip on with a capital B and an exclamation monk so that we can tackle boat off the second and third steps, which contains a capital P. So it's and not a low case, and it contains a punctuation. But even before that, we need to make sure that this data inside read or text gets inside this means or by problem. So we're gonna create a variable cord X pretty easy, and then I'm gonna write open which will open the street door text file on. We need to write down the name of the file in our cases retort, text, and then I'm just gonna tell fight on reader on one other thing I'm gonna add is something known as And according I'm gonna give it an including off beauty of cash it. So what is this? According for sometimes we need to copy and paste a blogger article or something from the Web inside our door text five so that we can analyze that data until what kind of sentiment that block post or a Web article has. So when you copy that inside this read or expired that data that text is in utf dash including, like most of the things on the Internet, off that according not everything, but most of them are especially blotted blogged articles and stuff like that. That's why you to make sure that we added according other White is gonna show us on air is just a way off writing text on the Internet. So anyways, after we have done that, we're gonna print out the steak, so make sure that our main door pie file has their text inside, read text on it has been printed on the screen. Now we can go to the next step that is converting the letters into your case. Now we need to convert it to lower case because the words are kind of the soul of analyzing text, not sentences, just words. Because they can convey a lot of meaning by themselves on when we compare words in national and word processing or what, like an apple with a capital A is not equal to the same word in a smaller case, for example, on Apple with a small eight. Therefore, to compare words, we need to make sure the entire text which we're going to be analyzing is in case this would make more sense as we go for the long the videos. But converting the Texas lower case is really easy in fight on. So for that, Piers is going to create a new variable, and we're gonna call it lower case. And inside this, we're gonna right next door. Lower on that is pretty much it. The text will get can water into lower case, and then we're just gonna print out the low case to test whether this is working or not. So currently, a read or text has a capital he that you want to convert into what kids. So that's does this mean dark by file and see if our program works. So, as you can see, I has also been converted into smaller case for what? That is also uppercase P is also now in small case. Now we can compare these boards probably and use them for a natural language processing. Now we're gonna go to the third point, that step that is, removing fluctuations like full stop korma, exclamation mark reader textile. There's an exclamation park and let's also Blake, add some stuff over here. Let's let's just right. Let's I at the raid love. Andi just very doesn't act some fluctuations in this case. Also, there's an exclamation mark, so we're gonna remove all these fluctuations from the text using something known as dot translate. So for that, we need to import something known as drink. So what we're gonna do is we're just gonna right over here import drink, and then we're gonna use the function that are inside the string class to make sure that we can remove the punctuation. But even before that, what kind of punctuation are present inside pipe on that we can use to remove it from a strength? The reason. A print out something known as string dark punctuation so that you guys can see what are the punctuation that are present. So we has no right strength if you just imported and then we're going to write dot punctuation on, let's actually printed out. We're gonna remove this from here, undergoing around the main door by program again, and you'll be able to see all the punctuation that we need to remove we if you have a strength that is in no case, and we need to remove all these fluctuations from it. That is where dark translate comes in. So I'm going to write the court, and then I'm going to explain what it means. So we're just gonna remove this print line and instead of that original, right, a new variable and let us call it clean underscore text, and it is the final step that we need to do so this cleaner the score taste will contain the final clean text that is in no case and three or four fluctuations after that year's men. Right, Lou underscore case, because this is a lower case string that didn't want to remove the punctuation is trump on after that visible right dot translated. And this is where the magic happens on lower case string gets converted into a string with our fluctuations. If you're confused about what this dark translate means, you can just think offered as taught convert we're converting are lower case string into a string that doesn't contain all these punctuation on by spring I just mean next if you guys are not sorry programmers. But Embry show everybody already knows what a string is Anyways, After that, I was gonna write str dark meat trance on. This is a function that is inside dot translate, which helps it to convert the lower case into a string with our fluctuations and this requires three para meters for a string which is empty in our case. Second string on This is also something in our case on the clustering which requires on the punctuation. So I'm just gonna write strength on this is getting we're getting it from import strength. Really is gonna write string dark fluctuations because This contains all the punctuation that you want to remove, as you can see all of these populations below. And then I'm just gonna print this out. And then I'm gonna explain what this line means against so that you guys can understand it properly. Someone's gonna write, print on them clean text, and let's see if the magic has happened. I'm just gonna print this out on the screen and you can see I'll buy. Pound is in low keys on doesn't contain the ad. Great sign on the exclamation mark. So our text, our conversion is walking not just to give you kind off a recap. What we did was we create a new, edible Cortines underscore detective, which contained all off our clean text. And then we defined a string that's need to be converted and then leaves the door translate method to convert the string into a string that doesn't contain punctuation on the function that helps it to convert his str dot make trance, which requires three para meters. The first thing that the second string, which we don't need in our case and the first thing is no one asking your punctuation, which will contain all of our punctuation that need to be removed on. If you want to really go into depth off all aboard these conscience means obviously you can google it, but I'm just gonna give you a little bit of an idea. So this first para meter contains the characters that need to be replaced. In our case, there just characters that need to be replaced. We just need to do them. That is why we needed str three or the third string on the singing strings with Spite the list of characters with which the characters need to be pleased. For example, if our first string let's write a ski armed one if our str one is, let's say ABC on our str Do is something like That's right g e f. So this str one will be this specified a list off cattle that need to be replaced so ABC will get replaced with G E F if he had put in the first drink on the second string as the para meters over here. But obviously we do need both of them. They just need toe believe the characters and that is why we are using only the card a meter that is the strength on this main taught str dot make trance basically creates a transition table on a table. That healthy draw translated function to believe these characters. It's just kind of help. Obviously, if you go more into depth, it's not needed for our purposes. But if you need to go into more of the step, feel free to Google it and these guys, this is pretty much it for this video. In the next video, we're gonna learn about organization and stop words, which are really important natural language processing. So I see you. What did? 4. Tokenization and Stop Words (NLP): All right, guys. Welcome back. The last video we learned how to clean the decks properly on heart removed fluctuations and how to make sure that the text is in lower case. So, for example, if a takes a zai ad rate love bite on with an exclamation mark, it will print out. I love fight on in lower case, and it will remove all the punctuation from it, such as, at the rate sign an exclamation mark. This we do, we're gonna learn about the organization and what are stoppers and how to remove them from our text. So let's just focus on innovation first. What organization is just a fancy? What way off? Saying that how to break up the sentence into words. So, for example, right now, clean tech says, I love fight on. If we bring the sentence into small words, it would say I love on Pike on. So they're three was in this. And if he store these three words in a list then basically known as organization, very simple is just like a fancy word that is used in national language processing. But don't worry about it. So what we're gonna do is we're just gonna, like, just create a readability. Let's just call it organized under school. What? And after that, we're gonna write clean text, underscore text, and then you're just going to use the split function that is inside by count to make sure that we get the words separately and they're sort as a list inside the stock nine sports. So now we're just gonna print this out to make sure that it's working. So let's actually just base this doctor Knight's words over here and run our main dot by program again, and you'll be able to see that the words are in a list separately for this process are breaking a sentence. The sentence that has been cleaned, the text that has been cleaned in tow, different worlds into three words in our case is known as took in addition. So this read or Texas right now bring small but imagine there is like s a war here, that country. It's more like 500 or 1000 words, then this technique is makes more sense. Another reason why I took my vision is important is because entropy or natural language processing, is the analysis off worlds and not sentences on. That is why we need to break the sentences in tow. Worse that we can analyze later on. Now that we're done with stockings, words, let's we're gonna stop. But so what? Exactly asked. But so let me just write it down. Write down a new wearable stop underscore words so that you know how it's pronounced. It basically stopped boards, so stop words are the worst that don't add any meaning to the sentence. For example, in our case, I doesn't add an emotion meaning emotional meaning for any kind of meaning to our sentence in terms off natural language processing. So this was that Don't add any meaning to the sentence can be removed on these words are basically known as Stop. What's So if you're using lt gave it is a package off by tonic, useful national language processing. It already contains a lot of Stockport, but because right now we're doing it manually. You need to copy a list off stoppers that we can remove from stock. Nice words. So what I want you to do is go to get up dot com How three about this is my name after about it. I know it sounds a little bit complicated, but just copy displaying and then sentiment analysis, and it will take you to this page. Just click on main door by file, and then you can just copy the stop words list from here on. This court also contains this whole page also contains the whole code that we are going to be creating. So this is the get troubling if you want a good but in race for now, just copy all of these lists from here. I'm also going to be adding them to the description of this video so you can just copy that from there. So this copy it forward now from here on, just based it over here. And just look on the list of how here and you realize there are a lot of words that don't add meaning to our sentence. For example, the first word is I, and as a tool, louai in our Kays doesn't add any meaning to our code or toe. Are sentence in terms of analyzing it. Now that we have stopped was we need to remove them from the Dark Knight's words on the way to do it is using a simple for loop. So what we're gonna do is I'm just gonna minimize it for now. I'm gonna add a for loop over here on first. Even before the for loop. Let's just create an empty list on Just just call it Final words. Because these are the final was that we are going to be analyzing on. We're going to initialize it as an empty list. Then we're going to create a for loop that is going to look through all the talk nice boards that we have right now. We just have tea words between you can do it manually, but because we might increase the length of the centers and might be an essay off 1000 words. Even so, that's why we're gonna use a loop. So I'm just gonna write for on then Woods interconnects words because we want to look through the Dark Knight's words one by one and temporally stole them in this variable off. What's and actually instead of words, Legis College would because it's gonna be taking individual words one by one, and that inside their spirit is not scroll down, and I'm gonna put a condition that if I took a nice board is inside stoppers, then we're going to remove that word from Arctic Nice boats list and store it in this final Woods list. One I'll do it is very easy and generated if condition. If the current world that regard is not in, it's just right, not in stop words and stop what is also a list so we can check it like this on then. If the word is not in stop boards, then the work can be finally stored inside this final Woods list on How do You do that? You just write final words. Don't up end on. Then you just append the word inside that, and now I'm just to check whether it's working or not. I'm just gonna print final words over here so that we know whether it's working on about the print line ledges, right? Print and copy and pay this from here to over here. And let's just print this out and see if it what's so let's for under mean file on. As you can see over here are the talk, nice words, all the words and then it checked whether the took nice words is in the stop words or not. If it's not, only then it gets appended to the final board's list. A lot. Get a light, this final words, lest by using our national processing concepts and just to kind of go through these three lines again because you might be a little bit confused. What we're doing is first, we created an empty list off final words where which will contain all of our final words that we can finally analyzed using an API. What we did was we first took all of these words that there were better in talk nice words on We made sure that we can individually check, but that these words are inside the stockers list or not to the first creation that I came over here and then it got checked. If I is not in stop, what's but it was there so it didn't get a rendered to file Woods list. And then it came love, which was not inside the stop words. So it got appended to this final words list and same thing with the Third World that despite on and you were a little bit confused as to how high tone is adding meaning to the sentence is actually in this case, it's not really the only emotion that we care about is love. But what happens is that we can't actually act all of these top pose in this world. So this is a very generalised off stoppers that we can analyze and say that these were should definitely not be in words. Otherwise, we'll be just wasting our time if these words are in a sentence. So first, what we do is we remove these toppers from the sentence so that our timing off analyzing text gets shorter right now. Just three words. So it doesn't really matter. But imagine if this contains, like, £10 in woods on then it will be very difficult to analyze them one by one. Obviously, because you have a good computer, you can do it very easily. But if this read or text contained like a book, this gets very important to remove the stop words that don't add meaning to our national languages processing. So, for example, I is actually if your hand realizing a book, I mean my these come a lot in Texas. That's why it's very important remove them just to kind of safe time. So now that we have removed text on, we have printed out our final butts, we can we can go on to analyzing what kind of emotions are present inside our final words on that we'll be discussing in the next video, so that is going to be really, really exciting, so I'll see you there. 5. Algorithm for Emotion and Text Analysis: All right, guys, welcome back. In the last video, we learned about organization, which is basically splitting a sentence in two different worlds. And we learned how to remove stoppers that are not important for natural language processing This video we are going to learn about NLP emotional Gautam. So this is the process through which we're going to be analyzing emotions and how much they are present inside our text. Five. Currently a text file is very shot. Let's expand it a little bit on before even we do that. Let's find out what this emotion startext find its but I want you to do is go through get up dot com After hurt sentiment analysis on just download this emotion store text file looking just click on it and then copied all the value over here. You can also just just go to raw and just copy and paste this clear new text file inside your project and just pay this as emotions start text file. Anyway, this text by contains worlds on what kind of emotion there presents. For example, if you use a word like victimized inside the street door text file, then the emotion that it will present is being cheated and similarly with other words. For example, it's a corned it's cheated on. Let's go down, I'm feeling down So it presents those This ward down represents the world on the emotion off Sad. So this is basically a text file, which contains a lot of words on the emotion that their present. We are going to be using this emotions in our text file to find out which boards represent which emotions by reading are read or text file and also reading our emotions start text. So if you go to the entropy emotional return first step, it's Jack. If the world in the file word list that is the final words that we have created over here, the final words right now on love and my phone. We have to check whether these love and fight on our in the word list or not. So currently the wordless and victimize accused acquitted on if they exist. So, for example, let's change our radar, take to something else. Let's just right. I feel victimized. Let's just copied the victim. It's were because I might most of this valley. Anyways, I just tried. I feel victimized because my current friend cheated on me. I'm a guy, so I should probably right girlfriend. But anyway, it's whatever. So I feel victimized because my current boyfriend cheated on me. So let's just run this and see what kind of file words we get. So I'm just gonna run this file and you can see the words are feel victimized current. The current spelling is wrong Boyfriend and cheated. So asking the list I think Victimizes were presented in Motion Store X. We'll just check whether the field world is present in this list or not. So we can just write, feel we're here and you can see it's not present, so we'll just ignore this word. The second word is victimized, which is present at the top on the current, which is not, and in the boyfriend it's not. Let's find out if the cheated this president cheated is emotion. But basically we can also search for like with its award or not, and you can see cheated. It's just emotion off cheated. So we have two words over here victimize and cheated that give us the emotion off cheated. So four step is going to be basically seeing if any other words in our finalist match up with the words were here in emotions dot Ex fine, but not the second worst that are separated by the school. And we're going to be checking whether the first before the school and all this list that is before Colon is in final words or not. And then we'll go to the second set, which is, if the world is present when we're checking, If the world is present, then we're going to act the emotion toe emotion, underscore list, feel we're gonna create another list. Empty list called emotion underscore list on if, for example, in our example, the victimized board is present over here on in our finalists. So you can see this is a finalist on victimized. This present in our finalists will check whether the final word is present over here or not . And if it is present, we're gonna store this emotion inside are empty emotion list advocacy in the second step on . Then finally, we're gonna count each emotion in the emotionless. So, for example, the in victimized and cheated both gave us the emotion off cheated. So we're gonna count it out. So cheated is there two times. So we're gonna count it out and also praying that inside are good. This is the NLP emotion algorithm. So we just if they read our X Files is really, really big instead of just one sentence, it contains a lot off emotions. For example, we have Right now we have to emotion shooting. But we can also kind of play increase it, let's say But now I feel happy because obviously they broke up and he is so happy isn't a demotion. So we have cheated to emotions and happy is one emotion. So we can, according to that, make a kind of a graph bar graph showing rich emotion separates in the most in this sentence in this reader x file. The cheated emotion It's president as the dominant emotion. But anyways, we're kind of kind of getting ahead of ourselves. So we're just gonna focused on the first step in this video that is checking if the word in the final word less is also present in emotion dot text. So the first depends just opening the emotion file. So let me just many my stairs and what we're gonna do is we're gonna write with Open because we want to open a file and we're going to write the name of the file. If you're finally missile, different are finally miss. Emotions start next. So if you have a different finally make sure right that in our case it's just emotion store text, and I would just want to read the file and not make any changes to it. So I'm just gonna write our that is read only more. And then I'm gonna save the contents off this file as a variable point as five, you can change the favorable name, but if you wanna follow along, just call it find Now all the contents off this emotion store text files are saved inside this file very. But no more people want to do is look through these lines one by one. And that is pretty easy by pound. You just right for line and this line is just a variable on. We will look through this fire so we're just gonna write four lines in file on this line. Variable will give us all the different lines that are present in this five. So let me just print this out for you so that you guys in the stand waters happening over here and you guys can also printed. So I'm just gonna print this means or by file, and you can see all the different lines are here, but you can see that after each line there is space over here. So what we're gonna do is we're gonna remove their space because every time it goes to a new line, it has all this extra space. So we want to remove all of that space, and then we want to remove this comma. We need to remove the single court on all these things. We need to clear them out. So the next step over here is looped through each line and clear, so we're gonna clear it. So what I'm gonna do is I'm just gonna create a new readable over here and undergo laconic yet line because we're clearing a line one by one. Because this is a for loop on. We're going through each line at each iteration. But anyways, what we're gonna do is we're gonna take this line. So right now, I'm just gonna write line dot on we're gonna replace it with something else. So the first thing we're gonna replace is this new line. All this extra space over here on a new line in Beit On is represented by this character off backslash. And so we're going to replace all this new line with a single empty character. So second perimeter is the start to place. Function needs is what do you want this new line to be replaced with? We just need to replace it with nothing. There's just nothing we don't want anything it will be replaced with. If you put the space over here, it will be replaced by a single space character. But we don't want that. So now we're just gonna print this here, line up so that you guys can see what's happening over here. So I'm just gonna print this out Print. Clear line. Let me just copy and paste this. I'm gonna run this again so that gets can see what's happening now. You can see that between each line there is no extra space. Now let's replace this comma with emptiness on this line is gonna be just all cleared out. I'm gonna die, doctor place again And now insert off new Lange. I'm just gonna put in a coma. We want this thing to be replaced with nothing. And then again, you got that right, Doctor, Replace on. We want to remove this single court over here, but we can't just straight single chord because it will get confused. So we're gonna write double courts on inside this double court. We're going to write a single court. We want to remove this single court on again. We wanted to be replaced with nothing. And you can't put a single court over here because otherwise it will become very confusing for fight on for bite on the radio and they won't know what to do. So, for example, if you just tried single course and inside that you can put a single court, they're just gonna be all right. I can see you for double courts on inside this. You can just put on a single court. You can only do this, That all of them. But just for the sake of this food here are gonna double coats. Actually, just to make it less confusing, I'm just gonna replace all of this vegetable codes that you guys don't get confused. It's removed us, but it got away. He also so you can also put single court over here. But just because I'm using a single court over here, I need to make sure that outside it article courts so that our interpreter doesn't get confused on now. We can just print this house, and one last thing I want to do is remove the extra space on different Andi back at the back. There is no extra space, but I just want to make sure this extra space behind it. That's what we want to remove. So and that has done very easily by using the dot strip function off pipe on the non. Let's just clear this out, as you can see and you can press control aren't on L on your keyboard to make sure that everything is formatted properly. But anyways, after doing that, pledges print. This means file again so you can see how it looks like. Now I can see all of the text has been cleared out, and now finally, we can separate the board on the emotion and how do you do that? It's very easy. You're just going to write would comma emotion and then we're gonna split it using this colon. So anything that is before the schooling is going to be saved in this variable off word and after that is going to be stored in this available off emotion. So how do you do that? We're gonna use our tear underscore line so this care and the scoreline is gonna be stored one by one. So first we're dealing with a vicious and happy at the first nutrition on the loop on what we want is to split it by using this colon over here inside the single court. I'm just gonna put in Coolum. So what this line does is it goes to this line on then it looks for this character off colon on anything after this character is stored inside this world off emotion, it is variable off emotion on anything before this character off Colon is stored inside this variable off would. And now that we have our word, we can actually check whether our final board list it's inside this word or not, or whether they are common or not. So what we're gonna do is right now, I'm just gonna print this world. So I'm just gonna write board on this, checking whether the final word this word is present or not I'm gonna be doing in the next video. So what, right now And just want to print out the word and the emotion so that you guys can invest time properly. What's happening here? I'm just gonna print out word, and then I'm gonna put extra space over here, and then I'm just gonna print out the emotion on after that and doesn't know, right? Let's write this and I'm gonna print out the emotion. Very. But let's run on means or by file again and everything looks good. Let's run on me not by file and you see what's happening so you can see that the world had been printed up, walked up on the emotion off. That is angry. So now I can probably separate the world on the emotion and the next video are finally going to be doing the second step. That is, if the board is present at the emotion toe, the emotion list that this check whether our final word list is equal to this word or not. If it is president. Then we can add it to our emotions. Underscore list Empty. Let's which you are going to be creating in the next video, so I'll see you over there. 6. Emotion analysis: All right, guys, Welcome back. In the last meeting, we note about NLP emotion algorithm on. We also learned about this emotion start text file with contains boards on the emotions that they carry. In this video, we are going to learn how to check whether the final words match up with the words inside this emotion store text file or not. So the last video, we also separated this emotions tore text file into words on emotions so that we can check whether the final words match up with the words inside this emotion store text file or not on if the matchup, for example, right now are read or text contains victimized. So we're gonna check if this final word is inside this emotions dark text file or not. And if we find out that it's that, then we're gonna count this emotion that is over here, and in this case it's cheated. So right now what we're gonna do is we're gonna create an empty file on empty list on really gonna call it emotion underscore list equals toe empty list. Find by just putting an empty brackets. So we're basically doing the second step that is if the world is present, then we're gonna add it. The emotion off the emotion underscored list file And how do you check whether this water is present or not? It's pretty easy. You just right? If the word that we're currently talking about this world basically is this variable if would is in final words. So this is the list that you want to check in. If the current board that is that we got from this motion does text file. So we're still retreating through each off the light. So we're gonna treat through this first line and we're going to check if this were victimized is present inside the three door textile or not on you already created a final words list that contains all the words inside this three daughters expert here is gonna check if the current board doctor treating through is in final words or not. If it is there, then you're gonna upend the current emotion inside this emotion underscore list and the list. What you're gonna do is we're just gonna write emotion, underscore list append, and then we're going right. The emotion we want to upend the emotions, for example, are victimized Would is currently present in the street door expire. So we're gonna take this emotion off being cheated on, we're gonna add it with our emotion. Underscore left empty list. We're just gonna write emotion over here and not just print this out so that you guys are understanding off what's happening. So I'm going to remove this print line from here and just outside this with statement. I'm just gonna print out the emotion list and they will be able to see what kind of emotions are present. So I'm just gonna run the play file the means or by foul annual believable to seize. Right now, our final word list is this on the emotions that they carry is cheated, cheated on happy and sad. So cheated is being two times and the happy and sad other emotions that are inside this read dot next fight. Now that we have the emotions over here, we need to count the amount off emotions under president cheated especially two times the happiest president one time and the saddest present one time. So we need kind of a counter to count how many times these emotions are occurring. Even before that Let's just copy of Biggest A instead of a small line. So just Google excited saying stuff. And I'm just gonna copy on this stuff from here and based it inside our read or expired. So at Casey, the current headline is a suitable sadness and whether it could be positive. So we're going to see whether there is some sadness inside this text or not. Something's gonna run this means or by file again on Let me just remove this final words from here so that because you already understand what this does, let's remove that. And I'm also remove this token. It was from him so that you only have the emotions that are being printed out. There's gonna print this means or final again, and you can see that the side is happening once the happiest happened place and then they're sad side again. So the side counter will be sadnesses up occurring four times than the happy is occurring two times. Then there is attached, fearful, loved. So we need to now counties emotional, so that is done by using something doing as collections. So what we're gonna do is we're gonna right from collections import something known as a counter. So we don't know, right? Condo on. Now we have important this. We're just gonna go below and it is gonna create available court the flu, and then you're just going to write counter on inside this brackets were going to write our list on this. Gonna count us how many emotions are present and how much they are present. And then you're just gonna print out this board blue so that you guys know what's happening behind the scenes. Let's just print this out and see what's up. So you can see that the counter state sadness is happening four times. Happiness is happy to times and other emotions are happening once. Now we can also convert them into presentations. But what I'm gonna do is we're gonna graph it and show it using Mad Lord Live. And now, just to kind of give you guys a recap of what we have done. The launch. The first thing we did was we cleaned our read or text file on recreated. We basically talking nice date and created final words list that doesn't have any stop words. After that, we just do this and I'll be emotion algorithm and it has three steps. First, check if the world is in the final word or not in the emotion door text file or not. So we did that by opening the emotions. Don't next file on. Then going through each of the lines be cleared all of the lines by replacing the new line , the commas and the single courts. And then we stripped them off extra spaces. And after that, we stored the words that is the victimized accused acquitted All these words inside this board variable one by one. And then we also stored the emotions in this emotion available on then reject whether the world, the current board and the situation is in final words list or not. So, for example, for us, it will check whether the victimized Woods is in the final list or not. Let me actually also print out the final words list over here so that you guys can understand what's happening properly. Let's pay. Just run it again and you can see that this is the final words list, and it contains a lot off names. Good earlier. Remember memories for these are all the work that inside final list. So first is gonna go through all of these words and check with any of these words are equal to victimize or not. If that equal to victimize is gonna take the emotion on the victimized that is, in this case cheated as good going to store it inside this emotion under school list. And after that, we just printed out this empty list on we're gonna count these emotions. So right now, the sadness is for because we have changed the door X files. The sadness is for the happiest, to which kind of explains why I say about sadness and whether it could be positive. So there's mostly about sadness, but it also had a little bit off happiness in it, which is mostly airlines. But anyways, it also contains all of these of the emotions. Now, in the next video, we're just gonna graph it on a mat plot lip chart. So and see you guys over there 7. Emotions in a Graph using Matplotlib : All right, guys. Welcome back In the last video, we learned how to analyse emotions and find out which emotions are present in our text. And we also learned how to count these emotions using our counter in this video, we're gonna be plotting these emotions on a draft by something known as Mac Lot lip, which is a library used in fight on Do create Jeffs. So what I'm gonna do is I'm gonna click on file. Then I'm gonna go to settings, and then I'm just gonna go to my project on Project Interpreter, and then I'm gonna click on this. Plus, I can on I'm just gonna service for Matt. Lord, Live on. We're gonna select this and click on strong package have already installed it. I don't know why I missed running it again, but Henry's I fear not using my job. You can just write on your command, Trump Mac. Large clip on Pip. Install it. So you're gonna write it installed, Mac large clip, and it's also going to be installed on your system. And after that, after it's installed disc lickle. Okay, It would look a little bit of time to make sure that added to your package. And then what we're gonna do is inside over here, just at the top. We're gonna import our Mac lip project or package. And I was gonna write import Matt Block lip, and then we're gonna import inside that something known as block. So I'm just gonna right by Lord on, we're gonna import this as he anything. So any time we need to use this Mac plot lived or by plot, we can just use BLT at the sharp form off plot to use this function off. Mac clark lib dot plot. And then you're just going to school down on over here. We're gonna create the graph by writing blog dot We're gonna create a bar graph, and it requires two values versus the keys. That is what emotions are present. So sadness, happiness. So where does not right the blue dot keys on the second value. It requires the values, which is 4 to 11 on one. So I'm just gonna right the blue dot values. And then after that, we also want to save the graph as an image inside a project. So I'm just gonna write BLT dot safe figure, and then what do you want the name off the image to be? So I'm just gonna write it as they just granted as graph dot PNG. And then the last line is that we want to show the plot on the screen. Summers been right here, don't show, and that's pretty much it. So actually, let's just drown our means or by phone and see if it's working or not. And even before that, let me just remove those printing final words for that we're not confused. And now we can just run this means or by a file and see if it works. So now you can see that the image off the graph is here. The sadness is for the happiness is, too, and all the others are one. You can also convert them into percentages, but it doesn't really matter right now. Now you can see that the image is also being saved as an image or grabbed or PNG. You can also open that over here. Let me just make it smaller or we can even minimize it and you can see that grabbed. Our PNG is being shown over here. Now we can just put in other things inside are read or text file and see how they match. So what I'm gonna do is I'm gonna goto our Steve Jobs speech that he gave up stand for. And I was gonna copy all of the text from over here. Just copy it and paste it inside or read or text file so that we can analyze the emotional letter present inside his speech that he gave at Stanford. If you haven't heard the speech, make sure you hear it. It's a big speech, very inspiring. And then just click on this means or by fire and run it again. And you can see that the emotions that are pregnant are different. The happy motions is five, then this Hated, entitled, Loved on. All the other emotions are one let's actually trying and lights the speeches. You can do these grass by clicking on this cross I can over here and then I'm just gonna just minimize it a little bit. And then I'm gonna go to Google and maybe open up this month's cover speech. So let me just wait for this to open up, and then I'm just gonna copy and paste. This all of this line just copied from here. Let's just keep on scrolling down. It's pretty big speech, huh? Hopefully we'll be able to find a lot of emotion from him. So I'm just copy this and pays this inside or read or text file. And their legislation are mean by file again. And actually, I'm curious because I know ran it before to see what kind of emotions are present. So as you can see, it's a little bit mixed on the X axis. This is what I wanted to address. So all of these, if there are a lot of emotions than these emotions, can get jumbled up on the X axis, so we need to make sure that they are a little bit tilted or dramatically. So what we're gonna do is we're gonna change it a little bit. So for now, let's just minimize this graph right now on instead of writing plt door part, What we're gonna do is we're gonna create to read ability. That is, if I d that sense or figure and then you're going to save the access over here. And then what we're gonna do is we're gonna create something known as subplots. We're gonna write pld dot subplot on. We're gonna use it as a function. That's why the brackets. So now our figure and the access will be stored in these two variables. Now, instead of writing plt door part, we're going to write extent X Bondar bar so that our access can automatically just And how do you automatically and just the access something known as automatic figured up dude on. People use this figure available for that, so I'm just gonna write f g dot a u t o that stand for auto and then format f empty underscored X date, and then it's gonna be inside brackets. So what this is gonna do is this is gonna automatically update the access, the X axis and the Y access to make sure that all values are presented properly and then we don't need to change the last two lines. So that's gonna be there automatically. Let me just for Mattis properly. And what we have done is that we have created a subplot. Dr has figured on that axis and we have stored them inside these two variables. Then we have used a explain dark part to create an actual craft. And then we we have used this figure variable. If I g toe automatically update X axis and a Y access and make sure that they fit properly . And then we saved the figure, and then we showed the plot. Now let's just run this means or by file again so that you guys can see what's happening over here and ensuring an error begat incident Subplot is supposed to be subplots on. Then we can run this means or by file again, and it's going to show you all the graphs. And now all the values. All the keys on the X axis are a little bit tilted so that you can see what's happening over here. So the Manticore both speech. Mostly, it's happy. And then there's some hated. I don't know why it is so much all painted in the speech on getting his off alone. Maybe he's telling his light life story and stuff. And then there's also a little bit of sadness. Andi Yeah, so these are the emotions that are present inside his speech. It's anyways, guys, this is pretty much it for this video and the next beauty, we are gonna in the light of the tweets. So, for example, if you search for hashtag Donald Trump, you can take all of the streets from Donald Hashtag Donald Trump. And you can analyze what is the emotions off the people who are treating those tweets? So we're gonna first of all, extract all the tweets from Twitter feed, and then we're gonna analyze those tweets, and we're gonna be doing all this cool stuff in the next video. So really excited to see you over there. 8. Twitter Emotion Analysis using GetOldTweets3: All right, guys, that come back in the last minute. We learned how to get data from the street door. Textile and shoe emotions on a graph using Matt Blood live. Now, instead of getting data from the read or text file, what we can also do is get tweets from twitter dot com and do sentiment on emotion analysis on doors tweets. So, for example, if you search for colonel lawyers, you will be we'll see what and emotions people are putting in into the tweets on. We'll be getting those tweets from Twitter by using this library and by Congonhas get old to its three. So this is a package of fight on library that helps us get weeds without using the offcial Twitter A B I That's just too cumbersome, and it takes a lot of work. I mean, you can do it if you want, but I'm just gonna use this. Get old to his three library on. How do you get it? You just go to get up dot com mortal immutable t l and get old to its three After you go there, just street a little bit so that you have an idea of what's happening inside this package on the install it If you're not using by charm, just open up your terminal and type in pip install. Get old tweets. Three. Make sure that the G isn't capital the odds and capital and the tea is in capital to If you're using find charm, you can just go to file, go to settings and then go to your project interpreter on Get this, get old Tree Street by clicking on this plus Aiken and then just tapping in g e t get hold on, please three and then click on it and click on install package. I've already installed it inside my projects. I'm not gonna started again. But anyways, after you have installed it inside your project, let me just minimize this or here. I'm just going to create a new file inside our project. I'm just gonna call it rid of analysis. I'm just not gonna touch my old means on my file. Just make make sure that everything is differentiated properly. So after you have installed get old with three package and created this new finally, we're just gonna import our new package on our cold packs. Animals get old too. It's three on. We're gonna import it as D o t. Game of Thrones. No, it's not given. Throat is just available. So that instead of referencing this big sentence off, get old we three, we can reference it as his duty. I will book saying you're gonna create a function, and we're gonna call it, get treats on. We're gonna put a colon after that. And now what I want you guys to do is go back to this. Get a page that is getting dot com model, get all these three and scroll down. And look for this example with you at the party that say's import get all three and sooty. We have already calling that. And now we want to get tweets by credit sort. So we're just gonna copy this from here on out, we're gonna base it on the screen. Inside, I get tweets function. And now over here, I'm just gonna print I'm gonna call this function. So let me just call this get its function on what dysfunction should do is it should get the query off European refugees from 15 4015 on it will only get one tweet and you can see there also zero here. So that's actually run this and see if we get our only one tweet about European refugees. So let's click on Let's Run This Means or by foul on Ashley were still running our means or by foul. We need to run this new data analysis by files directly calling Click on run Twitter analysis. Start by and end Turkey and see the tweet is yeah, so what we can do is instead of your hiding European refugees, we can just go with something else, like Orrin Awareness, which is all the rage right now and make sure that you are safe. But that's not the topic on the serials have hit skip Factor on. We're gonna put in the date off 19. So this is basically said since from when do you want the tweets on set? Until Zoff. Let's say we want that weeks from total 19. Let's go with a month and under total 20. Let's go bed, Maybe February. You're too, and they should be 28 because every 28 days on this was a leap year by the return on training, so it might have 29 days, but let's just go with Randy. It's fine. And then no, let's internal going with one tweet. Let's just go with 10 tweets and then we need to remove the syrup from here because we want all of the tweets on. All of the tweets are gonna be stored in a kind of a list form inside this tweet variable. So we're gonna just remove this over here on instead off make confusing it with just one tweet. Let's just changes to tweets. And now, instead, off for printing all this out. We need to save all these streets one by one inside. So basically the streets right now is object less so, for example, if you print this right now, we'd still be will see that it has a list off 10 tweets, but they're all informed off objects. So we need to get the text from all of these objects and store them inside under very book . That is pretty easy. We're just gonna create new wearable Andrew's gonna call it text underscored greets because we want the text from the streets and we're gonna make sure that it's equal to We're gonna put in a bracket and you're gonna put in another bracket. And I'm just gonna write tweets. This is the streets very bill over here, and we want the text from it. So I'm just gonna write dark text. So this venue where you want or text from an object you just started your text after it so that you will be able to get text. But now what we want is this should be just tweet, actually, because what we're gonna do after this is for treat in streets. So this is going to create through all of the tweets on one. But one is going to store it inside this variable off tweet. And then whatever the tweet is, it's going to start and get stored inside the street bearable. And we want the text from the street wearables were writing dot text on It's gonna save inside the street, underscore tex text underscore treats on. Let's actually print this out and see if it works. I'm just gonna explain this once again if you get confused. So let's just click on flavor than and see if it works so you can see that the 10 tweets on their California is monitoring at least talking for people, for career wires going states. All of these streets are here. So what we have done is that a list off objects court stored. So let me actually just added coming so that you guys don't can refused list off objects gets stored in Crete's variable. All right, so this tweets variable. And then what we are doing over here is that we're treating through all of these lists off objects one by one, and then we're storing it inside the street. Rare book on thestreet, variable on the text off and then we want the text to be saved inside this text underscored It's very bad. So first trading through beats list, storing them temporarily in treat variable on just the next line. Storing it. Actually, let's just right. Get next on store it as lest inside text underscore tweets. So that is why we're put an extra brackets. So we here. This is just a fancy way of storing and inside list. If you put a bracket over here, whatever the result off this for loop is, it gets stored inside this black it over here on this back. It also means that it's a list and then also brackets does it as a list inside the stakes underscore tweets. So we're just bringing them out. And you can see that all of the streets are over here. No, What we want to do is get emotions from the streets. Taffy ability, gardens, What I'm gonna do, I'm just gonna copy this from here. I'm gonna baste it somewhere. Maybe somewhere down below, and then I'm just gonna copy. Let's just got on base Tous Let's say it just been over here, Andi instead Off writing this text open this reading from a text file, we're gonna greet the tweets. What I'm gonna do is into dark printing taste treats. I'm just gonna return. This takes tweets. From here on, I'm just gonna create a new available off text text equals to get treats. I'm gonna just kind off just commended out for right now on duh text to lower nine. Giving us a kind of an air of it states that it's only for the string and not for a list. So, good thing I made a mistake away here because you guys understand should understand that this dark lower function can only be used on a string. And right on this text that is being returned from the text underscore, Priest only contains a list. So what we need to do is we need to convert this list into a big, huge strength. So how do you do that? What I'm gonna do is instead off right and get underscore streets over here. So what we're gonna do is we're gonna create a variable off expert and empty string. It's nothing inside it. And then we're just gonna just write a readable text underscore treats so disputable is different from what we're returning over here. But it's gonna be the same value. So I'm just gonna right get in the school streets, not a list is gonna be stored inside this text underscore treats variable, and then we're gonna get the length off this list. So we're just gonna create a new wearable called Lent, and then we're gonna get the length by writing in the end function and then inside this, unless that won't let off, and it is going to return a number of the length off the amount off elements inside list. So What are you gonna do now is let me just print this out first. You guys know what's happening over here. So right now we have 10 beats, so I'm just gonna print this out right now. Um, hopefully it won't give us an error. I mean, Bill Gibson error, because this thing is kind of empty right now, but whatever. So, as you can see, we have printed off the number 10. So that means the length of the tweets stem. Obviously, you can just kind of entered manually because you're also said the Mac Street then. But you're not gonna do that. It is gonna put in. We're gonna find the length of the tweets so that if you know, if even if you change this number, we don't have to worry about changing the number, or here too. So now we can just create a follow for I in range. So there is a function that we're gonna use. They're gonna go from zero to the length of the tweets, Reisner, right? And lend. Pull over here on right now. We're going to use this text variable. That be how? Over here, and we're gonna write text underscore tweets on. We're gonna get the value off the streets by writing I on zero. And then we're gonna add a space between different strengths and then we're gonna write text, so it's gonna basically add them. Tow this empty to Xperia bill, right? It's a basic addition off strings if you have the evident like in high school and stuff. So let me just format this probably so that it looks good. So what we have done right now is we have created an empty string on stored inside. This takes wearable. Then we have got in the tweets from this get treats function and started inside this available off text underscored treats, then record the length off the list. And then you treated to the entire leg, went off the list one by one. So this value will be 40123 and so on it will go to line. And then we got the text underscore tweets variable and you move through each of the elements inside the list. So first it will go to zero comma zero on, Then it will take the value of this so it can be any tweet about the current alliance, and it's gonna add it to the empty list or the empty string off. Next I'm going. Store it inside this text wearable. Let's actually print this out so that you guys can see what's happening behind the scenes, each iteration. So I'm just gonna believe this on see what happens. So it will take a little bit of time. It's minimize this so you can see that each of those strings. So First California has be friended. And now another attrition on the segmentation. It adds this string on. Then it asked the fourth string again. California is monitoring so slowly and slowly all the strings are getting into one string on there being stored inside this experiment. So now that we can just remove the Sprint statement away here on, we can run this program to find out the emotions instead off doing. Then we're gonna do 1000 so that you have a bigger later state off. Like the old emotions present. Let's run. It is gonna take a little bit of time because it's getting like 1000 tweets, and now you can see there's a graph with all the value. It's off the streets. I don't know why people are being happy and signed equally, but you know, like maybe they're happy that coronavirus have been treated or something like that. But anyway, this is the grab that you're getting after. We have extracted all the tweets on. Now you can just try around in different, like to eat values and see what's up. Your let's just try out something else. I mean, you can close down the video, it's already finished. But if you hang around and have fun, I was gonna go with Donald Trump on We're gonna go. Let's go with it on 18. On Uh, let's go with the first month Onda to find 19. Let's go with 19. Let's see, like all the emotions and stuff that our president of it in this one Yeah, let's run this again and we're gonna go with Talyn feats again. Otherwise, it's gonna take time like if you want, like a more accurate data said usually probably increase it to, like 10,000 or more. All right, so let's see the results on both of the people are fearful, sad, happy, attracted. I mean, like different emotions, but it's very interesting to see, like what's going on in society and what, like people are thinking using to stoop. So that's like the most fun I ever like ever have. I just play kind of check out the training tweets, and then I run this program and see what are the emotions present on different people who are treating them. Obviously, if you increase the max treats to a huge number, you'll get more accurate results. But anyways, guys is pretty much it for this video. Hopefully, you like that, and I usually don't ask this, but, like, make sure if you like it, just describe, like the difficult you do stuff. But anyways, guys, in the next video, we're gonna talk about the library off by pound for natural language processing and sediment glasses called N L T. K. It is like a very popular Doolin type Eitan and everybody does it for, like, sentiment analysis is, and like all kind off text analysis. So we're going to be looking into that like tomorrow and we're going to be changing a lot of things inside are means by fell on. We're gonna beacon boarding all the stuff that we've done manually to lt K programs, and that's gonna make it unblocked more quicker and a lot more simple. So anyway, guys, I'll see you in the next video, he said. 9. Installing NLTK | Tokenization and Stop words: All right, you crazy part on developers and this video we are going to be learning about the n l t k Library off. Fight on. Now, this isn LDK library is you for national language, processing is kind of like a trademark and like a standard which is used for text processing, text analytics and sentiment in Lance's bite on. If you want to know more about it, you can go to any lt K Dark arc. Read more about it over here and go through the examples on what you can do. What you can do it and I'll take it. It's just a good read. But anyways, back to work code. What we're gonna do is first of all, install and lt Gates. How do you do that? Very easy. If you're not using by John, let's go to your terminal and right Pip. Install on, then penalty kick. But if you're using my charm for go to your project project onto printer and over there you need to click on this. Plus, I can over here on just find lt gay on Install it. So it'll tickets over here click on install package and that's pretty much it now. We also need to install something known as a corpus, which basically contains a lot of words on. They basically tell whether the world is Bosnia or negated. It is basically a used data said that analytic a uses for Tex analytics. Just understand that. So what we're gonna do to install that corpus our data set is we're going to create a new file in our project sentiment analysis. After we have installed penalty gave where it is going, click click on our sentiment Analysis project full. Go to new and click on new pipe on file, and you're just gonna call it settings by click on OK, this will create a new file for us. I know Here you can just strike import penalty game on if anybody gets installed, this shouldn't give us any errors, and then you can just write lt gate or download on. This will download all the necessary things that are required on your computer. So let me just rectify the spending. It's a multi gate or download and after this just run this setting store by filed by right clicking on it on. Just run setting on this kind of a window will pop up on. If it doesn't pop up, Just wait for a few seconds. Sometimes it takes a couple of seconds to open up. And after that you just need to go to all over here, select this and just click on download. And it will download everything that is inside and decayed. And this is a little bit bigger insights. So it might take a couple of minutes, like maybe even 5 10 minutes to download it all. After you have downloaded it, you are ready to go, and you don't need to install anything else, and you'll be able to do all the national language processing in the world. So what we're gonna do now is I'm just gonna delete the setting start by file because I've already downloaded it. I don't need it now. I'm just gonna delete it from here on. Now, in this video, we're going to be using this analytical a burrito, recognize that it's played the sentence into different birds and also remove stop words from a reed dot text file. So how do you organize first? The first meaning to important analytic elaborate of the adjustment right from and I'll take a dot Duncan eyes were gonna import something known as the board organizer. So we're just gonna write what? Organize, and then we're gonna come down on. We have already converted into lower case. We have cleaned the text off punctuation mark, and now, instead of doing the dot split, we are going to be using the organized functionality off and multi cave. So we're gonna write word underscored, talk nice, and it requires some perimeters. That is the text that you want. Oh, Duncan eyes and the language. So you can actually took a nice different language like German, Spanish, any language that you want an analytic ape. The good thing about lt case that its multi language supporters So you don't have to worry about the language anyways for right now, our languages, just English. So first batter meat of us is gonna give it Dex. And in our case, it's clean. Next on the second para meter is English. As you can see over here, it's really so we're just gonna put a double course, and we're just gonna write English and the boat should have been split. The question is by shouldn't be just used dot split instead by using this word underscored Doc and eggs. And the reason is that when the reader text file gets very, very big, for example, if you're analysing a big research paper or maybe like a book or something to find out the emotions, darks plate is gonna take a lot of time. What this does is word on the school took a nice that it's been created by, like, scientists and like, cool python people to make sure that it transit the most optimum time. So this is gonna run like, really, really quickly. So that is why everybody in national and processing uses word underscored, organize and set off dot split, obviously their other corner cases, where the border in the score dark nights is better than tort split. But we're not gonna worry about that right now. And now we need to remove the stop put. So for right now, what I'm gonna do is I'm just gonna remove all of the stop words because we don't need this . And now we're gonna import something else from lt gifts of yours right from and I'll take a door corpus. So this is basically the data said that NL ticket has on from that, we are just gonna import the stop Woods. So it has a list off stop words inside its corpus data set. Corporate is just like an English word for, like, a data set. But anyway, it's not that very important On the stop words, we're just gonna remove all the stop words from our final words from Mark coconuts. Words. So we can just go down to every removing stop words on Now, what we're gonna do, because this stop was doesn't have, like, underscoring at the bottom. So we had a copy that, and we're going to just remove the underscore. From here on, we're gonna put in the brackets and tell little language. So the good thing What, lt case, that, like stop work. It has top words every language, like German, Spanish, Like I told you so, it has a collection off. All stop would in every language that this makes like multi lingual support Really, really easy, but anyways is pretty easy. We just wrote if would not in stoppers on we give it the language that you want. That is the English. Then it's cook gets stored inside this final under school board's list, just like we kind of did when you were creating and even Beaver just removing the stop words. Alright, guys, this is pretty much it for this video. Let me just run it so that you guys can see that it's working properly. So let's see our read or textile. How does it look? All right. Still the Steve Jobs coming since peace. I'm just gonna run the mean file again. All right? So it gets giving us America or lazy Corpus Luda. So what I've done is that I'm not included a pyramid over here, So let me just come down to stop boots on. Do you need to include in the dark functionality over here? And it's called dot word with the esto. It's words so nice that should be working. So he's gonna play it again. So stop was docked words because we want words from dysfunction off Stoppard's or class of stoppers. Ridge is gonna run this and hopefully everything will be fine and you can see all the emotions are being shown properly. Sorry, guys, is pretty much it for this video. And this we do we learn how to talk tonight on you. Remove Stop was using analytical, celebrity and then excreted, which I'm really, really excited about. And the last video of this series is how to find the sentiment and it's it's like, What stuff? Is it the positive sentiment or is it negative sentiment? So we're gonna find those sentiment using anti celebrity in the next video, so I'll see you were there. 10. Positive or Negative Sentiments | NLTK: Oh, right, guys, welcome to the last video off this sentiment analysis, Siris. In this video, we are going to be finding out that the text inside the read or text file has a positive vibe or in negative pipe. Or we can also say that this has a positive sentiment or indicators sentiment on. We are going to be finding that out by using something known as sentiment analysis Inside and lt kick. First, we need to import something known as the sentiment analyzer from an anti gay. And how do you do that? It's pretty easy. You just writes from in a decade or sentiment dot rado and we're gonna import the sentiment and density and ELISA on, we're gonna scroll down. You can just create a function anywhere AM was gonna created where we're counting emotional stress below that. So I'm gonna give it value off like the name off sentiment analysis. So I'm just gonna write sentiment or maybe sentiment analyzed, and it's gonna take, like, some kind off text, so it will take the text that is inside, read or next. Currently on that is stored inside this cleaned underscored Next variable. We're not gonna give it expects its worst artisan sign A list this analyzer requires The whole text on that text has been currently stored inside clean underscore text. So we're going to be sending this variable with our sentiment on the light function. Something's gonna call this available as sentiment. Next on, when we call the function, we are gonna give it the value off clean text. But right now, we're just gonna call it sentiment text, and then we're going to create a new wearable known as the score on this will give us the score. The vibe is positive or negate, or the sentiment is poor that they were negative on inside this we're gonna call our sentiment and intensity and legs are function. And make sure you add brackets on the right is not gonna work after that. We want a polarity score on inside this, we're gonna put in the text. So right now, the text is just sentiment underscore decks. And I'm just going to print this out right now so that you guys know what's happening in this function or sentiment. Analyze on just below this, I'm going to call this function, so that's just color function off sentiment Ramallah and the lights. And we're gonna part the value off clean underscore, Tex when we call the sentiment on glass function and we passed the clean underscore text function is gonna go from here to over here, and it's gonna get stored inside the sentiment next available. And then the sentiment intensity analyzer is gonna do it's magic and give us the scores of the Politan negated on neutral, right? Let's actually run our means or by fans will be able to see it in action so you can see down below it. We have a couple of values. This is the score value that we're getting. So let me just close these crafts, which you don't need right now so you can see the negative value a 0.91 the positive value 0.16 months on the building. On this, we can find out whether the whole why bother Texas positive or negative. In this case, the positive is created and the negative. So it has on overall for the fight. So what we're gonna do it now is that we're gonna get individually the values of negative on the positive on this is basically neutral. So if, like both positive and negative, it's kind of like the board I don't exist, then this neutral value is closer to one. But for I know he's gonna think about whether the positive is created and the negative, then the whole sentiment off the Texas positive and the negative is greater than the positive. Then the whole vibe off that text is negative. So we're gonna just do and if else if if, like, basic if conditions to find out where the device it's positive or negative. So let's do that. So first we need to get these values of negative and positive on this, if you don't know is a dictionary and we need to get the values from inside the dictionary . So that is pretty easy. We're gonna store the value in available called neck any G on. How do you get these values you just tried score and inside square brackets, you just put in the value off. Make that is this any G and then we just get the positive value Bearden right us on. But we're gonna do the same thing over here on the brackets from here missing. So that speaks that I'm gonna give it, though, who would be us? The positive one. So I'm just gonna write that over here, and then we're gonna check with an if else condition, whether the white best, positive or negative. So if neck is greater than or the diff, then we can just print out negative, send him in and then pensive. The positive is greater than the negative. Then we can just print out that it has a positive sentiment. So positive Send demand on the last is the is condition. Both of them are equal. Then we're just gonna print out, like, the neutral way, new proof white, and that's pretty much it. So let's actually run this now and see if it works. Let me just format this properly. And if you also want to if you're using patch arm and if you want any time formatted, probably you can just press control or an l on your keyboard on. Gonna make everything look good. Now that we have over here, let's actually print it out on, see if it works. All right. So this the Steve Jobs takes has an aural a positive sentiment. And these are the emotions that are presented inside the graph. Not feel free to, like, change the street Gortex and find out something else like do I kind of play around with it , Just like feel free to the copy, different text from the web and placed it over here and find out what kind off sentiment they all have and see what kind of emotions they have by using this Mac plot lip graph. Alright, guys, that is pretty much it for this sentiment analysis course with has been exciting hanging out with you guys. Like teaching you all this cool stuff on. Please mention the description or the comments below. Like what? Well, but what about all right, guys? So that was pretty much it for this video. Siri's off sentiment analysis. It has been fun hanging out with you guys, and hopefully you learned something. Cool. Andi, I'll see you in the next video. Siris on. Yeah, that's pretty much it be so