Sentiment Analysis with Keras in Python | Abhishek Kumar | Skillshare

Playback Speed


  • 0.5x
  • 1x (Normal)
  • 1.25x
  • 1.5x
  • 2x

Sentiment Analysis with Keras in Python

teacher avatar Abhishek Kumar, Computer Scientist at Adobe

Watch this class and thousands more

Get unlimited access to every class
Taught by industry leaders & working professionals
Topics include illustration, design, photography, and more

Watch this class and thousands more

Get unlimited access to every class
Taught by industry leaders & working professionals
Topics include illustration, design, photography, and more

Lessons in This Class

19 Lessons (2h 48m)
    • 1. Introduction

      2:10
    • 2. Introduction to Sentiment Analysis

      2:12
    • 3. Sentiment Analysis Algorithms

      9:49
    • 4. Workflow of Sentiment Analysis

      4:19
    • 5. Our Approach

      6:56
    • 6. Loading Data and pre-processing

      14:28
    • 7. Decoding imdb movie reviews

      10:06
    • 8. Working with Custom Data

      10:26
    • 9. Defining our Model

      10:36
    • 10. Word Embeddings

      17:13
    • 11. One hot encoding Keras

      10:00
    • 12. Embedding as Layer in Keras

      15:17
    • 13. LSTM layer in Keras - understanding Input and Output shapes

      11:20
    • 14. Training the model

      9:52
    • 15. Evaluate the model

      2:31
    • 16. Making Predictions on new Reviews

      7:43
    • 17. Correction of previous lecture

      5:50
    • 18. Making our model Deep

      14:51
    • 19. Conclusion

      2:40
  • --
  • Beginner level
  • Intermediate level
  • Advanced level
  • All levels

Community Generated

The level is determined by a majority opinion of students who have reviewed this class. The teacher's recommendation is shown until at least 5 student responses are collected.

58

Students

--

Projects

About This Class

Sentiment analysis ( or opinion mining or emotion AI) refers to the use of natural language processing(NLP), text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information. Sentiment analysis is widely applied to voice of the customer materials such as reviews and survey responses, online and social media, and healthcare materials for applications that range from marketing to customer service to clinical medicine.

Simple RNNs are not good in capturing long-term dependencies. In this course we unleash the power of LSTM (Long Short Term memory) using Keras.

Meet Your Teacher

Teacher Profile Image

Abhishek Kumar

Computer Scientist at Adobe

Teacher

Computer Scientist @Adobe

See full profile

Class Ratings

Expectations Met?
    Exceeded!
  • 0%
  • Yes
  • 0%
  • Somewhat
  • 0%
  • Not really
  • 0%
Reviews Archive

In October 2018, we updated our review system to improve the way we collect feedback. Below are the reviews written before that update.

Why Join Skillshare?

Take award-winning Skillshare Original Classes

Each class has short lessons, hands-on projects

Your membership supports Skillshare teachers

Learn From Anywhere

Take classes on the go with the Skillshare app. Stream or download to watch on the plane, the subway, or wherever you learn best.

Transcripts

2. Introduction to Sentiment Analysis: Let's start with a basic introduction to sentiment Analysis. Sentiment analysis is also known it opinion, mining or emotion area, and it refers to the usual off NLP External icis computers, little linguistic and biometrics to systematic cleared. And if I extract, quantify, study affective states and subjective information and in sort, you can see that sentiment analysis is the automatic extraction off subjective content from the still text and predicting the subjectivity, such as was two or negative sentiment. So what are the different applications off their sentiment? Analysis can be used. Who did our multiple applications? It can be used to understand the intent behind the language in an unbiased manner. In particular, it can be used in different business areas, for example, retail in retail. It can be used to introduce new products more successfully by understanding the sentiments of people. And it can help in understanding and responding to the customer needs. Using the reviews or feed rex. It can also used in entertainment industry where we can create or interest or excitement award movies by understanding market segment. It can also help in there getting movie advertisements and recommendations based on social commentary on collaborative filtering. It can be used in healthcare industry as well, or it will help in our learning or the patient wellness. It can enable to learn about the pace and satisfaction and what is working. And it can be used in virtually any customs centric or medicine customer centric organization. And it can help in responding to customer complaints with better. Asshole, listen, so I hope you are as excited to learn more or the sentiment analysts as a human teaching this course. 3. Sentiment Analysis Algorithms: I know that we have some basic understanding off. What is sentiment analysis? Let's look at the different approaches in which we can ordain sentiment analysis. So we will look at different sentiment analysis algorithms so it can be categorized in tow . Mainly three categories. One is rule based where you put some hand crafted rules on the system. Performed sentimental in analysis based on whatever rules has been defined manually in the next approaches. Automatic not rely mainly on machine learning techniques to learn from data so their rooms are important here. Date Asian port and in hybrid. We combined the best of both of these approaches to achieve even better readers. So let's look at these oh, approaches in some more detail. So first uproot rule based Here we define a set of rules and ah, these rules are used to identify subjectivity, polarity or subject off an opinion. The rules may use a very different goods, such as oh, classical NLP techniques for stemming stemming means. So if you have, like go going go on. These old words will stemmed to represent the world. Go then, kokanee Jason parts of a speech taking parsing. So these are also different techniques and these air for Italian cooks and other resources such as lexicons or the list of words or expressions. So, in order to understand our some are typical rule based uproots, let's look at an example. A very simple approved could be that you you define two lists. One is for you to sentiments. Another one is for negative sentiments and in positive sentiment, your body to list. You heard different words which the more positive sentiments making good missed great, nice, beautiful and form. So you treat your exhaustively stuff appointive sentiments similarly negative. You can have bird worst were and so on. So once you have your list Oh, when you're given a sentence, let's see. Ah, the movie was very nice. Really. Couldn't huh expected a better movie? Well, it's a decision. Small review. So here you see one nice is there, So watch TV. Corn becomes one, then better it can be stemmed or limit Taste good. So we have another push to sentiment. And here let's say there is no negative word. So you see, that was do Countess to negative countries, you so you can simply are define a ruler tiff or two counties More than negative Condom. It's 42. No sentiment distributes 42 0 but this is very naive away. And you will see that it can easily be manipulated to change the meaning. Even if the word is nice letter, you write not nice. And ah, then, ah, I expected a better movie. So you see that it starts to fail so you will start to modify your rule. So you will start considering negation of world like if not occurs then instead of what you do, take it negative and so on for you will start adding rules. Who? You heard some rules. You heard more rules and soon you will find that this system becomes very complex and it's very difficult to skill for new rules. News with Tucker Maybe your old rule does not hold true, so you may need to mortified art also and solar drawback Is there hard skin and initially it may look simple, but very could leave it a reason of fuels. It starts to become complex. So let's look at us off the recruits. This is that automatic approaches. So here no handcrafted rules, but instead it relies on machine learning techniques, which in turn relies on really later. And this is mainly mortal. Let declassification tasks. So in machine learning of er, classifications and regress and trusts regressing means predicting a real valued or group. Really Classifications means Ah, predicting some or put will be some class, class one, class two plus three and so on. In this case, let's a simple will be two pluses. Why would you even need to? You may have three classes were your new trim. You can have more classes like very positive, very negative a technique and so on. So it's more allergic. Classic isn't task and this classification algorithm can itself be different like one can be now you basic classifier. Then you can have Ah, there s Veum. You can have also other class Ifor's like deep learning based class affairs. So in naive base, you will, uh, predictor. What is the probability off? Let's appointed sentiment given a text text. So this kind of prediction using the based here, um, in his Veum you map the different examples different reviews or different pics to different points in a multivitamin is no space. And ah, in this approach, it will be assumed that all the texts belonging Toa one sentiment map to some six treason in this multidimensional space and other sentences. Text belonging to other sentiments. Map to some different regions, and when you are given a new data, let's say the new deter maps to this region. You'll see that this maps to positive sentiment. Reason so you will closely for you. Test what you tip sentiment. But all of these known machine learning based classic fires take into account individual words or Mary characters. But let's say it's words and not the sequence in with Decker is not taken care by these. So we use deep learning these techniques. In particular. We will use our in and recommend neural network units called a less teams. Richard Good in capturing long term dependencies, long range dependencies. That is what occurred word One word toe work three word for so it extra may have multiple words, so it will not just consider this world, but also the sequencing with soccer. This is important on this Celestine will kept to this long term dependency like what works have already occurred, or even there can be extension of this nerd by your listing bi directionally list him that is works in the neighborhood off current word that is being, um analyzed left side as a less right side. So this kind off for techniques will use the Consider the sequence of words because these are very important, you know, analyzing the sentiments and ovular study about the typical workflow RSI automatic approach in the next will you or what's the typical workflow? Now let's look at the last approved, which is a hybrid uproot. So here we combined the best off both the worlds. As the name suggests, we combine ah rule based as well s or chromatic approach. Oh, so this is mainly used to achieve some last level off a Christian precision lets it using simple deep learning. The generator. Your learning algorithm has that you to 85% accuracy, Noah. You add some rules, some handing in hearing some expertise to improve it further to, let's say, 89% off 90%. So bizarre rules are usedto achieve some additional accuracy and precision. So these were the main approaches for ah are doing sentiment analysis. We will further continue in on this or thematic uproots and how to do it using a less teams in Cara's 4. Workflow of Sentiment Analysis: in this. Listen, we will see what is the typical workflow sentiment analysis. This is not a hard and fast rule. There may be some reasons they're different places, but this roughly conveys died. So the first speech is together. Instructor data so and this may not be required always because sometimes you may be working on some already available electricity. But if most of the business applications you will need together some instructor data the new Lueder, data cleaning and pre processing, this also may not be record always because for some, because it's like we will be working with, I am to be abused to tacit and there we already have. Ah, collection off. 25,000 reviews and even the words are indexed by the deer Buddhism. The ranking. How frequently, Decker. But still, some pre SIPRI processing may be required on about that. For example, all of those abuse may be off different world land, and you like to work with reviews having same virulent. So you would like to either drunk it or paired with some, uh, me, uh, in this is so even if we have three persistent er and clean NATO, we will still require some pre processing and hair data cleaning means to get rid off redundant works. For example, O e and the deer, the stop words and some punctuation marks which don't convey too much information because e and e and these kind of words, you mean these can be present in, ah positive sentiment as the less negative sentiment words. And these don't generally changed the meaning. Not here comes under the data cleaning part. Once you have clean and pre persistent data, you are ready to convert it to trace data. For example. Oh, your model will not train on simple English words are words in any other language. Oh, you need some numerical representation onto with words. We call it with trade ism and here also there are different techniques. For example, you can have ah, just a bag of worship orientation where you represent the world. By dear frequency, you can have ah, corpus awful attitude in good works and you may have one hard with representation were Let's say you weren't proficient. Third word. Then you will write one and everything else zero you can have. Ah, other approaches like contra trade your Deif idea and there are so many ways off wreck tries in your data once you have victory dure today you have to that I want bitch algorithm . You weren't for the classifications. Sentiment classification for example. Some algorithms may be very acquitted, I inaccuracy. But dear performance or ah, efficiency, maybe very less What? Your application recourse A high performance algorithms even if accuracy is slightly less eso you may ah work on tooting retail Brigham best suits your needs. For example Random Forest classified as a high Chrissy, what may not be very fast in on the cases. Then, once you have decided on the algorithm, you will start the training and cross validation on you were trace data where the model will try to learn is done whatever data you provide and the corresponding sentiments And once you have trained your model, you are ready to make predictions on and newer data. So I hope you understand our this is the typical workflow. All of these blocks maybe combined or split into even more blocks. But this is the overall work fruit 5. Our Approach: before. Actually moving to the implement isn't. Let's see what will be our approach towards sentiment Analysis what steps we will follow. So you have already seen this diagram the workflow which is a typical workflow for doing sentiment analysis In our case, we will not We are gathering instructor data. We will use regiment there does it and I will shortly give the details of what data said we will use. And then you are data cleaning part. We will not do. But we will do some pre processing in our court or because our different re Bulent are off . Reviews are off different lens. So we will try to chop them toe fixed length or if it's sort and well, guard them with some dummy works. So we will do some pre processing. So this will be part of our work Fluke Then this Vectra GIs and park we will not manually do. But we will use emerging Lear off. Get us where it can do the wording waiting for us. Then we will define our Morgan. In fact, this layer will also be part of our modern. And we will also use a less team in particular for capturing the long term dependencies. That is what is the sequence of words and not just words as discussed in over or listens. So this philistine can capture those sequence of word in his decker on Ben, make the predictions and was were defined our modern. We will train it and do cross validation, and once we're so destroyed with it, we can use the train more than four. Prediction. So let's let's first to see their detested. We were using a very popular I am Devi movie beauty to sit and this duplicitous he doesn't reviews ordered alert 25,000 days for training set on 25,000 trees for just said and half of the reviews are poised to on our first negative. So this is the over you off over there doesn't. And then let's look at the model. So this is our mortal, the layers that really fine get us. So once we do pre processing, we will feed their data tour model. First layer will be embedding there, Andi, it will do the return the word in beddings to the esteem layer, which will do the major part off training and then finally it will or put with fully connected earlier visual huge signboard activism on this signal right activist envelope, give a lieutenant general and one well, you sigmoid function is like this. All values are between zero and one, and this is your importantly for its 1/1. Plus, he was two for minus six, so this will return overly. Whitman Giron one. Let's say your point seven it so this will mean that it's very close to it was dissented, and so we will market us was to So we can make it Jiro or one depending on the value return with this last layer, their dense layer. And once we have trained it, we will make the predictions. No, let's briefly look at our approach and what functions a PhD we can use in the charas. Charas is very simple to use a p A, and it's good for beginners. You can easily learn it. So first is loading the data set into the memory, so this can be easily done using terrorists nor datasets dot I am Devi. So you see in built him Devi inside the terras editor circle every and just called Lord later and you have to look at the perimeters like numbers. We will discuss it when we start according on it, so it can be done using Lord return. Next is defining the model. We have to define the model so we will. We can use sequence syllabi or we can use functionality. In our case, he can still appear. You will surprise riskless sequentially. Oh, if layers are sequences, that is no are part of one. Larry's fed to the next layer and there is no parallel lives like this. Then we can use sequence LAPD's so we will use those sequences immortal on dot aired. So this area is for adding one layer. We were defined the type of player and we can have more layers. So first we will heard embedding layer, then heard a less steam and flamed leave alert. Weird, insular. So we will have just three layers in our case on then. Ah, once we have defined our model, we will compile the model and it's also very simple and introduced, so this will return and model object. So on this model you have to just call compile the different parameters which we will see later and once compile, you're ready to train the model. So for training you need to run the fit on this same word. And here you have to pass there training in boy train. So then put under Chris warning, or this will report your negatives and during this will really, actually oh, emerging so awards. And once we have trained it, we will evaluate our model on new date. Is it so busy? Also Very simple. It's just even ready P. And once you see that it's high precision accuracy, ease I. Then you can move forward and during fit you have to give the number off. Epoxy Want to print a box means how many passes through the there Doesn't it has to make. So once you are satisfied with, they were loose and face. You can save the model for using it for their because every time you want to find the sentiment, you will not reach in the more the land and predict using that model you can save the model in some format with which we will see later. And you if you have saved it, you can let resist Lord, that train the modern and start. The predicts and bill are training next time. So once you have saved on a little, you can make it prediction using that same model. So this really our typical work through now we will dive before into each of these individual components. 6. Loading Data and pre-processing: No. We will start with the force to stage in center Internalised, which is loading the litter and then re processing. So just briefly kept this was the typical workflow for sentiment analysis. Today we will focus on this part. First, we will load the data. We don't need together it because we will use I am David a tacit off movie rib use. And then we will lose him re processing. So the row I am David data is a structure it 50,000 individual text files to eat. Review is in a separate textural where sentiment is also part of the file, either zero or one. So working with this is very difficult. But you're a slavery has building some model called it deserts and we will use their from this some more. Do you can directly use this data search? Snort? I am baby and then we will call lord bigger. So this becomes very easy for us And here all the values all the words are indexed As dear rank the most frequent words will get rank of one Next most regrettable word will get rank off two and so on. So let's start over. According So I will be using Google Columna book for most part of this course, and this is very similar to Jupiter Notebook on your local system. But here you can leverage the power off movers GPU So let's begin so we can write X you can see takes time court. So it's very single to do, Colonel. So So first we need to important, um, play because we will be using number Lord in all off our projects. Then we need to import care us and we given earlier systems. So there we don't need to write grass hope at all. The places we will is a straight Okay, at the moment, we don't need to import this, but let's do it and add some message, sentiment and analysis on IMDb. Notice it with us. And in order to run it, you can simply click this button or sift flys return key. So no, you can see it's using tensorflow is back in so desert before back and for cure us and kill us. Just level appear you're up off tensorflow. You can even change the beckoned. No, let's lord their data. So I will insert text six and so first we will sit. Max works on. We will come to it. Didn't he told you aan den? Let's bring some message lording Nitto and no Food Lord the detained to train and just starts and we will call Kira's do test. It's this is the submergence and then the name of the details. It I am the week it's even suggesting us. And then gorp Lord deter. So even if you don't remember these active ticks you can use this suggests in years, Lord deter. And then we will pass numb words. So if we don't pass it, it's fine. So you just lorded. And then let's bring the 1st 5 rows a boat train X and train way train. Why will be the value why you which is your dear old one train? Next will be a long list off words or terror. We will see that in this is and more back to the words. So it will take some time because it will First time you're on it. It will don't lord that it is it. And these air the 1st 5 abuse and the circus morning sentiments. So one means sports too. Joe means negative so prostitute maker to maker to or is to make it to. And you can see the long list here, and each of them has different land, so we will lose some pre processing. But first, let's say we have 34,000 these older dear Franks. So let's say we don't want to hurt everything. We will limit the number of words soar. Pick the intelligent, most frequent words. So we have a optional perimeter here where you can specify Max Wards, which is 20,000 in this case. So now you will never see any sank more than print decision. If the rank is more than 20,000 to you, see a place order in place of that. So, in fact, 20,000 is a big number. Let's try it. Small numbers. Let's 100 just 100 most frequent words. This is a very small number, and we should see the difference. All those big numbers will be gone. You don't don't see Noah bullet intelligent and 16 million, but you wonder why you are so many to air. So that comes from the fact that whenever we encounter any word who index whose rank is more than this Max on them. Words better. Me, too. Then there will be fixed index in place of that. So by default that in Texas, too, you can change that, using another parameter gold. Who will be there? You can see the first in so you can set it to, let's say, minus Tim. So this is not a valid index because it's negative. No, let's run it again. And you see lords off minus 10 year because these indices will be more than 100. So just to remind the Genesis dinner Hope Frequent award occurs and voted for this values. So you can skip this for no and let's keep it some world number. Let's 20,000. So no, we have lorded their deficit into memory into a tree. Next train way and it sticks. Destroy? No, we have already seen. Oh, we have very well lent off rib use. So people's one well chopped him to a six length. We will top them to ah and maximum rebuilding Tor 50. So it's still entities more than for yours. It will simply chop it, and if it's less than 50 then it will pared some values. And that is what we will do you in the pre processing. So let's add some text to Let's decide on the maximum. Bulent. Let's keep it a terrific to be reasonable number slightly on the lower side. And then we will obey Train X entry in ways train accent districts Because these way values are just hearing what these don't need. Any people are listening. So Train X equal to get us nor three processing door sequence North Barrett sequences and these assistants would be very helpful because you will not remember all of these. Exactly where underscore occurs, we're oh, space occurs art sequences, and then we need to pass Train X and we need to satisfy. If the limit exceeds you want to chop from the end or from the beginning, so you will have to Watson's. So this is specified by truncating and it will take to will use either pre or post. Let's truncate it from the beginning, and then we're well, so the 2nd 1 parting in case it's less than those satisfied limit then shared values. This also were doing look beginning the and then we need to truncate or paired based on what limit their guerrilla satisfied Maxim and this is Max, it of you lint. So let's who do the same thing for just and let's also bring to the first free growth so that we can compare with the earlier will use. We don't need to print the way because doing are unchanged and knowledge Run it so you see year or the general same side, these air length 50 you can even count and it will come out to be 50 now each of these air , same land, no, just toe. Make some clarity Here. You see, it starts from 07 months and in the end it's this. So we have trimmed and trinket truncated and better both in the beginning, so it would have trim some values from the beginning. Let's match the last few parts. So it's 1 70 32 same as here than 19 for three full for you, then 16. So in the end, it produce because we're truncating from the beginning. Similarly, you will find who 071 somewhere here. So it takes from here to land and trinkets all values before that. If you right here drank it most, then you will see the difference. Okay, so we do not see any difference. What is this? Because the first time the renders the already or truncated this train next to will enter 50. So tree next now contain just 50 words. So next time we run it, there is no effect we got in there in north youngest trunk is and there is no parting. So in order to see the difference, you will need to run it again. And then if you run this, you should see their difference. Yes, so you can see it. The foot 50 words have seen So I will keep it brief or no. So this way we can lord and do the pre processing. Now we're reading, Ready for or the next to stage another replaying. 7. Decoding imdb movie reviews: in our produce. Listen, we had lorded of the IMDb movie reviews detested into memory and we had all certain, some pre processing on them on. We had seen that when we Lord of the Memory, using this Lord data function, we get these two couples e to it. Our train, extreme way and mystics destroys. So we were We have seen that destry next will country in a list off different reviews will , we will have some index leg one couldn't be 1000 Pippin and then this list ends and another begins. So this denotes a review. And I had said that these numbers, you know, the index, the rank of words, How frequently, Baker. So that is true, but not exactly true. Why is that? Because these indices are not exactly the rank here. One does not mean that this is the most frequent word and turn. It does not mean that this is the 20th most frickin word. But this is close to the rank. Why? This is because first few or numbers are reserved for some other purpose and the actual drink starts from three. So when you see this 20 them, this is the not the 20th most reconvert, but 17th most recon word. Similarly, this is 9 97th most stricken world and this is just more stricken Ward. So what are those numbers reserved for? We had also done some pre processing where we said that we want to keep the length off all the abuse constant. So it's any length exceeds in our case, it been drunk it or the review and effortless than it did, then paired it with some values. So you see here, herding is DM. So the first index started. Jiro Jiro is reserved for birdie. Then you are would have noticed that Ah, we're getting one in front off more all the abuse. So this was not a coincidence that every year train weaver, uh, getting the most frickin word in the beginning. But, uh, this was reserved for start off sequence, so one denotes start off sequins. That's why one is present in front off all those reviews. Then we're also seen a perimeter wound. Care the blue V, which is turns for although vocal ability. And we're trade yuning with it and we serve. When we kept it minus 10 and we kept ah numb words Very small 100. Then most off them were replaced with this director. Richest terms for also or somewhere you can see unknown character. So this was turns for or vocabulary. So who is buried before trees or for in there? And so three is the most recon word by before. So no, let's see this in a record. And we will also try to dick or do it abuse into words. Because we are. The humans cannot or die. Just the thing that the rib use the same way that computers cannot digest the natural language via the human beings cannot digest these numbers. If somebody told else you d these are there different movie reviews? This will not look much interesting to you. You want to know what are those that still reviews? So in this case, we've been discord that. But first, let's see some off those indecision literature. So we have kept Maxim Bulent its beak. So in order to see the effect of parting, let's make it a very big number. Exceeded Pozen. No, we have prepared values because it 1000 is a very big number. Hardly any review lab it opposing characters or even 800 solar. It's just start with the 800 and men views he defect off burning. And I said there, Jew, Israel for parting. So we should see locks off Jews for beating this part or let's run it again. So you see, Lord sauce, Jews are there and then some values. So in order to make the rebellions 800 it has to parrot Jiro because there is very less chance that reviews will be 800 words long so you can see Jews pattern here. No, we have to see the Knicks won, which was reserved for a start of sequins. And you see that all of these have one in front of them, so we can change that before it will you using start care. Let's say you want to start from three and let's run it. So no, use it. See three in front off all of these, so one has been replaced with three. So let's keep it to default. And we had already seen we care, which was before, too, for any value which goes out off the strange to Teligent. No, let's d chord these values, so this would be interesting you want to read the reviews? Who lets heard cooled? If so, did a function called get word index which gives you a map, Were the first He's the word and corresponding index is the value. But we want the reverse in order to be called that first, let's find the word index, get word index and then we will let sprinted. So this gives us the map. But in orderto record that we want these as the keys so you can see the world and corresponding index so we can reverse this. Sochi in value becomes value in key so we can create in new dictionary. We call it index word and what we will do, we will. I trade the first dictionary so the wait, Wait treaties four Keiko movie word Index So what? We will do this. Well, put it here the kama cake. So we have reversed it and this would give us the dictionary. We also need to call the items and no, we have a dictionary. Were the keys index and the values word. So using this weekend no for each index and get the corresponding world and we can create our rebuke so No, we will create our rebuke in next ward North get Are you dukes for RDX in? We will pass the this review in this form which is Dream X. Let's read the first tribute but we need to do 1 10 year these indecision three more Then direct your values to make them my industry. And no, let's bring the rib youth. So this one is Nord actual part of the review. This is just beginning of sequins and here the review starts. So this film was just brilliant casting locations, Injury story Derrickson Everyone is really suited the part they played and you could just amazing being there. Uh, Robert been some hold the word is an amazing actor and no, the same being director. Oops, Father came from the same score Tous Island as myself. So I love. So now you can see this makes a proper or English sentence The fact there was a real connection with this film so you can see that this recording seems to be correct. So this way you can record it. So I hope you get some interesting information from this lecture. Although this will not be helpful in directly analyzing the sentiment. It's very good toe. Know what we're doing 8. Working with Custom Data: for most part of the course we will be using. I am individual tested, their digitally available in Kira's and ah, we had lorded this data set using Lord later. But for most business applications, we would need to work with our own deficit. And we will see how to work with our own data sets and how to get train extreme by just text test away in the same format as lording I am lividity. So here, if you remember this loaded, I returned for people's compulsive consisting off train extreme very districts destroy where this why did not the level zero and one which do not put your negative sentiment. And this tray Knicks returns a list off abuse and not as works but bear in quartered in the foremost numbers, for example, one which denotes start off very beauty, then pulled in save and so on. Similarly, for the next tribute. So this trilogy, North's other and most frequent ward in the a M. D videocassette because we have a job the most one do you want to buy before to You can change these. So do you. Israel for parenting. 14 Start off the review to four auto vocabulary. So how can we get the same form it with our own data? So did you beat those steps? First we will load the seas. Rio or your you may have sees your Excel. Former like. Reviews on the second column can be sentiment, so this can consist off a long text. The movie was good too, and so on and sentiment May leader one. Similarly next tribute. Next sentiment and so on. So once we load it as a metal frame double, this consists off the X and this is the way notice level. But it's not in this former that we used in. I am really so first. Ah, you need to define it organizer, where you provide how many, most frequent words you want. For example, if you want to just work with 1000 most frequent words, then you will provide numbers. We will shortly see all of this steps in our court and then it will be more clear. So you have to be fined a token Asia and then are using talk in Asia, you will create embedding source this list off reviews so in this is steps and this level will not be used. So this will give us a word index, which is a dictionary off word and the index like word one. Oh, been Were who? Five. So visitor index by volume more frequently award occurs, so it will be very similar toward index week. Or for I am David abused. Then using this dictionary, you need to convert these reviews to this apartment. And for that we will convert. Are you another epi? A for converting text, two sequences. And this will give us exactly in this former. And once we have this, uh, ex, we can use way by just taking this column oath. Get off him. So just have a look at our small since we foil. Besides just three rebuke. So it has two columns review and sentiment review consist of the actual text and center and consist of dear old one. So now we will lord this as the difference. So first we need to import one does and then b d Dort read sees me though new, more function. And here we need to provide the name off sees Refile North sees me. So this will reading two reviews death the Let's print, though it abuse. So in this case we have just three tribute so we can print the entire review. But, oh, if you have a large detested, you can just print the 1st 5 rules off that in this case it does not make a difference because we have less than five reviews and we will also read, though text that is the first column this column into a separate text, because we will be doing the according based on this only. And the name of the column authorities attribute you should read correctly this one. Similarly, we can read the levels so this will read thes two column separately, which will be used later. No, we need to organize these. So for that we will import from Caro Snort pre processing dork fixed in court organizer. And then we need to define their took teenager the various perimeter. So number words is how many more stricken words you want to work with. So we have a small example, so let's set it toe five. For practical purposes, you will certain in terms of intelligence like 30,000 but we are just illustrating this point, and it has. Let's it won't be unique words and we weren't five unique words on Lee. Five will be the maximum. Next we're we will reserve some words. And if you don't want to difference here between four Stephen Negative Ah, operable or kiss. Then you can use Laurie call to True and this is the default. Everything will be treated as Lord and then you need to define out off the table token. You can also skip this in their case. If intern Gordon in countries, any art off vocabulary Ah, a word then that will be simply skipped. But if you define will be talking, let's say we define it as unknown. Then it will be put into that unknown. Well, we will reviews once we have to find number organizer, we need to use it to create tokens of organizer's door. So one fixed and here we will pass this Oh, the list off reviews not the levels just reviews. So we will pass or e b txt. Then this stock in Asia will fit on this rev exterior tokens and we can save it is we can print the word index netted return their dicks lately that it created. So let's printed. So you see, we get a list off words and you're responding or in this is so here you will see everything more than even five Ah index. But when we actually converged the reviews toe dear sequences. Then we will see that it will be limited to only five. So let's create the sequences. Organizer dork, Fixed text, two sequences. This is the A P who won these free turn texts. Then text two sequences. So what will it do? It will convert all of these words these reviews into They're responding in courting using this dictionary. Let's print So you see over rib user low converted to list off numbers. So you see Oh, it's limited to food. There is nothing more than four. If we make it then then this change. No, You see upto name So maximal Louis nine Wyndham World system. So this is how we work with our custom, greater search and obviously for ah, in training you will need X and way so extreme this will be the least or fixes. This is the first tribute. This is the second and similarly you can print the liberals with collectors. Why for your training, So you need to convert it to oh be similar list 9. Defining our Model: in this listen, we will start defining our model. So earlier we had seen a brief glimpse off our model. And, well, I had said that our mortal little consistent three lives one will be the first will be in birding live on the job off him, meeting Larry's who can work a word to a word. Embedding and the sea is an embedding, but so every word will be represented way. If explained off Victor's Let's Eaters Frantic Components District, so it will have 20 components. We When we do, we 20 and there are different ways off embedding it. We will not be bothered with that at the moment. So the idea is that every word is presented in the form of Victor on the words richer, similar. For example, men on men these would be would have very similar, but not exactly same because they are different works but similar victor as compared to a man and woman. So these would be different. Similarly, women and females should have similar victors, and we will see or different emerging technologies later in the course. Eso, once we have the emerging victor, were demanding off each word. Repeat it to our LST model Ellis timelier. And this ls team leader we love are different A list in blocks and, uh, this will capture invades order. The words have occurred and it will take care of that. Goes estimates Gordon capturing long term dependencies. So the order of words he also important and not just the individual words. We can also use some other RN in units in place of this for jumble Jiri Gear you is another alternative gated recurrent units. But here we will use a less teams. And then we will feed this twit fully connected lier and it will put the value which would be between zero and one. And we will be using signboard the reason for that. So if its value is close to one, they will say that you're supposed to rebuke. If it's the loose, a low value that is less than 0.5. Then we will say that it's a negative value. So that's a start defining over Morton. So first we will need a few layers So really important news from here. A start lose import a list, um, and dense. And from there are snore loose nort in ridings import invading list and from carouser models we will. Imports he concealed were also are talked about sequence in Modern, that if there is no parallel branching, then we can use sequence in model. For example, if we have one layer after another in the network, then we can use the C conceal mortal. But if the Epsom at little Brunch and we are merging somewhere, then this cannot be represented using Sequence Hill. But in our case, it's a sequence immortal, so we will import to see, conceal from or curious, and we can also right in line these things like Kira's total years Dort LS Team. But it's always need toe import these before in and just straight. A small works later so that our model is in order, very complex. Now we will actually start the mortal building process, and we will store the model into a variable model sequencing, and we can also add do it in length. But let's keep it separate or rating it cleanly. So know we learned our first layer, which is and reading, and here we need to provide the input dimension and this will be done. This Maxwell, which was 20,000 in our case and artful demonstration. So how long you want you're invading that just to be So I said that eat a word will be represented with its 60 lend victors. So what would be the embedding off a excellent so that we need to define so 100 would be a good number. So that's do fine. Young. These Iqlim equals 200 So each of true love 100 conference 100 demonstrators. So here we need toe give that will you and then mosque Geo he could talk to And then we will learn over a list in here Mordor lurker. Similarly, exactly Same on leader partum interested teams. That's if be a units. And finally we love it fully connected layers which is dense and we will predict just one value. It is a value between zero and one on Really used the sigmoid activism for that and we have already seen What did you sigmoid function this? So just a brief recap The sigmoid function zero It will lose one at negative. Infinity is close to diesel. And that boy, Steven Finn t It's close to one search of the elusive between Joe and one and it's represented way one over. He stood or minus six. No, we have to find our model, so we'll want to visualize it. So there's a very cold somebody you can call somebody mattered on it and let's run it So it has successfully grin on. You can see this some reprints all the layers. So you're is he layer along with the sea on the number of parameters. So 1st 1 is emerging. You could also give name here for each of these. So by default, Kira's will give some name like LS team under school one. But let's give name to one of these sentiment analysis A list IAM looks certainly to again and now you see that name occurs here. Similarly, you can give name, toe oh, other layers. Also, other ways guitars will give some before values and there is another way of visualizing it , which is even better than this summary. So for that, we need to import Lord more than and then we will call it Austin Martin. So stapes equal to and you can save this it'll or Putin image and you can say with your file. So this is the output of summary. And this is the Lord model. So you see, it's even better, and you can see the sequence in modeling O R. Diagram form. This is the and it's for remembering Lear and it Priest. Our proposal esteem. Live images given to the fully connected Lear for are putting a value between zero and one . So we never for the lessons. We will start looking into detail these lives. For example, we will look at in bed England. We will see what our word emerging Zwart are different ways off are doing word embedding or calculating the embedding victor. And then we will look at what is ill esteem. I wonder how it works. What are the alternatives off a list in? And then we will start predicting using these modern. So first we will train it and then don't make the predictions 10. Word Embeddings: in this. Listen, we will start over discussing onward representation or word in buildings for any NLP task of we need the words. The representation of words on RN in units will need dude word representation zor feature vectors. So the way our and in units work and I am growing a generalized out in unit this can be an analyst in just this can be Jerry or simple other than units. So this he'd a word. So any sentence may have multiple words like the blue one. The blue, too. Let's adjust forwards for simplicity and these are for two different or in in units. But we don't feed that to Lord. Instead, we feed Victor, which represents these words, and this victor sage will be fixed for each of doors on the first unit will predict some more what you want. This the blue one is the X one, and the second will predict y two. But it will not just use the word, too, because you're sequencing is important in most of the NLP tasks, or the order in which words occur makes a huge difference. So this first unit will or personally pass and activism. Similarly this will pass that person to the next unit and using this activism from the previous unit, sort of low, trivial used that there is an honorable to on the X tree to make the prediction wait three . So we will start our discussion on hope to represent this these words because there are many ways off representing this. So the first and the foremost a step in any word representation is to come up with vocabulary or dictionary. So you have to limit that this is their dictionary, from which all the words are drived. If the word goes out of that, we will use a special symbol called you in Cake or Unknown Word for that. Also, let's say we have defined a dictionary and it can. It need not re dictionary off any particular language like English off, for instance, in English language, we may have a and burned and ability, so this will occur at reason. One dessert too three and somewhere in the middle of your name, the nettle and in there and let Sylvia and our dictionary SAGES intelligent, which is very small from modern standards. Let's see, this narrow year 6000 so This is the Dictionary of English language. Let's It is a brief small version off that. But in some other application, for example, we want Toe are do some sentiment analysis on movie abuse. So in that case, all the reviews will come from a different set off dictionary. Not all English words we use there. So based donors. Some data set. We can build our own dictionary, which will be used for movie review. Test it and they're the words may be totally different. For example, um, age people may be using amazing movie Amazing Acting. So maybe it has amazed word. And there are lots of reviews a warhead reporters where he comes from there, then Lord of the Rings, who Lord will come here and maybe movie some there don't the importer then drinks on their intermediate words in between. So this is over or dictionary four movie abuse. Similarly, it will have some poison we can have, let's say 20 Italian words. So first step is to build a dictionary next hold to represent a word so witty each word. We know that each word as a unique position at the sentry abilities there, so no and other world leaders. It's not that I ever and then also at airports and three and ability also decrease entry. So each word has a unique property riches but index off that word in the dictionary. So this property we can use and using their the simplest thing we can come up with this one hard representation. So it means that lets ever dictionary SAGES this 10,000 and we want to represent the world narrow. Which is that a recent six Stojan Oh natural reasons. Extortion on our dictionaries. 10,000. So what we will do, we will create director of sales 10,000. They're at position six talent. We will put one and every other place. It will be Jiro and we can represent refusing little for one work and six told you so this will denote one reason six told you So we see that all these vectors similarly we want represent the world abandon What it's a representation will be like this. It's hard second reason and we will represented us war who It's demons in his 10,000 cross one similarly 10,000 cross one So this is the simplest way of representing it similarly conversion. In other words, Ah, but you see that in each of these words were just using one place and all the 9099 places just D rose. So this is a very long represent ism and very sparse represent ism, and it has some decide want digits infected. Lots of the servant is one is that it's a user to cities and those in dimension. On the second is that streets eat work independently, like the world or man. Woman King, queen Let's divvy up these four words on early to be up some off. Other completely different thing like that one or any other throop. So we see that disciple or orange water banana. These are somewhat similar has compared to man women, King Queen bees are some people and the year fruits. So the representation of these should be somewhat similar because, let's say we have, um sen. Ben scored. I want to drink manana buried another sentence. I want to drink mangle Felicity networkers worked on ah, this sentence and it had pawned somewhere sick. But the network has not encounter descendants mango. But if oh use one heart represent is and then each word is orthogonal toe every other word because is retake dark product off any to world. It will be deal because none other two words will have one at the same place. So if we do element wise multiplication, it will always be Jiro. So it treats all the words equally So in this case, are predicting a mango king is equally likely as mango sick because each word is treated, say Marie. So we see that in using one hard representation, though some algorithms learning algorithms not generalize well saved algorithm learns on this. It will not. It's not likely to General Asian predict that language very similar to banana sweet but on a sec occurs in here. Then mangos can also occur in peer, so we need more Better represent is and were instead of representing just the index representative works by dear features. So we weren't feature rise to presenteeism here. Also 20 future. The feature was that we had one feature. Would you send off word in the dictionary, but there this not where is relevant. We need some better features. For example, let's say these are the six words man, woman, king, Queen, banana mango and I have returned the postseason. Imagine the people using these words in our experience place 10 Pozen Ah, no be Specify some features. Let's redefine hundreds of features. So gender. Let's a man or it soon so woman can be minus one and we learn Oh, for other home were total on debt suits 0.98 So you see that gender? It's very close to man the king and similarly Queen and women's gender will be very crude. Let's seize its minus do you 2.964 banana and mango. These did not do not make any sense so we can keep it close to zero. Andhra oil men is not rule similarly woman virus king and queen off royal so it will give a higher reality school And in case off fruits, these don't make any sense. So it will re close to zero is a man a oh can be off in years but king and queen are generally owes Ah, or little kids earlier there Prince and Princess. Then they become king. So maybe we will assign some high value to it. Um, again some Jiro value to these for food man is no different Woman is not the fruit. Similarly, King and queen are not fruits but we will assign a very high value to these and ah similar group of a representation of one heart we can use emitting that he's before you wonder. Similarly, this can be represented. No notice and used can be you 95 under you've had 2 100 You 7000 he told wondered, and e 5600 And on these are 100 features. So it will be 100 Gross one No, you see of Yeah ah moved from one heart to print isn't toe in more features Radio presenteeism So one receded the better sides for each word has certain from 10,000 the book ability size We're just 100 feature and all of these values have some of valuing them all All the components of these directors have some feature future value So this is not dispersed representation. It's a condensed representation On another thing is that here Each word was treated separately So the world Ah, Epple and man is equally similar at the world man and woman. But here you see that if you compare But I man mangle the victor of these. We have not listed on the 100 features. But even with comparing just four features you see there, these two vectors are very similar. If you compare but on a bit of mango, it will be very closed. Oh, very similar on if you compare, but none of it, man. You see that it's different there. Here it's 0.97 Here it's Jiro here it's one hair, it's 0.1 But if you compare oh, man and King, these will be similar on many features, different on some features. Similarly, woman and queen would be also similar. Oh, now if you write the same example that is, I want to drink Banana Onda and sick on Dhere. I want to drink mango sick so the network will see that What is the world that this crew just to manana? So it will find that those feature vector been embedding off this world. Mango is very similar to banana. So if it has to predict, I want to drink mangle and it has not seen mango early. But still it had seen Monan earlier. They're sick, comes after banana, so it will find the word very similar to banana. Very similar to men Go. It will find banana and we'll see that banana comes more often etc. So it will try to predict that recent Phil's sake here and are using this representation. You can think that Ah, if we right your men minus you asking it will be very similar to your full woman minus youth queen. So you are just are doing some vector or prison on this. So are there are different reserve for or bring this feature I did represent ism. I have been talking word a generalized from book where we want a futurist representation of the world. I have not hit any particular algorithm for this. We will see different ridge off achieving this on da then once we have ah featurettes represent is, um we can visualize it. So we have 100 dimensional vector. We can ah represented in two dimensions using an algorithm court t sneak for regulation purpose. And then you can easily find out if King Lazier and men lazier, Queen Lazier, woman lazier, then flukes big banana and mango. This will come somewhere here and some other works. The looker somewhere else. So these will occur close to each other. These will occur close to each other. And Ah, if you subject men from King, this will be the victor. Similarly, if you subtract Queen from women, this will be the victor. So you see that man may nous King Victor is very similar to woman minus queen. So later we'll see or different ways of wording meetings. 11. One hot encoding Keras: continuing our discussion on word mornings, we will look at how to do one heart in cording in Paris. So our ultimate goal would be to calculate the emitting metrics. So if this is our vocabulary and let's see it say just impose int And this word here police and one Cree six games 6001 and lets it disease. There isn't 1000 so we need a embedding off. Eat a word so the embedding metrics will be Let's say we want to represent to eat a word with 100 Dame is not Victor Then this matrix will be 100 gross in present on this day will come here then abandon and finally you who it will have 100 components in really this will have 100 Each word will have 100 features so before that of in order for calculating the mating metrics also has one embedding live which will give you the embedding metrics but it needs integral represent is enough words. It will not take the words and give you the inbreeding metrics. First you need to convert each work to some In teacher, for example, there are some in built, it asserts in Caracas one of the popular is I am baby movie abused their trust. So if you lured their data set and try to print all the values you will see there each review We love their lives like this in flight toes and flame. Don't be flame. It will start from one. This is this physical character and so on. Then another rebuke. So you will get a list of this. Let's say 25,000 reviews and each tribute is represented like inlay stuff numbers. And these numbers are nothing Good words in I am Debbie does it? This is the hope you currently a word or cursing the vulnerability of this I am digital. So this 10 this word occurs much more frequently. And then the next word because it speaking it occurs there, Drink 5000. And you took us that strength. And so this is just one veils, including your words. Another way is to use readymade broken function off grass called one heart and what this one heart function does. Ah, good words. Oh, it calculates the has off each word and us specify the vocabulary size. In this case, it's 10,000. So if you remember how passing works. Oh, it has to be loved and maximum index. So all the house keys will map to that. So in this case, from 0 to 9999 and each of the world will be map toe anyone value between 0 to 9999 And in Cara's there can be re petitions because these are just has values and not the actual positioning. The vocabulary so just briefly kept off one heart representation, for example, in our vocabulary, let's in name occurs that we since explosions and our vocabulary SAGES 10-K then one hard to present is in off the world will be off state imposing gross one and all the positions will be Jiro except one at the reason 6000 in this case for the work name. So this loop is Internet six Teligent. This denotes the word name in one heart form one hard representation, no wound. You hear us to a calculate the one or representation. So first we need to import the one heart function from Cairo's airport. Three processing north fixed import one hope. And then we will define no comments in an l Beatem. Ah, so I was telling that in I am David it is it. It's review is represented as a list off ah, numbers. Which numbers denote words, and we call each detested each day tire. Each details is reviewing this case movie review. We call it a document disease tendered from that he used across all NLP applications. So, no, we will define our no comments glass off orange juice, more close mangle use, so I will not defined too many heavily fined. 67 Send Pence's or documents. - Let's to find one month men and women. So we have defined our conduct mints. Next, what we will do. We will define our vocabulary, sage, because in order to define a has to find the husky, you need to have a range within which you have to have mapped the house off. That so we have Sage will help there, so none of the Indus is should go beyond that. Each word will have a value less than this hookups age. And in our case, it's 10,000 and Mex document land we can define, which we will do later when we calculate the actual imbedding metrics. So right now we're just calculating the ah one or two presenteeism. So, no, we will elkridge war north for each document so clearly will mean one create freeze descendants this entrance. So we will treat for older documents. And then we will pass the document and woke up sage. And this will help create the corresponding one order Print isn't it Will not print 10,000 gross one for each word. It will just assign value. For example, six talent would mean it's does Ah oh, every indexing directive Intelligent Cross one. It's Jiro, except one at reasons. Excellent. So now let's print the in quarter docks. So this gives us the same list of documents, but no, as a one heart representation. So here you see there 20 to 9 off he's the second word and here also it's the second word. So it maps to the same Lucas. Um, our mortal values can go into the same cell using asking. But ah, the same values would always mapped the same key. So that is mandatory. So here, although offs should go toe to toe to name similarly, you can see the second index for all of these 1st 3 documents is to Europe remain, which denotes off. So this way we can calculate one heart representation off words. Interros 12. Embedding as Layer in Keras: in this Listen, we will see how you learned the embedding Zoff forward, and in particular, we will be focusing on one of the approaches off emerging, which is invading as the first layer and which will be fair to further layers like ls Tim Markle units off Philistines. So first, let's see what he's emerging. So emerging Third methods for learning that two representations off categorical data. In our earlier listens, we had seen one heart representation off words. So given a word, it was converted to a categorical later which is this form So district er was in over kissed 10,000 cross one with just one value on dressed. Others do so This is the categorical editor in this case. One hard representation on this categorical later is for duke embedding live, and this gives word emerging and there are different approaches off learning it or there can be some approaches like words to it were our earlier discussion or for hope King and King is to queen work men is to woman. So we had subjected the a trade representation off king from Queen, and we're got some results and future Victor and seemed relieved. We subtract the men but to from women. Then we get another and these two will be very similar. So this kind of religions, it's horse true when be I learned the word embedded Using or approaches like work toe on allies do test it on large number off words on a large vehicle vocabulary often language. So here, this kind off patron will hold group. So if orange juice is there and then mango, you see there. So in this kind of training, did you know spinoff context toward and target word so are given issue works within a word will belong to this context or not This kind off thing or will be achieved using were too weak. So there will be some 60 per term off the representation If we know the representation off Oh orange. We will know that for mango it will be very similar. But there is another approach. Start this embedding live it just in leer and we are creating buildings on the flight. While training here you create embedding metrics, which is the global community sage and then the number of teachers in each director. So this imbedding metrics me train and conclude but other approaches using embedding as it first layer. So feed it the categorical data or one heart. And this one referred to a list iam Lears. And then it will do the predicts and, for example, sentiment, analysis or any other application. Also, NLP So we will focus on this approach where we will create in buildings on the flight and these in buildings will keep one more refined with time. So this is just an are put off from our previous will. You were weird. Created one hard representation off all the documents and their 215 means Gerald Gerald you and one on Jiro. And this will be a place yet to one thing. So this we will Frito over accordingly. So how it functions? Uh, let's say we want to do sentiment classifications and we have some A list IAM units. And instead of feeding the world, the police said though inbreeding so awards you open flame Judah point who flavor and so on . So this is word embedding similarly second words inbreeding you doubloon a doubloon cwt you the beautiful and these come from embedding there. So this embedding live ex urgent one heart in cording, and it feeds embedding Victor. And based on this, it will give some critics, um, waiver. And it will urges newly freed activation off this unit. And based on this and this, these twin puts it will. But the way to similarly y three and you're a fool. And let's say we want to do sentiment classifications in that kiss, they will pick all these fools or boots and give it to it fully connected, live and using a sigmoid activism. It will give a little bit with Giron well, and then we can based on the value. If it's more than Europe inside, you will say that it's a positive sentiment of the voice, the negative sentiment. So there you see that embedding is just the first live in the network, so any silly, let's say this word embedding was some value, and based on some supervised setting, we will give some documents as well as the corresponding sentiment. So even every poke these imaginings will get modified. So these in Beijing's air not learned beforehand. But these are created on the flight during training, and here you may find that the embedding off orange and mangle may not related at all. That's its job minus 0.95 Here, the first component is plus Joe. Point it your 0.750 point 01 So these may not be related. And those kind off vector or prisons which we can do it were too weak may not hold true in this case because here's the purpose of this is not to learn them buildings of the language but to get doomed for a specific purpose. In this case, sentiment analysis so here that cannot be traced or prison will not work. Let's see how toe create the first layer off embedding oh, in Cairo's so first ribbon or paired sequences. You see that all the documents are of different length. So here, the first thing we will do is that we will make all of them explained. So if it's more, we will trim them. If Rich list, we will hurt some values. So for that we will need re processing are sequences and we will also create a sequence immortal. Here we will import the embedding leaders, and they will also using number. And now let's defined the length off him breeding so we will keep it, some of them evil. You. Let's say we in court all the words and we're better off size for you Save Grossman. And here the document lens are four. Maximum is 2467 So let's keep them excellent. I asked him. We already have this in quarter docks, which is this categorical depression Decent or works. So we will paired or dream the words truncating takes values pre or Khost whether you want toe during the values from beginning or end. So we will trim it from the end and starting We will also compared to the and and then on work, Lind Max, Look do then. So this would return the murdered documents but its strength and see Or it looks so seize bigger the same things. You do it to 15 to your pool. 91787 So four values are there and then six g load are there because we have made all the document length Exactly 10. So different Number of Geos will report it so busy over off, deterred in quartered hook. No, what we will do We will create our morning which will be a secret until mortal because it has no little branching. So we will keep it secret still. And then we will lead embedding leaders. And here we need to satisfy the workability size and they had defined woke up, say Justin present then remembered England which is slaving this kiss. And in Portland, tickle do been which is smacks in Oakland. No, we will complain the more than I mean it's coordinators and no well Oh, see, What is the embedding off each of these words? So we will or put finally rude. So these are there different in buildings, off documents. So you will see that Ah, we're 123 full 567 So for all the seven words, all the seven documents we have emerging and each document has 10 words because we had limited us a just documents and you may notice that in the in, then beat six Will user seem because these their own zeros departed values. So dear, embedding all embedding victory. Also same for all of these. This one broke respond to one word in a document. So this is the first a document which is glass off or in youth so This isn't glass embedding off glass decision burning off walls busy one for orange and this Rocca's forced to juice and rest roller Jiro, similarly off, was in Ah, second word also glass off mortar loved mango juice so often you should be same. That it's second and fourth should be, say, important both of these. So let's compare. So this second drew and this again Drew. It seemed similarly for true, with his juice is come common towboats so dearly. Meetings are also same. So this is Oh, you can add emerging layer in the beginning off your tent Work here are just herded one do But this will be on Lee the first layer in the network and you cannot add it towards the end And these values keep updating in different epochs depending on the application 13. LSTM layer in Keras - understanding Input and Output shapes: in this video, we will, oh, explore the esteem, leer in detail on how to use a list him in Cara's. And most importantly, we will explore. What are they input and or puts apes to give way? Ls Tim Layer in Kira's Most people are, even after understanding the to reveal finalist teams, the struggle to understand the ah actual picture. What is the input to the list him and what did our puts it? So let's explore that. So we have ah, in our earlier really well looking at imbedding Lee were devised a small mortal sequence. Immortal Air didn't embedding Lear and were printed its or to understand how this works. No, we will add one additional. Ls Tim input Klier? But before that, let's see the sips so forced. Let's look out in Group C. So ah, religion, ozonoff Bad Saiz who are dealing with training deep neural networks. So what that means is that for example, let's say we're working with I am D B movie beauty to sit So this love training set off 25,000 tributes and each three bugler words w wonderful little and we're also seen there. Generally we limit the number off words we make all the reviews off fixed length. Each of you will know help it, the words this number can very depending on your requirement. So it's the Children you had more than it be. We will trim that if it's less than it really well, Baird of it fixed values. So this one review is one day and of then we yard draining your network. We need trespass, faith, bad sides. That means we will Let's say that Sertys 32. Then we will take 32 reviews and process them in one batch and then operator who? Different weights. So now you understand them that say it's No. Let's understand. Input safe off ls timelier on. By the way, the way to Adil s. Tim Leary. Simply whatever model you have created just aired than a list Human caps. Then this unit is the number off output units and they will come through this shortly. First, let's see the in group size input safe. So we have Let's a three these blocks. So the judge that is, in this case, three wretches so bad Sage is one of the import perimeters, then. Oh, yeah, the reason along Let's centers, it's review B A. We see that each of you has some words. So one division is this message one dimension. Then the second dimension is we go one layer deep on Rebic One tribute and within their tribute via it works. So this is the it in this case we call it aims steps because here we're dealing with C conciliator and it hasn ozonoff sequence. When we speak a review, that said, Then we will speak the first word, then second word So they're juicy Consensus theater. So this will be a time t one t two and so on. So in this case, this is Let's see it be the same steps Then this Lind we will call in program itself or the length off in port sequence. So each word ah, regionally used embedding leer as the first layer while dealing with the conciliator and this inbreeding Lorcan words every word. So we have the blue end of little poodle blue tree nobility in a review. These are in order to buy index and so on, so it converts the numbers into director imbedding rector offer given land. Oh, in our case Let Syria it Victor has emerging off 100 to one in Texas, converted to irritate offsite 100. So this will be no that, will you? Let's say 100. So no, If we have a less steam in propio saves or three trees the birds sage notice it will process treat abuse. Then it be gross. 100 100 is the embedding limp. So this is the no we have understood in group Let's see the order food safety. So, while creating the earlier were dissatisfied this units perimeter on. This is mandatory. There are some other perimeters also, but this is mandatory with this already notes that number off or put units So this will or group in this case 64. So it will or 64 values? No, we're giving three batches a bad side of trees tribute so it will have three cross 64 values. So 64 values And for each attribute there will be 64 value or Lear for each of you beard it across 100 values it words and 100 for each word emerging Richter and it converts it across 100 to 64 and it will do for each tributes on that same remains same. So know that we understand the importance or put saves to realist him. Ellis, Tim Lear. No, let's eat another court. So this was ever earlier court. So we had taken a dummy example with seven reviews, it has different length earned. The we can work to eat 21 heart representation. Then we paired it with extent off in so each document along her length off Ben and in breeding Victor SAGES fight so each word will really noted way of it off safely. So now you see, after parting each of these seven reviews have or linchpin, were better Jews here, depending on the sorted. And then we activated a model. And when embedding Lee So this really make all of these ah abuse and convert each of these numbers Director of stage fright Because we have defined you made England for you and we specifically all of these parameters here. So the safeties, seven crossed and cross VIPs or seven reviews. So for each review, we will have this. So this is the first review and ah, this is the first toward for you limp and we will have in such words ET We will act in such words. So in this weird understood how in reading list gun words, words to imbedding Rector's No. Based on our understanding off a list IAM which we just saw, we will aired one more layer after embedding, which is unless steam on Mac Syria imported l esteem here and we will specific unity Quito Oh, what we saw in the here we saw six people. So we will keep not number only 64. So what should be the airport? No, let's Forster try ourselves. Then we will see their or proof. So we have currently seven reviews. So seven cross or in is the woodland and each word has each award will be presented with a trough land five So what a Lestienne bill are 64 units we ever specified. So for a tribute will or put 64 values even the to the 64 we have 7 30 reviews, so seven. So it should be seven gross 64. So seven gross. Tim Groce Facon works to seven Cross 64. Now let's see it in our court whether it mattered with our understanding or north. So there are good safety. Seven. Cross 64. This is the Forst Review, and it will have 64 values. This ends here, by the way. You see recruiting races and we have 1234 So 64 mins. 16 Drugs. 123456789 10 11 12 13 14 15 16 16 Gross four years. 64 This is the or put off restored and further what we will do you lard or fully connected earlier air dense layer, which will take input from the 64 values and predict one or so. I hope you find this video useful, and it's generally not given in their documentation off Chara's. And it's very many people use the less teams, but they don't understand what staying put, say safe and works the or could save, and this will help you in building complex model 14. Training the model: now that we have seen the individual components off our modern and we have also seen what would be the sleep off in ports and airports with those layers. Let's put all of those things together and the start training the mortal. So we're already or defined over model in some of our earlier reduce. We created this sequence in modern and I have already explained why is he considered selfies here? Andan? We added one embedding layer and this layer originally started as the first layer which will give the words in the form off imbedding victors. And this will refer to LS timelier. And here we have specified this or put units which you saw in our previous listen and finally Caesar going toe fully correct earlier, who lets under standard. So we have ah, and nothing on bad state and my bad side. You mean Khomeni abused to process in one match? So one review has it the words the blue end of little ability. Similarly, it 32 off those our bad SAGES 32 in over keys. So if you wonder Butoh all the way up to give you 32 and this will reset inbreeding live and then, ah, December tingly herbal or what it will do. It will just convert the words which are just numbers. One would be in sworn each word director, and our imbedding leant will be 64. Let's the need to index will return water to Director of Side 64 to here or produce input was 30 to cross. He, because it's review, has a length off it. And earlier it was different, were very persistent, converted to it. So this is in group on. This gun works to 32 gross. It gross this length 64 and this is for two A list IAM and working a list in blue We ever specified 64 or probe units. So for you attribute it will give a vector off 64 numbers. So when reviews can work it through deeds, 64 values even re too. The three the 64 and this will return for all the 32 rib using the badge. So 30 to cross 64. And for each of these dune I have one more layer called densely or fully connected lier with sigmoid activation and all of these 64 includes, We'll go toe this final Live with a single Lord and this signal it function always gives have eluded me. Zero Anyone because those sigmoid songs only defined 1/1 plus years to the par minus six and it's graft movies X zero. It will be one so 1/1 president wearing sleep. If it's a very large number, then this becomes zero. Here's to the part minus and 30 so one over us to love our infinity, the Children zero. So it will become one and minus infinity. It will become the stumble become very large And it will come geo. So this sigmoid already gives civil libertarian Jiro and one and raised on the trestle if any values more than going faith Revilla Jim, is it to be or you do sentiment otherwise Negative sentiment? No, we have understood the complete picture. So let's complete the training off our mortal. So we have already oh imported the deficit on deliveries and then we pre process and limit the sides off all the rib utility. Then we had already been to the mortar With these three layers Onda, we can visualize the morning so where you can see the important or pokes apes off different layers Here. None means the bad state because we are feeding the we have the flexibility to change the bad stage during training process. At runtime, you also can six to a bad state by providing input. Sager's that inputs aid and we have to provide No. But also here. In that case, we will have to stick to that. That said, What if we don't disperse? Five. We have the freedom to change the bad side. That's what Bettman comes. None appears here here, the bad stages. Now let's begin their training off our modern. So what we will do. Let's also print a message training, and then let's define our bad stage 32. Then, in order to train, we will call just mortal or feet and it boxes How many times you want to. This model toe was through your interrogator. So when you poke means this, more than less are going to all of the rib using this notice it, which is 20 factors in tribute. Then it could have changed its rates, and then again it will pass through the same did is it, and I didn't keep on mortifying years. So let's say we want it for any books. You can try it for more reports and then compared or if there is an improvement invalidates and it'll be X test in weight ist. So that's all. No. One. We're on it. It will drone start training and it will, or on 14 and books. So just look at first IPO. Then we will get back when this complete so you can see a poker one off them. So So step. Okay. Started train on 24 Italian samples. Validate on on divide Italian samples. And you would also see the progress We're here. So this is it started and it will complete four or 25,000. And you see here the loss is decreasing where they could conceivably increasing this will be The general trend is by any means, losses starts to increase. I have seen sometimes our loss starts to increase. And that means that oh, you're not using appropriate loss function or you need to make some adjustment to your hyper perimeters. But in this case, it's a group saying that loss is decreasing. There's a very to plot also how the training vent, you can see Ah, the loss and accuracy graph over thing or this model or if it will return a value history. Oh, which you can capture any variable so you can explore under. So no, I think the first book has completed and the second a poker has started. So let's rate for two complete. Then I will get back and no or nflx have completed. And you can see this law says decreased. Their Lord Onda Chrissy is increased a lot and no, we have three, never modern No. We can use this train model to evaluate it and then finally make predictions on new data. Certain notice we will give new rib use on this model will predict with it it is it a positive review or a negative review and also the school. So you to the school, the more positive it should be so great for these 15. Evaluate the model: that we have trained of immortal. We are ready to devalue the accuracy off modern. How will the more the last trained? And it's very simple in kilos. You need to just call more than Lord, value it on whatever model you have defined in trained, and you need to positive X test rightist that just status it on which you want to value it . Full received in the court. So immortalise contained in this very well mortal on our bad SAGES. 32. So it will run for older 25,000 or abusing the tests. It Onda receive their just to score ease. 28 it on Chris's point it one. So in the next lesson, we're even make predictions um, neuter and there will read a fun part. 16. Making Predictions on new Reviews: when we were ready to make predictions on neuter. So let's begin. So let's write in new review and, uh, let's call it Oh, it was a great movie. So clearly it looks for you to notes also contribute No what we need to do. We need to convert this to or in Peter's, because by embedding, Lee expects in Peter Sweet toward with represented way hope recurrently that occurs So we will offset the World index. List the map from IMDB, which we saw earlier. Oh, when we were recording the review. So we will again fit in that I am Devi North Good word index aan den. We need to state the word so we will use the split function so it will split the world into tokens. The sentence in tow, tokens that each individual awards and it's create. And in Pierre e were we will enter numbers crisp morning toe. These words denoting are frequently there are currently I am DVD. It is it That's fuller Tsai treat for on the words in this token list off tokens. If word North indie noted, it has not, according name David a tous it. Then we will use the Hui character, which we had seen or earlier, which is by default to and we did not attain that. And all of these transfer or provoke ability. So is any unknown workers everyone hardcore that with six number in this case, too. So actual numbering starts from three were one denotes the start index to denotes the or provoke ability and you ease reserved for burning. So there. This will also vocabulary other ways. E If it's present, then said the the no, we will hurt miss number Andi afford whatever the index plus three because actually ranking is removed. So most frequent work or Cozart drank three and north you. So once we have created distributor's also printed to have a feel off. What is the review? No, our model was trained toe on reviews off sites. It is so easy. Tribute a fix lint off. So we need to do the same for our best case. Also for making predictions on new data for dream from the beginning, all to pared to the beginning. In this case, it will not required. These are much less than it. Peace Only parting will have happened and Max Lynn cause it be? No, we're ready to make the prediction using more than Lord predict. And this is a trained model, and it's also print what it has printed. So you're Jiro means good to on one means or you to on people also right critics in schools or abuse, nor define new review again. Another PayPal split. So this is the review so it can work it toe this aan den. It used to score off Newborn 67 so this seems to report it to No, let's tend to reboot the move. Iwas a great wrist off train. So now it should be *** do on Let's certainly to again. And now you see, the school is very less 0.4 that is very close to zero. So it's able to distinguish between towards a great movie you give 42 and the same. We used a great word again, but we write the movie was a great waste of time. Then it gives negative sentiment so it doesn't it's here. Job off this lead. Willa, get confused on some non tribunal sentences. You can try you in there on also play with their different hyper parameters and primitives and CEO, you can improve the accuracy. So I hope you enjoyed the whole process off training the Morgan and making predictions on nudity. Is it? It would be fun to drive it your one reviews and let's see if you can fool the model or not , so all the best for that. 17. Correction of previous lecture: this really use for a minor correction that we made in our previous listen? Or where we were making predictions on a new order. Dessert. So what was data? So this is the court sleeper on here. You see, that of its he'd in new review is er, custom review. And we have already trained the mortal and the split doored distributing towards and then for each order, we tried to find the word in the dictionary off. I am leaving. So this is the word in next. So it consists off word transforming UNIX. Are you on, Luc? Ooh, I to And it consists this dictionary and here by default, all the words are in lower case. Oh, you must have also noticed in ah, the organized electoral beer we were organizing on our custom reviews. So there you have gloves off, making lower case, true or false? By default, it's true. And then I am DVD. It is it Also, the words are in lower case. So the error is that Ah, you can't just, uh, look for the word in the word in next six scenario. If I am, do you have to first convert this word to lower case otherwise, what will happen that it's by any means? Let's a distributes completely upper case. Then none of the words will be formed in this dictionary and you will get only two there. So to denote the auto cavalry characters. And there may be cases that a lot off words are kept late capitalized. It's not lots of them then at least at the dinning, off a sentence. So for doing and necessarily you're losing or turn towards door, you're just go to here so that we will make the corrects in here in the court. So here we obtain our modern and we're really to make a prediction on our customer. Does it? So see if you do there or but they're not too much off. Ah, Topo Cavalier. Only this does is kept a little, and use can see two here somehow. This time I think it's due to this door. Tish, it's looking for time Dordt! And now it's not toe only. Tease there and you also see the chorus increase increasing. So decision negative review on If you made it nort so it should be as close to zero as possible here, Studio going 037 or close to 0.4 So we make it this and we also make the Corixa notice were recorded too. Or nor Karush. No distress would also be gone because it's e it should be able to find so you can see you're going on. But it has become 0.24 from 0.37 so it has become closer to zero. Ah, let's try on this and coming toward this part and the score is 0.29 You see, want to hear So nice is a strong word. It should not be ignored. It's three. Make it Lord and run it again. You see the crazies? 0.644 No, it is a nice moving by just this change and I am not just thereto. If I trained the mortal different times, we get these values keep on changing so one region can be their toe. In between are bad surges 32 so and training centres 25,000 reviews between February 20 Italian By 32 you get roughly seven inch to 71 point something. So in each epoch it's sticking 7 82 steps. So each step in each step Oh, it will process 32 reviews. So after processing 30 to reduce, it will take one step. So in total 72 times 32 Easter, roughly 25,000. So in each Ah, a poke. It may be taking ah picking the it abused randomly and that may result in or different rates off the model. So for here, it's giving your 0.644 So in next to you, we will play with our model for them. We will try to make our model DePorter and we will add a few more layers. Ah, more Ah ls Tim Lear to our mortal. And then we'll compare what is De Crecy? 18. Making our model Deep: our sentiment analysis model consisted off and embedding Lear a list in leer followed by air dense layer. But sometimes these models may not be enough leisure Very sell old networks being like to make our modernity. So in this listen bill to see how we can add multiple esteem leers and what does this actually mean? How can you visualize that writing the court really very simple? You will just do more alert head and the real esteem in Kira's. But it's very important to understand what's going on underneath in that. So let's have a brief recap off our mortal. So this word over off there Morgan of the third Ah, a sealant rib use into our embedding Lear and it returned in bed. England in our case was 64 for each word. So for each word, it return Director of 64 and it was fed Brill s team leader and in a list in blue. Since we had it your words What will happen? Oh, is that we have Let's see here first we have a list in layers. Let's android fris. So we have embedding Lear somewhere here and we have so what people are in in our disease. Here we feed the first word. It will give some more proof and activism is passed to the next next Saarinen unit. In our case, it's ill esteem. Similarly, this will take the second word. This is the blue one. This is a little too And this is wavin way to and so on. And here we will have some esteem Number E or were Dulue ity. So this is coming from our fixed land reviews and finally it will be giving or put off times to it. So in some in Olympic tasks, you would like our put off all of these. For example, you're translating from one language to another language. You mean like to have all of these? But in our case, or we did not require that we just needed the final or so. If you see it, there are produce this in order to decide this way to this way to depends not just on the blue too. But you see it is not in court. Activists in off mostly coming here, First unit coming here. So this way to depends on why even as well So it has already accounted for the blue one. Similarly, the blue tree takes activation from this devoto, which in turn has accounted for the blue one. So given our depends on the previous values. So when we're looking at this hour, it has already accounted for previous unit or previous words. And we were just fitting. Ah, this to a densely connected lier. And this was if you remember, it was a vector off 64 for each word, because we had given our units equal to 64. So let me show you the cord. Ah, here was the court and we had provided units equal to 64. And this is different from this emerging length, which is the Devin turn off each ward. And here you saw that this is yours. The bad states we can skip for no. Eso 64 is coming from this unit 64 for each word. It will give 64. And here we're seeing just one. Or put wise that, um Because were made this return sequences equal to false. So what does that mean? So that means exactly same thing what we have seen here. So here we're not interested in now. our boots from. Why've until way 79 were just interested in this sort book. And this is itself a 64 dimensional director which is connected to it. Fully connected layers. These values and all these 64 values come here and it gives one or one value between zero and one using the sigmoid function, which I am been repeating multiple times. So this will give any between John one. And depending on threshold, we will return zero or one. And this is controlling returns sequences perimeter incurious. We had said to fall so these we're not to return. Just last one will be litem. No, let's update our model. So what do we do? So we aired in Newell Esteem, Live. And this is the new layer you can see And over here at certain return sequences equal to false. Due to that, it was just returning for the last word at the end off Let him do no hope can be aired. Another arginine let another real esteem leer. So this was the first layer. It has many other nen units and eso First of all, we need these values all level use. So for that we will make return sequences equals true. After that, we will get all of these. Then we will aired another ls Tim live. So what will happen? This will go as input and again the same thing. It will have multiple orchids. And if this is the last tell Esteem Lear we can again make return sequences equal to true Here, if this is the last less temblor, If you intend toward moral esteem Lears then use old Mick returns to France's equal to True for this one as well. So we will make it falls on then you to win. If we keep a number of units to 64 then it will return a 64 Lin director for each of these 80 in esteem units and these 64 will know referred to on this men's lives earlier. One will not referred and it will work again this sanely. Let's see the Corp Oh, so here we have just one list immune it first we will make it true and here you can see the same It's, uh it's just returning 64 visitor or put off last idiot unit. So we made it true, So it will return. No, it devalues. Then we will do more than Lord aired a list. Team units equal to you can change it. Also, you can make it 32 or any other value. 64. Just ah, matting to embed England. That's not required. But let's keep it 64 and we will not add any for the perimeter by the fort returns he considers false. Only so this work and we will see how the shape changes here. So you should look at the staple once again and no, let's around for the new mortar. So this is the new model you see here. It was just returning this thing 64 this is the bad side. That's it can be one also for you. Can a gym offer it for one rebuke for simplicity? So it was returning just 64 notes is turning your DeGross 64. So for each word, it's returning one. Or which was same as this embedding Lear or put. So this was coming to this. No, this is just a personal esteem. Lear, which is taking a ticker or 64 returning it to grow 64 and this has no impact, no on this it gets 80 64 return 60 whole. The 64 well user fed too weird, Fully connected Nord be to will combine the 64 values in some Richard Value like the blue one value on the Maluku value to Till the blue 64 value 64 for each of the village off the 64 demonstrable victor. And these rates will also we learned during training. And then finally, though Richard, some very well referred to sigmoid function which will give a value between zero and one. So on model has been updated. Lett's run it again with 32 bad sides and 10 approx. So first of all, we're started. I will come back when completes, then it books. So no, we're in the tender book You may have started toe over Fit our data. Well, sort of the sort. We see that if that is the case, let's take our Mr School and accuracy The mystic Chrissy's a concern earlier I think it was more than you For 35% we will see their learning tone distribute. It gives 0.99 almost close to one. It might be the case off. We're fitting off later, since we have ordered one more A list. Imlay. So this sort of you some negative value, but it still gives 0.94 so looks like the model has over frittered. So it's Trey. One more thing. We will reduce the number of books. Let's train it for five year books. Onder, if you're running it on, will call of involvement. Make sure that oh, you re start during pain. Otherwise, what will happen if you restart training? It will not start training from the scratch, but whatever mortally test trained, it will start training on top off from their details. So you should really start. You're in pain and it's showing your initializing, and then you need to Ah, do all of this again. Let's make one Morton year in the last less Tim Leary. We will make it 32 on and a knowledge start training, so I will come back when it completes fire books so it has completed five approx. No, let's sick. The just school and secrecy slightly better than last thing. Let's straight oak. Oh, it's 00.7 lane, and this is a party to review. Look, see this one. This is again poised to It should be even more questions and 10.9 flight. Then disturb. Eat. Oh, it was nice, but not working. Maybe between 50 and 60. Somewhat neutral. We're in six main. Okay, so not that was, too. This would be a really posted in its 0.81 That's would, in order to take building negative case, the movie was a great waste of time. Energy gives 0.0 Toto Mrs Expected So it's giving good region for all of this. Maybe for this one. It gave points explain which could have been slightly lure It may have been if it was closed 2.5, it would have been much better. But still, that's not any soup you can trade with some more changes and you can play with this mortal . The more you play with it, the more you will learn for No. I think it's giving a decent really 19. Conclusion: Congress listens on completing the course. Everything to learned in this course we saw how to solve a very fundamental NLP problem called sentiment analysis. And for this we used a very popular and easy to use framework called Kira's and we used fightin. And in particular we used a list IAM layer and embedding layers to train our model. So just a brief recap we started with the introduction to sentiment analysis. We also saw various applications off sentiment analysis. Then we had a look at the various approaches various algorithms for sentiment analysis, namely rule based automatic and hybrid approach. Then we saw the work floor said generally, sentiment analysis algorithm been way started with our own uproot. And we designed a model consisting off three layers imbedding Lear a list in there and a fully connected, dense layer. And Ogi made a bind, re or classifications. Using sigmoid activation and for our experiments, we used an inbuilt movie, d tacit movie beauty to set called I Am Leaving on. But then we looked at how toe pre processed in data where we added or truncated the reviews depending on its land. And then we saw a very important concept gold word, embedding zor word representations. And this will be used across their different NLP applications. And we saw in particular one heart in cording, and we also saw at embedding Layer that is presenting Kira's and hope to use it. Then we also saw a less team leaders understood. Or what is the staples important? Or put real esteem how we can use a list in after we have added an embedding layer weekend , or at another lyrical LST um, and then how these work together? And finally, after looking at the individual components, we put our all knowledge that we learned so far into or six and Gore putting it all together where we trained our network. We evaluated our models, and finally we met prediction on new reviews, which we give, and I hope you learned some good concepts from this course, and you cannot play most of these concepts to various other NLP applications. So thanks for being part of this wonderful journey