Web Scraping Without Coding

1. Introduction to Web Scraping: Hey, guys, welcome to the lean Web scraping course and is the first course in the Lindsey's Andi I Lost Leniency is with the aim of teaching your one specific skill in each course, which you can learn very fast with less investment of time and resource is, but it can be applied to different areas, but it's a Web scraping work. Something like that. It's It's a very specific skill, but it's very generous in the application of the sculpture. You can apply this to many different areas. Some of the areas listed here. You can go to most of these on the landing page, but that's a legion business and have been listing gathering. This is interesting because I have a client who actually scraped a lot of listings from different websites and built their own listing website. Obviously, it's not gonna be the next big thing, but it's definitely a good side project to work on email addresses, scraping email addresses. But a review is product reviews is interesting because let's say you have three competitors were also working on a similar product like yours, and you can scrape the reviews off all those products. So you can analyze the feedback that your computers are getting and how you can improve your own product based on the feedback. Social listening is a very interesting for me because I run a social media company and the way we can export garments for Facebook and tweets for a specific hashtag, for instance, or tweets from a specific user name and then see what kind of sentiments people have for this specific page or celebrity. Your handle and stuff like that. The different applications of social media sentiment, analysis and itself. Combat isn't websites you can actually create a combat isn't website. If you know how to scrape data, you can just pull out information, compare prices, compare features of products. Example you can you can create your own website, which compares the more different smartphone models and stuff like that. So yes, you're talking is another addressing one for me because I have an issue, of course, and I also teach this in the court. You can actually track the rankings off a specific you, but that you're targeting for and see how it's changing over time. So again, an application that scraping. So the good thing about Web scraping is that it's just one skill, but then you can apply it to different areas. Now the problem is that most people I don't know how good and biting into Web scraping, and we're not good as we're not programmers, some off in some market here. Some of us are entrepreneurs. Some of us are just learning I t. And just learning something new. So I've put together my I am not a definable. I've put together my ways off scraping, which does not need any coding. So that's the first problem of speaking, which I'm solving with the scores. The second problem, which a lot of students come to me and I suggest them some tools to buy for scrapings that the tools which are available in the market from observing are really, really expensive on day. Don't do specific jobs now. I'm not saying that this course will help you. A Bill bill will make you a programmer can scape anything in the world. But you can scrape anything you need it most. Most off the things you can escape without having to buy a paid to. So in this course, I'm not gonna be selling you any tool. I'm not gonna be putting any their free links for any tool because my focus is to teach you how to scrape data from websites for free, you know. So you have bought this course. You've given me your trust to buy the scores, and now you're not gonna see any products being sold. So I have a few problems that I recommend, but we're gonna be just using the free models of those products. And what I've done is I've put together all the ways that escaped different specific types of data in the scores. And I also structure the lessons so that you have application based learning. So every lesson is an application of web scraping and how you can do it. And my goal is just to expand your mind. I'm not in a spoon. Feed you however, every application a little. Did you step by step? How do do the were sleeping for that specific application? And I have all of these lessons for you, which I'm gonna take you to one by one. And from here, from this course, it's a lean core, so you can take it in one day. You can take it in two days. If you're taking anything more than that, you're probably going to slow the idea of the lean core sister. Deliver a lot of value, make you learn a new skill. And as this time it's possible without having to buy any tools. And from here on, you can. It should expand your horizon to help you apply map sleeping two different different other various other fields and applications. So Web scraping application is only limited by imagination by how much you can imagine to apply. These are only the basic few ways that the common with people used that script. So if you're ready, let's let's get started unless lets guests get into the first application off the scores. 2. Scrape Email IDs from Text: So we're story gonna go from basic applications and then keep increasing the difficulty levels as we go from the starting lessons to be concluding lessons. So this lesson number and teach you how to extract human ideas, subtext and the specified example that we're gonna use is legion ation from Facebook posts . So when I say extract immunities from Texas means you already have the immunities on a page , let's say you have a list off people with their names, and the humility is on. You have to extract those immunities and put them in a sheet. The other thing is, if you have names and company details of people, but you don't have the email ladies on the page, right? So in the first case, we're gonna just use the student I'm gonna teach you in the second case. What you're gonna do is you can still export the names and the company names of people in the sheet and then put that she'd into our email hunting to like Hunter. And that is gonna suggest you the ladies of this people. But now let's let's export. Let's see the example of regeneration from Facebook posts What I'm gonna do is just a place for dot com, and this is one of my grow tax. Also, where I exported leads for either my courses off my email lists and stuff like that, and with a search for, let's say, a distant marketing email. So the email comment is actually fixed Suffolk's, which I always use. That's the hack. And here you want to put a key word, which is related to your industry on two main and related to the search that you want to perform. Here's so here's an example break. Um, three district marketing tutorials, commenter email it in and send you the tutorials to your email. You have all of these people who keep posting stuff like this on groups you also see established the shocking agencies. Please comment. Your Babulal. It's national immunity. So if you're selling to destroy marketing agencies, right, this will be the right threat for you to explore the email ideas. So let me look at this one. So this one has won 28 immunities off people who I know are interested in learning destry marketing, and what I'm gonna do is I'm gonna export these. I want to send an email to all of these people saying that Hey, why don't you check out my back scraping course? Why don't you check on my S u of course. Or actually give them free, distant marketing materials, which is what they asked for. That would be the right approach of what I would do is I would send them off the e book, are the tutorials and say, Hey, he had feeling to smarting tutorials that we promised you on Facebook and then follow up after a while. The sales email. It's a dirty hacked, but it works really well, right? And the reason why it works so well is because this is really your target audience, because these are people who want to learn district marketing. It's not like a Facebook ad where you talking some people and you may or may not get the right audience, but this is definitely the right audience. So we're gonna go to the stool called, I bet if I email address, don't I owe and they have a specific to your email extraction called even extracted. A lot of these tools you can just for a search for email extractor from next and you will get some tools. But this is the one verifiability desert. I was the one that I use. I'm gonna put a link off this tool bit with video lesson so you can go and check out too. But essentially, what you have to do is expand all of these comments, right? So you have to click on you previous comment so that all of the comments on the same page and once you have all of the comments on the same page, you're gonna just basically copy everything. Okay? Now, this is really basic because we're not doing any automation. Your of your actually doing a manual script, it's the first lesson. So we want to keep it really simple. And then we copy everything here and just like on extract. So the job of the stool now is to extract all of these email ideas on this sheet. Right? So you have all of these emails and accept and I mean, it's it's on the sheet. I If you wandered in exile, all you have to do is click on export. Right? So I have these email ideas now which I can, which I can import in on Convert kit, which is the mail email marketing tool that I use. If you use mail chimp or something like that, that's the place you want these email ideas to go in, and then you can send them a mailer thing that heavy. You ask for the few tutorials and here at the free tutorials for you and then followed the same. But the idea is that you have just done what you've done. Here is a manual scrape off immunities from social media comments. Right and Web scraping can be managed in our companies who still use people to do Web sweeping. We're going to slowly move ahead to more advanced ways of doing it. But this is a really useful hack. And there are a lot of places on Lincoln, for example, where you see that people have posted something and ask for humanity is that recommend some Web design companies or something? I see a lot often, but people don't just come in the communities. They comment a lot of text with the latest example. They can say here you can connect me at Lalibela, so this tool is going to get all the Bush it out of the comment and just give the humility and give you the list off Pure gold off all the main ideas then you can now use for marketing and think of different ways off using this tool you can actually just keep to just use the same same string for the search. But you can keep changing the fish marketing to save bad designers, for instance, and then you should probably get some dessert off the designers. Right? So please commenter email. I need a Web designer, Commenter charges along with your email right now here, we should get a lot of immunities off off designers. So that's that's the simple. It's a simple script. It's a manuscript and it's escaped from Facebook. Social media comments. We're gonna go to a little next level with the same tool in the next next 3. Scrape Email IDs from Websites: you guys. So we're gonna go a little next level now with the with the whole scraping blast DeGeneres growth hacking thing. And this lesson, I'm gonna teach you how to extract email ideas from a webpage. So we're gonna still use the same tour, but the hack is gonna be about respecting different companies using my locations and email search. So if you haven't seen my video on this hack before, I'm gonna be searching for a specific keyword which is gonna return me the email ideas off different locations off a company which I can then use in my lead generation work for market research work or my business development sales process. So, um, you just go on google dot com and you have to search exactly this of locations and e mail. Okay, so now if you if you think like a growth hacker, you can imagine five. You're doing this because companies will make a page on the website where they less their locations. And also that email ideas we want extract all of those email. Ladies, I'm just going to click on search, So let's just take the first example for now. Now, this is a list off locations off this company, this bureau in different countries. And since we also source for and email, it's showing US results off pages, which have the word email that you would email and it has if you can see email, email lover, so many email ideas of different officers, different companies, different branches. Great. And if I really want this company as a client and I'm not really worried about which location I get a response from, I can actually send an email campaign to a lot of these dislocations. So one way of doing it is to, for example, if I want just Asia, I can copy, paste the text off for Asia and based it in the same tool in the text format and then get the email ideas extracted. Now, what need to understand is that this is, um, this can be scaled toe any any list off similarities, but sometimes you don't have the exact text to copy pair. Sometimes you want the entire list of communities on the page. So what I'm gonna do is without having to think about the varieties on this page, I just want all the immunities which are present on the speech. So I'm just gonna copy this link. So called me the link go back to the same tool. But in this case, we're gonna select webpage, okay? And I'm just gonna based this link on an click on extract that salt and this stool is now gonna extract all of the email ladies from that page. Now it's it's obviously showing me the old immunities, which I extracted from the text. Obviously you can You can restart this or figure it and excels. But now you see, the the humility is Wichita's extracted from the new website, and there are 358 email ideas that I have now with me. So I've in the last lesson, and this lesson generated 358 new leads. So this is Web scraping apply to lead generation and protagonists. I've used my grow tax to generate leads using my skill set off scraping, but this is a very basic and manual scraping process. However, I just want you to get turned on by how amazing Web scraping is and the kind of thing that you can do if you just sit and do this the whole day today, Um, you might have 35,000 email ideas for you to send email campaigns to, and you probably don't. I need so many similarities to close the number off sales or clients that you that you want . So that's that's the amazing beauty off simplicity off the stool, its simple to use and it's a very basic tool, but the application is immense and the kind of results that you can get. It's amazing. So remember the sort string you can also, but another key word but the searching and see online marketing on over locations on humility. And then you have more specific desserts. But because in this case I just want to show you how to use the tool. And now you can expand on this and think about other ways off the Bring this again. You can now export all of these email ideas in a Google sheet imported in your email marketing tool and so on, so forth. So that's the lesson about exporting images from webpage. I'm not gonna go into the lesson off exporting it from a file, but it's the simple option. The third option on the stool is files, and you can upload a file and and get email ideas from that specific files. So that's some of the lead generation go tang application. In the next lesson, we're gonna be making off born listing website, so it's Ah, it's Ah, it's not a regeneration applications very different application off how you can actually build a side project using just web scraping. 4. Scrape marketplaces to make your own listings website: Hey, guys. So welcome to lessen for and in this lesson, we're gonna learn how you can make your own listing website. Not this lesson comes from actually one of my clients who we were discussing this scraping method with and they went on to make their own side project by scraping different listing websites and putting together that information on a single platform. Eso let's let's see how this works. This is gonna be a little more advanced than what you've done in the second and third lesson, which is humility, extraction from a page or text. Andi, In this case, we're gonna install a new tool which is called crabs. So you just have to search for G r E P s are You can go on reps or dot com or just install it from the chrome Web store. So it's a chrome at on if you're not using chrome. If using Firefox, please install chrome because most of the data escapes that we're gonna do are gonna be done on growth. Okay, so So this is the tool I quality installed this. The first step is go install the stool, pause this video, come back and then we'll get started. Okay, so if you have installed the stool and you're all set and ready, let's see how we can. We can make our own listing website and basically to build a listing website. What you need is data. You need the information you need lists off off things. In this example, we're going to make a list of bars and what's a better place to signal data from but no matter dot com. This is a website which tells you about bars and food. And so, if you could use a matter comets going automatically the director of the closest location to you. I'm in Poland and hence it redirects me to Warsaw. You can do the same, just goes a matter dot com and see where it redirects you. And then you can export any of the data from the matter. Now the idea is that, um, we're gonna have to do Barsa. Let's let's go for drinks and nightlife. Okay, now you can go. You can do this on any any other linker, even any other website. But let's let's just follow the same process for not, but it doesn't necessarily have to be in Warsaw, you can select your location that you're in or whatever location by default. Sematary directs you to know there's a little bit off technology that I still you don't need to go. But I still want you to understand. Is that how this works? How this one application books is that if you have no idea about technology, this is to you about databases. So all of this information, the name off the restaurant or the bar, the location off the restaurant, the cuisines across what are different data sets which are stored in a database. So there's one table in the database. Let's say which has the name that Derrida and different columns to save all of that data and then different rose in the table for each of this item. Okay, so this is one item. Okay, so RSA is the second item. Okay, it's punished. I don't know if I'm pronouncing it right, but the alchemist gastropub is easier is another item. So you have different moral is one item in all of these are different items, and each one has different data points. Associate ID in the database, which is cuisines cost for two hours available. And even the rating words. They're all linked to the specific item because they're all together in one database. Right? So that's how this works. That's all you need to know. I actually don't need to know. There's but it's good to know. And now what we're gonna do is we want to export a list off all of these, um, and make over one block or make of a little side project with the best nightlife bars in Orson. Okay? And we're gonna call it bars in Warsaw. Okay, So what I'm gonna do is buy this domain by this hosting. This is exactly the same process that my client followed and then export all of this information from the matter and just pull it and put it on my website. And then I have a bars enforcer dot com The reason I'm big, this demand and being misdemeanors become maybe I drank better on Google search. Compared to the matter, someone searches for bars and more so because the name is also in my domain. Okay, It's a simple hack, but the application of scraping here is more to create it all website or your side product or to scrape listing websites. It's now we're gonna go to the pool that we just installed, and it's gonna be on your room somewhere. Here, you can see it for me because I usually hide my tools in the bookmarks bar and at a specified given dime. I have only the tools that I need. Now, for example, now I have the tools which I need for this course. So I have all of those here. So I'm gonna go to this website and then I'm gonna go to the two that I just didn't start. Okay, so perhaps there's going toe give you a demo, which is not bacon. So you can actually skip this. Okay? And this stool is actually one of the simpler tools off doing more advanced scraping. It's not like the manual escapes that we did on Facebook or the other locations hack. But this is more of an automated school, but it's a simpler versions. I want to teach this first before we you know, and then we can proceed to more advancements. So the first thing that you want to do a stagger first feel that this means let's say I want to export the name off the restaurant and the location it is. It isn't okay. And maybe for now, let's just export these two. So the first field is gonna be the name. Okay, so I selected the name, but you see here that the location is also selected by default, so I'm gonna remove the location. Okay? That's what I have now the names off all the bars, nightlife, restaurants, pubs except selected from the speech. Okay. They're not even gonna think about the next page. For now, it's Let's keep it simple. So I'm assuming that is the first time you're doing scraping. So I don't want to, um, teach you all the stuff in one lesson, because it's gonna be too confusing. So let's go step by step. Now you have the name. Once you have the name, you can see that 15 items are on the space, which is selected. All you have to do is go and click on save selection. Okay. And now here you can say name ump off the bar. Okay. This is how I'm gonna call my field because in the Excel Column, I want to be able to make out what the field is about. I can also extract the link, but that's gonna be this matter. Language I don't need. I don't get I just need the name of the restaurant. I can link it to their website leader from from my blogging this there's another way of doing it. But for now, let's just export the text. The sample. It is the first instance off the field which the tool is gonna export, just so that you can see if the data is correct. In this case, yes, it's connect. The first name is the sample leader. So I saved the feet, right? And now it says, tag your next meal. So imagine the next field as the next column off the accept. And for us that's gonna be location. This is more like the legality or the area in which the restaurant which you can see here these are the different locations and more so where you have restaurants and this specific location, which I have no idea how to pronounce, is, uh has the highest number of rest on. So anyways, now it's selected all of the locations on this page. As you can see it says 15 items selected, which is the same as a number of restaurants. So you can match it in your head and be like, Yeah, this is fine and just click on save selection. No, the sample, it is correct and you can same location or bar. Simple name of the bar as one column in my except and the location was second. Now you know how the stuff is gonna come out in the Excel or the people sheet and the CSP. So we just saved this week. No. Two feels is OK. We can go ahead. We can also have the menu. We can have four number. Let's let's do this. Let's also have the phone numbers of these places that people can see bars and call for reservations. And website is basically gonna be like a responsive maybe an app, which will let people reserve colanders of restaurants as symbols that so let's also export the four numbers. So if you see, I click on the call, but the view menu is also selected. I'm going to remove the view menu. Be careful when you select the fields, it has to be the same. 15 items because that's the number of restaurants and location. Right now, Here's the interesting thing. When you click on save selection, you know that you don't want just the text because the texture says called Okay, because the text, the sample data is just called. Which means that in your column in the third column, you're just gonna have call, call, call, call, call 15 times, which doesn't make sense. I'm going to change this to there's a data phone number string, which means that the data off this particular element, which is called, has been captured in this data for number string element, right? That's the thing, which I want to extract as soon as I take on this, you see the samba. Little has changed for love. So the Mato has captured the full number inside this field, which we don't want to care about, how the technology works and how they've programmed the stuff. But all we want to see is this. List off things and choose the right one. So it's a simple game. You have to go there and choosing. The developers have chosen the name so that it's easy for them to remember, which is which makes it easier for you to select the right field, which is gonna give you the phone number. So now you have the four numbers and the locations off these bars. Now, people who are going to a specific location tonight can call and make reservations. It's actually easier to do it from my app. Bended went from so matter because I just have the phone number straight away, right? And we're gonna sell here is maybe sponsored listings from restaurants. I can have the sponsor guns and I have my own bars in bore. So app in how many minutes? And I think 5 10 minutes just by scraping. So now let's call it is phone number off bar. So we have name off the bar. We have the location of the barn. We have phone number. And what I did here again is I changed by default. It's gonna be text. I changed the field from text to the one which I think is the phone number. And you can always test this. You can change this and see the name comes here. The class is some, um, CSS stuff which we don't get. Bartok. So you can actually change this. This is the date I can. So you can change this finally and select the one which is actually giving you the phone number. The data that you need. Okay, so now this just get really interesting here, but nice click on Say, field, I'm undone. I can select more fields. I don't want to make my app complicated. I'm just saying, here's the bar. This is what it is called them to get a reservation. It's actually faster and easier than Samantha Bee. You can go to the matter to order online and do other stuff. But if you just want to make a reservation, my app is gonna be much easier because it sort of dreams out all the bullshit, and it just keeps the stuff, which is really, really up on DNO. I'm gonna click. Um, next. Okay. Now, for now, this is this is a little next level, but for now, I don't want you to get into pagination. However, you can also make this app select the next link and it will select it. Will you get all of the data from the next page as well? For now, I'm going to say no. Okay. And here you see it. It shows me the name off the bar, the location off the bar and form. Yeah, And when you click, continue. And when you have this, do you want to go on any of the findings? It means that the APP can actually go inside the restaurant and then pick up stuff like cost for two. How much does it cost for two people, or the menu photos of food and stuff like that? But for now, we're just going to say no, you can try this. I'm teaching you the basics, but then you can go inside and actually bring out the information dressed for my app. I don't need to do this. I'm just selecting. No. For now, that's the another beautiful thing about this app that sometimes if you're scraping a social media website or something like that, you might have to log in. So it lets you give your credentials, log in and scrapes all the data for you. But in this case, we don't need that. So let's keep it simple. So here you go. That's all the data that you need, right? So now you have the name of the bar, the location of the bar and the phone number and that state are scraping for you for creating a new listing website. I can actually use this and a lot of data from the different pages from using pagination get all the pages, and basically I will have a list off all the bars, their locations and their phone numbers. The ones were stoned. Her phone numbers. I'll probably find a website or find the phone numbers manly or just removed them from the list. And I have a new product parts in Warsaw dot Right, so as simple is that this is just one application. But imagine what you can do with the stool if you extract data from stuff like so mato are any listing website for that matter? You have all of this data to reuse to rebuild as lead. Now look at this if you're also selling to restaurants. If you have a product is, let's say you have a restaurant management system, you have ah order management system or you have you are a Web designer wants to design websites specifically for restaurants. With this scrape, you have four numbers off so many new restaurants to lead to prospect, you know, And you have all of these different, uh, leads in your shape and you can start calling them and pitching to them. And so so it's also lead generation, thoughts of building on listing website. You can also do some research stuff and pull out the weightings off these restaurants and compare them to their reviews and stuff like that. So you can actually do a lot with this single, too. So this is one application. I'm were teach you something more interesting in the next one. But as you see, we're increasing the level slowly. This is gonna be there's gonna need some practice. You can try the stool a couple of times. Um, it's a good tool. It has paid options. Like I said, I don't have I'm not gonna give you a deafening, but you can do a lot of stuff on the stool for free. You can have 1000 leads on the stool for free, and for now, we just have 15 leads, right? So so you can do all of this and more we can. We will also learn how to do this exact same process with another tool, which is the freedoms. We don't have to worry about buying this, too. But that's the beauty off. How you can actually use data scraping to build your own listing website. I see you in the next lesson. Thank you. 5. Scrape data for analysis or research reports: Welcome back, guys. I hope you're enjoying the scores and the best way to rate. We learned this way, fastest to practice. And, um So what? I told you a lesson for about making your own listing website. You should go back to lesson for seed a couple of times and then practice making your own listing website. You export some data and put the data together. You can. You don't need to actually go out and buy and domain. And by hosting and off the stuff, you can just write a block, for instance, about the best restaurants in New York or the best you can also do. You can also export some e commerce websites or any listing website and make a nice blogged about the information that you pull out from the website. So go ahead and practice so that you get an idea of how the scrapes work. And when you start selecting the elements and putting things together and seeing what the export looks like, you will know you can map exactly how the elements are coming together in the table and then you know how databases work and you'll actually learn you'll actually reverse engineer your technical skills, and you learn a lot about how the technology works without having to do programming yourself. So moving to a little next step with same tools. It's a lesson five, which is great leader for analysis and research reports. Not this is this application is specifically derogative words research, and there's a lot of data research which needs information to be picked from some website and then put in the form of charts and graphs. The process off converting data into chance is very simple. You can do it on Google sheets or accept, but getting the data from websites is what I'm gonna teach you. Now. In this lesson, eso I'm gonna take the example off. Pricing worse is waiting Analysis from face from Cape Cod Flip card is a e commerce website . You can pick any commerce website, for that matter. The reason a big God is because I use this more often and what this means. Basically, the pricing, or worse, is rating analysis is, um, I want to search a specific category of devices and see what kind of ratings have people given or what kind of readings are there on the platform voice is the price point off the device. The reason we're doing this analysis is to see if there's actually a concept off fake ratings, and the next thing that you can do from this analysis also export all the ratings and see which of trading seem to be automated based on the time difference off the ratings. But that's that's going toe head off what we're learning now. So let's just focus on pricing. West is waiting access. So the two data points I need for this analysis I'm doing some market research and the two data points I need is the pricing and rating. Now just decide not before we proceed. If you're a crow attack, early generation business and open executive, please do not skip this lesson, even though I'm talking about the application of market trees that you can still use this to do other kinds of scapes and apply this lesson. This course has not been designed in a way that you take up the lessons which are applicability to you in your profession. It's been designed in a way to teach your different concepts off, scraping with applications in different fields, but you have to go through all of them so that you loan scraping as a coach and they're short lessons and you won't spend much time doing this. So what we need to do is go on flickr dot com. That's the website that I'm gonna use. Ongoing need, two things prising and reading. And I'm going to select a specific, um, category. Let's say I want to do it for smartphones. Uh, let's say I want to specifically do it for Samson. Okay, so I'm gonna I'm gonna see Samsung smartphones and with Samsung smartphones have What kind of evading, actually, let's let's make it for General. Let's say smartphones and see what comes up. Yeah, so, no, you have different bands, you have results. So the first thing now, if you put your web scraper had on you know that these are the items let's review what we discussed last time. So these are the items, and these are all the data points and this is a table. This is wonder off the table. So redmi note five gold 64 GB is wander off the table, which has different columns. Um, I don't need to buy food, which has different columns, the price point, the specifications, the rating and love about from now, we're just interesting. Interesting. The name off device candidate in just two fields, keeping it simple. But you understand how this works. This is one drawer, five years, one drawer, right? And all of this is in a table in the database, and it comes from the table to this user interface. And now we're taking it back from the user interface, scraping it off the user interface and putting it in a table again, which is an excellent for us. Simple. It's, ah, very transparent process how it works. So no again recorded the same do we're not gonna change. As for this lesson, um, so start by doing the first field, the same thing which we did in the previous lesson. Now, you see, this is the problem. Sometimes you have so many selections which happened, so you have to manly. Don't forget to do this, you have to manage, moved in selection and now you have the only the phones. It's his training for items of their training. For results on this list which have been selected, I can also later select pagination like you saw in the previous lesson. There's an option for that. But for now, I'm just gonna keep the straight for vitamins. And you can always do pagination and get more items, right? No, I saved this, and I'm gonna call it name or smart. I can. Okay, now, this is, uh this is a problem. It says edged Yemen. And I want to show you this problem because sometimes on some websites you'd see that you cannot just extract the text, which is the name off the phone. Like in the previous website. We just got the name right. So there's a hack around this. I'm gonna cancel this. I'm not taking this field. Okay, I'm gonna close this. I'm going to discuss this and I'm gonna go back to the search results. Now, if you see here, there's an image for each phone. And usually there is an old text which develop was used, which is the text for every image. It's like a caption. Think of it like a caption. We had we not going to go into the court, but think of the all texts like a caption. If you really want to know what? The old extends. That's basically it was devised by search engines for people who cannot see images, um, to be able to read, to be able to have text which can be translated to voice or whatever, which is text to describe the image. However, websites use it as a s. Your toe websites use all decks to include the words of in the image itself to rank better for those people so we could try to extract the name off the phone from the image. Now, this is a work around which you need to learn, which is why I took this example. So start by tagging your first in this case, I'm gonna attack the image. Okay, Now I will click on save selection. So if you see it takes the image and it says the link of the image. But what I need is I need the all texts. I quickly all decks And there you go. They have the name of the phone. So flip God is using the name off the phone as an old text. So that's good thing for us because, you know, it's a hack for us because we got the name off the phone from the image? No. And now I can call this So the data which is gonna be extracted is not the image because I selected the all text off the image night. So the data which is gonna be exacted is this is the point text off the image, which is also in this case for flip card name off the phone. But it's not necessary that in all cases it will be the name of the phone. So it's get the tool is there for you. Everyone says, Hey, you can use a tool to do this. But these are the problems when you use the tool which I want to clarify. So you can easily go ahead and escape things without having to worry about the troubleshooting in the problems that you will come while using. So I just don't want to give you the tools and culture teacher, the prop common problems and how to solve them. So this is the first problem, and this is the simplest work around is to find the time Miller the image and become the old text off the edge. Now I can call this name off form, right? Because the data is gonna be this. It's gonna be the name. It's not gonna be the image, even though we're pulling it out from the image through the or text you. All right, so try to do this. I suggest pausing the video at this point, going ahead and trying to do this. You can first went on flip card so you can follow the same and then try to do it under free websites and then come back and play the video entry to the next fields. So, sure, let's proceed. If you've done this, save the field, and now you do next field. So the field that I want to see is, um I have stars, which is my ratings and number of meetings. So I also want to do some combat, isn't on. I just don't want to see the the rating. I will see the rating for the average rating 400 people or something like that. So let me pull out this as well. Is this okay? I think it will be useful for now. I just want this, but I'm gonna pull out both anyway, so let's do this one for us. Now you see that all of this stuff is getting selected. Let's remove it. We don't want that. We just want this. So now all of these ratings have been selected, right? So I'm just checking if it's been selected completely. Yes, it has. So I saved the selection and I say text sample data for 0.6, which means it's correct, I say waiting user rating off the phone about its collect. So let's just save it right now. We need the price like, because we need to do the analysis between the price point and the star rating. So let's select this. Then you can see here that some of the items this is these money to Scots. It is the second problem you might see. So you see this 42 items selected by Lucien between 3 24 which I don't train if you train for eso. That's because of this, because the items which I don't discount the tool is also picking up those values. So just click cross and it's training for right, so you see. Okay, so we have 24 which is the number of the smartphones, and that's the thing that we have here. So let's save this. And this is correct, because this is the price that I can say Price. Cool smartphone. My chief in Zion, if I also wanted to do some analysis with this, but that's OK. You get the idea of what I'm what I'm trying to do. So now I have the fading I have the price point. I'm gonna do some analysis on, um how many you know, phones. Let's say which above 10,000 this is the Indian currency. ₹10,000 are also rated 4.5 plus. Because I'd leave my my hypothesis would be that expensive bones should be highly elated. And But that's not always the case, of course. And we're gonna prove that. So next I'm done. So I click on next. No pagination again. You can always say yes hasn't next link and then select us so the scraper will go on and on , right? I'm not gonna do that now. It's just gonna save time. And, uh, I'm gonna just say no. Okay, So let's review Does the name of the phone which we got from the image All text. Remember that the user leading off the phone, which is the text speed. That's fine. The price of the phone, which is again the next week. We have a specific currency. In this case. It's Indian rupees and we can continue. No, we don't need a log in. And yes, that's that's the data sometimes is another thing. Sometimes you see that you might have to clean clean up this data because it's the way it has fasters. Not on the same road. But that's easy. You can use a script on Google sheets. I don't know how to do it. In Excel, you can probably google it, but Google scenes is simple. You can clear the empty cells with one formula and you get everything together. So anyways, the point is you have the name of the phone. You have the user rating off the phone and you have the price of the smartphone and this all the data that you have from from Cape Cod. So it's really easy in this case for me to do the analysis. No, no. If I was a blogger on market research, analyst and report content analysts writing something about the comparison between prices and ratings and have to manually do all of this. Can all of this information it would have bean along? It would have taken a long time for me. The other ways you can use this is, of course, scrape equal, worse websites, as you can imagine. So I want you to think of different applications of this now. I have given you enough power. How fits with the course on now you can actually escape e commerce websites you don't really need. The Jason finds that there's a lot of stuff that you could do with this with CSP Financial be enough. And this tool is really is really a good tool because you don't really have to think about the elements and selection, but the but surely you're also going to discuss another tool in which you have to actually draw the site map. But that gives you more power off the kinds of things that you can do. Eso that's less and five guys. And in the next lesson, we're gonna scream searchers us from social media. So it's you there. Thank you for watching 6. Scrape search results from Social Media: Hello, guys. And welcome to lessen six off the lean Web scraping course and blessings. Sex is all about scraping search results from social media. And I'll tell you how I came up with this application. By the way, I hope that you get the flow Now off the course, we're halfway into the course. So just review what we're doing here is we have one application of data sweeping, which, in this case, but this lesson is how to scream Search results from social media. And we have one example off that application. Um, uh, which in this case is we're gonna scrape the list off Facebook pages about Cryptocurrency. Why Cryptocurrency? Because it's trending, of course, on why we're scraping it. I'm gonna tell you a story behind that, but that's the format off the course. I hope it's you get the flow now, Like in the previous lesson. The application was how to scrape data for on analytics or research reports and the example waas The analysis we did from flip, right. So this came from a client who was actually doing starting a magazine on Cryptocurrency and its online magazine slash block and what they wanted Waas basically cryptogram people who wanted to read a content about Cryptocurrency, and they decided to grow attack, to run agro attack on Facebook, wherein they would docket pages of Cryptocurrency, which are not really the official pages which are not really brand pages, which are like community pages, right where community pages are essentially just pages which people and students and enthusiasm put together. And then they stopped managing it after some time most of the time, that's what happens. They don't have a really source to, I mean away doing to sustain it because that's not a place where they could make a lot of money. They might try to do some paid posts and stuff like that, but it's not really a good business. And it's just a Facebook page, which people running without linked to a brand. And my client wanted to buy some of these pages by off the pages the complete which is not just the legs so that they can get access to that audience and then use the post from their blawg to share to the audience that they by priests from re branded pages to their brand. If possible, Facebook has some restrictions to that. But that's the grow attack that we would be. And our job as a team was to first find out the target pages. So we wanted to basically send links. Now there's a simple thing, right? Simple thing. We wanted to send links off the top, Let's say 50 pages on Facebook about krypton. Now, of course, you don't want to do this stuff manly. You want to do, we want to scrape it and then you want to scrape their websites. Then you want to scrape their number of legs and then you want to run the script notice. This is a little ahead of this outside of the scope of the score spread. And then what we did was we learn about which would actually go on all of these pages and send them a message with the proposal and also go on their website, speak up their email ideas and send them a male about your proposal to buy off page right? So let's now come back and focus on the first step, which is getting the legs off all of these pages in one sheet. And that's the application that off scraping that we're gonna do here. So the first thing is simple. We gotta face the dot com. It's easy. You guys do this and, um, I'm gonna search for Cryptogram dzi. And then in this case, no, you have to see this as a general process off scraping social media results so you can escape people. You can scrape people's profiles, you can scrape people's photos, you constrict videos, you can scrape up everything. But I'm gonna inform my case from my application. And when the script pages. So no, you see, these are the doctor. The results when someone searches for Cryptocurrency. These are good results for us because they're coming on the first few. You know, I'm gonna load a couple of results because I want to scrape this eso that's that's how it works. The scraper, the tool that we're gonna use is going to run only on the visible results. Because we're not selecting the next or be infinitely scroll. However, you can select that and then you don't need to school so much. The reason I'm scrolling manually and not selecting the option the Infinity school option in the tools, it's because I only want the top ones. I don't want to school. Do much. That's enough for me, right? If you select the infinity school is gonna export like 1000 pages and then say, hey, you know, way have the limit has been reached. So you don't want to do that. You want to just select the ones that you So the information, Basically, I just need the links of the pages, but I'm anywhere is gonna pick up the name of the page is ultra. So the client knows the name and the link off the page. And in this application, also, we're gonna use the same too. It's a go and open the tool. I hope you're doing this with me so you can just pause the video. Go and we're Facebook Search opened. The pages are in this case, you can also do groups or people's profiles. If you're looking for students off a college, you can put the college in the surgeon, then export all the students and the profiles and then do some shady stuff with that, um, for now, let's Ah, let's focus on real work. Okay? So start by talking your first field, and I'm gonna take the name of the page. Sometimes this happens, it seems one item selected attack a similar ones. When you select one more and remove the blue ship and now I have and names off the pages, you see that it's exporting 71. I need 50. But it's OK. It's explain, please. As usual, we scroll down and make sure that it's going So all the pages have been selected there, highlighted in green. So it's good. I'm going to save the selection and this is all cool. It's the text. It's Cryptocurrency. That's the name off. So that's what I'm gonna call the title because that's gonna be my grow on excellence will be my heading off the column in Excel. Name off the page safety. So all year so far you can pause the video again. Go back, select the name of the field. Make sure all the names of selected scroll only to the ones that you want and then put name off feet. Okay, let's proceed. The next AG is gonna be from me. The link. Now here's the interesting thing. This is something new. I'm sure there's no basic estimate, but basically the link is within the text. It's It's the name is H R. Yes, it's difference to that click. And I'm just going to select this thing again. The name and then select do the same thing. Basically select one removed this. And now I have I scrolled a little more when I was checking last time so from 71 became 80 . But we can ignore the last nine rose off the extent I hope you can imagine I'm talking about we scrolled anymore. So now the wind is picking up the new scrolls which, which we did I don't want to screw it again So now I'm gonna save this So you see, it's again picking up the text. But now I need the link off the page. So what I'm gonna do is and will go to the extract and select the element age. That's all I want you to learn in this lesson. Basically, the new thing is you can export social media searches. You can scrape people's profiles, anything on Twitter or Facebook ling. Then I will be to be lead generation and a lot of places. But be careful off selecting this and now you have a link off the page. Right? So that's the simple way to get links. A link off the bait should show up in my Excel assembling off, bitch. And I'm saving this week. And now I'm going to see house. Now, you see if you also have an option? Yes. Has an infinite scroll, which is gonna keep strolling the pitch. But I just want to talk when someone assembled. So this looks nice. Name off The page. Text link of the page is a link link format text. Commit to feels you can actually go inside the base and pull up the number of fans and bloodline. You do some analysis, I would actually go in the page, pull up the number of legs, and then filter only the pages which happened within 5000 and then run the automation to send them messages. But that's right outside of the scraping course. That's more like building robots and automation, but for now, but scraping this is the data that we need. And you're doing a log in for this. Um, I'm going to select no and see if it works. Yes, If it doesn't sometimes because it's Facebook. You might have to give you a look, but if it works without it, it's okay. This is the sample leader. You can always exporters to win acceptance stuff. So a Zaken see here we had to remove reading, Jack. But if you check and remove some of the extra friends, it's gonna be funny. So these are the names and these other links. Now I can send this reclaiming of that. I don't want export this. So that's all. You can go to Facebook and export any of the data export people's profiles pages. This is just one specific applications. You can export multiple elements from social media and do all of that stuff. Now, at some point, you're gonna hit limit off using the stool because it has 1000 of clubs. It's very cheap, however you can pay, and by the tool, because if you like it if it's simple. But the next tool that we're gonna discuss with a couple of off to victorious two of the next tutorials is a free tool with unlimited exports on the middle scrapes, and you don't have to pay anything for that. So I don't think that the soul course you need to buy any. But if you really like this one, if you have a specific, like, go ahead. Um, I'm also going to go a little more in depth about the elements on the page and the Howard Accord works and how the database works. So we're gonna do that in the next lesson before we actually start using the next tour. I hope you're enjoying so far. And thank you for watching see you in the next. 7. Scrape info of companies in any industry (Part A): Hello, guys. And welcome to lessen seven off the lean, escaping course. And so we're going to really turn the volume up and take things up a notch here because what we're gonna do now is explore a new tool which has unlimited capabilities of exporting and also try pagination and multiple exports and other stuff like that. It's gonna be completely automated. We make the graph, which is the site map, and we let the tool go and export everything for us. Scrape everything for us. So, um, so in this lesson, I'm taking the application off scraping the list off top company male names in an industry . And this was this would typically be a useful in the gym or grow attacking or business development or just market research, multiple areas. But what's more important is for you to understand the process of how we're gonna scrape of this information. And the specify example that I'm using here is stop accounting firms in New York. Now, this is just the example, of course, and you can also do stop financial banks or companies in Singapore or top I D forms and band. You are whatever you fancy eso before we get into the example. Before we start discussing this, I want you to install this new tool. It's a Frito again, a chrome extension web scrape up. So that's the dude I want you to search for. And preferably just install it from the chrome Webster. Okay, so you just need to go on the tour search for Web scraper on Google. I'm gonna put the links of everything but the videos. However, if you just search, it's gonna be the first result, and then click on add to grow. So I already have this. You can pause this video now because the stool is a little complicated on, but it doesn't have good documentation on chroma anywhere. So you have to follow step by step. Suppose this video. Go ahead, install the tool and we proceed. Okay, so now that you have the tools installing on chrome, what we want to do is wait. Look for this list off top accounting forms in New York, which we're gonna them scrape data from. So I'm really but a google dot com. And just for now, you can follow the same example and then you could look for different kinds of bliss. But Phanom very do top founding companies in New York. Okay? And we're gonna pick up the first result for the ease off the videos. I'm just picking up the first result. However, when it comes to grow attacking legion, Um, be a little careful about picking up stuff from the first place. Be more focused on the kind of results that you think will match your target audience. I found a lot of good lead sources, which are like on the six page of seventh page of Google. And they're much better sources because they're outside the view off of the Legion people or other growth hackers. So go a little off beat. In that sense, I'm just thinking the first result for years have continued it. And, um you see, in this case, we good. We have all results on a single plate, so I'm just gonna scroll and make sure everything is okay. We don't have imagination. We have all the results are just to give you how this works. When you think like a web scraper, you have to see this and imagine these as Rose off the table off database on dumb every draw has one company in all the information of the company. Let's say this is the first company and this is the information. So there's a There's an image. There's a name of the company that a small description, What service focus. They have their minimum project size, $1000. In this case, the average hourly laid the number of employees and location. And let's see, what's the kind of information for my purposes? Let's say I need, um okay, let's say I need the names of all the companies of 40 forms. I need the minimum project size, the average hourly rate, employers and location is okay because it's the I know it's New York. So that's those are the three things I mean, I can also, by the way, extract and scream their website and then, with all those little I'll give you a sample case of what you could do with all this. You can pull their websites and then have an automation which would go and find contact pages off the websites on email ideas and then send them states campaigns. So that's one application off all this data. So here's the process you need to think off why we're scraping this data than the process off screaming the data, the process itself, and then what we're gonna do with this data, Right? So that's the entire application comes together when you actually use the data for research Bt Legion forever. Right. Let's do this. Um, Now you have the, um, black and installed. I hope you haven't touched anything in the bag, and you don't have to go to the plug in it somewhere on chrome. But don't worry about that. Let it be inside grown. What do you have to do is access the application from the page which even scream. So now this is the page between the script. This is the link of the page. What I'm gonna do is just right click and you can inspect. So if you haven't seen this before, don't get worried about this. How it looks, it's it's just the court of the page and inspect means that we're going inside and looking at how the quarters. But don't worry about that, but not don't do anything with this. We're gonna go to the last tab, which is the web scraper down. Okay, Now you might have this inspect window. If you're a beginner to in spend of it is the first time using this you might have this on the right side. Please put it on the bottom below. There's an option to do that on, and then it becomes easier to use this tool. You can't run this tool if you have the inspect this whole window, it shows up in under the page on the right side, So move it to the bottom now to the click on Web Scraper. As you can see, this is actually a new tab on your console under inspect because you installed the extension. If you haven't installed this extension, you won't see the Web straight adapts. So again, if you haven't so far, please go back, install and come back of and proceed with the video. So now that you have this Web super, you'll see this blank screen. I've deleted all of my side map so we can both work on this together. So click on create new site map. Like I said, it's not really intuitively designed, so it doesn't feel like this is the first thing that we should do. But that's exactly what we should do. Supplicant. Create new site map and select Create site map. You have these two auctions come up site map name and you are the u. N. It has to be Just copy the U N from the website, which we will escape sometimes when we do pagination We have different pages off site. We typically copy the U N L from the first page bread escape. It has to start working in this case, we just popular you are really be saved. Um, clutch accounting. Okay, because I'm just gonna do accounting, and I'm gonna scrape the websites on this the company's armed speech. So it's good, but the name this name is up to you. It's not an idea. It's not something in the court. So don't worry about what name you put. It's up to you. Um, and the U. N has to be the same as the page that you're scraping or the piece that you're gonna start your scraping from. So this is a little different from the previous tool, but it's completely free. So that's why I wanted you to love the stool as well. And throughout the course as you see, I'm giving you multiple tools so that we're not dependent on a single tool. And if one doesn't what we know the other one, and also the tools and just re years of doing things. I just want you to make the mindset of off of extreme Burr and how you can use it. Um, so let's get started. We add a new selector first, and the selector is basically like it's election element on the page. Okay, Before I go into the individual elements like the company name and the validate and stuff like that, I will select the entire drove. Okay, so you remember that each company is a road. That's that's the mindset. That's the thing. That concept I need you to understand. So each of these is a rope, and that's the first thing. And Melissa next to select now on the speech as you see again, not really intuitive, but the first thing we need to click on select to click on select. And now you can take your mouse over to the grow where you have the company information that you want to scrape. Okay, so now you see this in diarrhea has been selected, which has all off my information. I need the company name. I can also instruct the link from there. But for now, let's do the company name. I have some description I have, but they are validate a number of employees and stuff. Project size, you know. So I select this. Okay. Now, with this tool, you always have to select two items off the same time for it to put it to scrape everything else. So I selected the 1st 1 and I'm also going to select the second. So you see as soon as a select to do of them. It's elects all of the roads which are inbred, not the green ones in red automatically. So in selected all of the rows. So let's just confirm this. Yes. Said selected all of those, but not on page two. Right? So maybe $20 because the total 42 pages I'm just gonna scrape the for space to keep things simple for law. In the next tutorial, we could go into page initials, but for now it's selected all of these. Okay, so once that's conform and military con done selecting. So as soon as you click on dance like, did you see that? Some, um, identifying a sort of text has come here. You don't need to worry about what this is. This is basically how they've specified the names, the glasses off the road in the estimates slash CSS after. We don't need to get into that. We don't get about that. Um, then this comes here and that sort. Now, the idee is on this role or I can say item because this is one company, Uh, and you can call this anything, Nobody about ideas, not a technical term Here. You can call it anything type in this case because it's an element. It's not exactly I'm not exporting a text or lingering ending from here. I'm just saying I'm just telling the tool now that these are the elements within which I have information that Evert scraping future. So it's an element. OK, I predict this element I will scream information that you see in the next step. Okay. I will always select multiple on the stool to be on the safe side. Just select multiple. What it means is, if this company information appears strikes piece, pick up the information from boats, right? So always select, multiple, introspective off. What do you think is on the page or not? OK, it always works better that way. And you see that this new selector has been created here? Item. But I'm gonna keep it for root. And I'm gonna save select. And, um, I have my item here. I have the selector, but I chose from the mouse pointer. I have the type of just element multiple. Yes, always. And parent is root. Root means this page. Right? So since we put this link on the route No, it means this page. So now you what do you want to do is you want to add a new selector, and now we're gonna go into the specific data points that we need to extract. All right? 8. Scrape info of top companies in any industry (Part B): so guys to go into the specific elements that we have to select. Let's review what we have So far we have the we have the I too, which is the rope of the company name. We have the selected type as Clement. We have multiple. Yes. And the palace Leichter as route, which is the space. So make sure that what if you have this, you can see this in your web scraper tool inside cruel. And now we're gonna try to export the elements which, in my case, I'm looking for, Let's say, the alveoli date and the number of employees. Okay, also, maybe the project sites the project size. Typically, it since either top companies is, as you can see, all $1000 for each. But we can also s so we can ignore that field. But let's let's not get into how to do this. So what do you want to do is click on add new selector And like I said, the first thing you do on this page is the select. So click on select and then just take your mice. Most pointed to the first element the specifications mint now inside the role that you want to script. So let's say with like this and that's let's select one more. And then we have all off the just make sure that it's all selected. The red ones will show you what all selected and then just click on inspecting. All right, so always just to make sure that you have selected died elemental, slick on Element. And you also want to see how this data scraped will look like when when this element of Scripture you want to click on Data Prevue. So it says $1000. And that's exactly what the state should be like. Um, and also So this is this is a little advance, but tried moving the first Dave text, which you have in the element and click on data review. You see that it stays the same in effect, local element prevue. It's still selected, so the good your score will still work. But I'm just sort of reducing the clatter by keeping just the specific element returned on . But I'm gonna select idee as project sighs. All right, now, since we're speaking in sight now, we're working inside the element we don't need to select. So when you have the main elementary rules that multiple rows you need to select multiple. However, the project sizes on, even after once in the store in this item into only occur once. So we just keep it once you know, you should understand how this tool is gonna work, is gonna go to the main element and then scrape the individual elements like project size and so on inside each element. All right, so, of course, the parent now is the item is the main roads. The main element that we've selected for the item. The parent is the root, which is this link. So this link is on the top is the big daddy. And then there is each item, right? And then their grandchildren on the on the site map tree, which are project size and affiliated number of employees. Think of it like this family tree or O R. A site map like so. And let's just click on save selected. So when you click on Data Preview now you see that it's gonna pick up all the project sizes , which in this case, it's the same for all the companies that $1000. But it's gonna pick up all of those elements. Also, you see that the project size has gone inside the elements of route is the main link. And then the item is the row and inside the item we have product size. No, just like we added this. We're going to stay at root and we're gonna keep adding the other elements. Add new selector. Select was the first thing you pickle, select the data, select the data for two off them, and then click on down selected. All right, so you have span dot oddly rate, which is the element that's gonna patch from And then we have hourly rate type text. We just need the hourly rate multiple, not needed because we're inside the element and make sure you select the parents like there's item save selectors inside. I don't know. You have two things project size, Another great again. Always click on data review to make sure the right data is gonna be scraped so you can see the the probably rate of all the companies is showing up here, which is exactly what's gonna be one column in your excel. When you actually scraped the state which we didn't do in a while. But before that, I was sure you something interesting Visual representation off what we're doing. Just click on site map and the name off the site. Man, whatever you call it will appear here. In my case, I've called it clutch accounting and click on Select a Graph. So this sort of shows you the map and how it looks from the root, which is the page is gonna go to the item than from each item. It's gonna fetch the project size and the absolute right. So if you see something interesting is missing here, we did not fetch the name off the company's. Let's also do that so that in one column we know what's what's the name. So go back to selectors. Go back to route, add new selector. The first thing you do is select, and then you want to select the name of the company sold. Let's select two of them. Don't selected to see what it does here. Basically picks up a ST dot campaign in which is a heading back in next email. We have six heading tax each one through at six on, but it looks like this website is using H tree for their company name. Okay. And maybe edge to for there for the term topic holding companies in New York and each one for plastic. Now, you don't need to know that. No, this but it's just a quick side note for s U heading bags and also useful for search engine optimization. And as he executives, probably why here will know what I'm saying. But just again to check. We just click on Data Purview and it shows the name of the company so it's collect. And then, for the idea, you can just say company name again. We will select the parent as item. And since we're inside the item now, it all really notice like multiple save. And this is the name, the project size and the ability It Let's also put the number of employees. So select is the first thing you click on and then take your mama's pointer. Select number of employees. You can also know you can do this twice now See here the name off the just follow my mouth's and see the name. It stays alive. Remember black and then span employed now, as soon as I select one more. It supports expanding. So that's exactly what I did manually. In the first case, I removed the four spot. Once you have some experience working with this tool, you know exactly which elements should come into selector. But till then, I suggest keep using your mouse pointer and use the vision way to do it on. But as a number off again next, no need to psych multiple pins like the item and save. So now we have good inside. Would we have idea, which is an element type selector and inside that we have tech selectors projects? I believe a company name number of employees. But let's see the data preview for each of these project size are really great. Okay, It's good company name or names of the companies and the number of employees. Now, basically, the stool has to scrape all of this Dana pull, pull it all together and give us a CS. We filed s so we can do our analysis on accounting company easily general accounting companies off whatever we want to do with it. So let's one last time just for the sake, off my right brain. Look at the visual representation of this route idol and over four elements that were scraping from the idea. And then once you have this ready, it's time to watch magic happen and let's do it together. Just go on with the link and you have to pick on the four killing which is scraped, I suggested to keep the intervals as it is. This is basically to tell the website that he of your human, you know, it's very human behavior, but not autumn of doing automated scrape on your website, which is what we are doing. But this is This is the feature of the tool, which prevents telling the website that so start scraping and then very important, you keep your hands off by tunas performing the script and then it's gonna come back and you'll have a be fresh button. Just click on the refresh button, and now here you see that you have all of the information that we needed. So it's in the order that I scraper. You can also do the company name first and company named, and you have a very clean excel with all the information. So you have project size are mediated company name a number of employees. The last step, of course, is to pull out all of this information and see it's me. Now You notice that there are a lot of other fields that I can pull out. I can have the image. Also, I can have description. I can have service workers location. In this case, it's OK because I know I'm ex scraping accounting companies from New York. But if I'm scraping, let's say this as the parent link, the main website is dependent link, and then I probably want to get into their specific locations that this website has. So I wanted so much for their 401 falls. I can either scrape all of them with imagination, or I can actually go categories or, you know, by location and then New York and live above a block and make a big Excel with the location people. In this case, since it's just location, it's just New York. I don't need the location called him in my export, so we have all of this information now to put this in the CSB so that it's usable for us. You have to go back to the site map clutch accounting of whatever the name you've given to your side map and then click on export data as CSP. Um, and then it says, Don't Lord. Now let's open this file, and we have We should have some beautiful information, Um, here on this fight so you can see we have exactly the data, the data that we wanted, and we can arrange these columns and put filters into stuff, but this data So this tool and this process that you just saw is actually scalable to scrape anything. Once you know, imagination. That's the only thing you need to learn now, which is gonna be in the what in the next lesson. But then we're also not really limited sets to the stool. And we're gonna see how to use for basic scrapes how to use stuff like Google dogs. Very simple and easy to process. So this was, um, the top accounting scrape. And the next lesson we're gonna use pagination to scrape some e commerce website. So see you there. Thank you for 9. Scrape data from e-commerce websites: Okay, guys. So turn the volumes and get ready. Because now we're gonna have a lot of fun with it origination and actually see the real power off all that we have learned, Andi scraping the websites or buy in bulk with just a single click. So this is the single click and you have page nation by pagination data scraped to a CS reformat. So that's lesson eight on In this case, we're gonna be application that we're going to see is scraping data from imagination, often e commerce website. And you can basically use this application on the process, which I'll teach you in this tutorial and scrape any information from any website with which has beta nation with a single click. And the example that I'm gonna use is I'm gonna find the best nonfiction books to buy online. I'm essentially what I'm doing is making a list off really classic nonfiction books for my block on. Then maybe I want to get affiliate from Amazon and sell it on my block. So it's not for me to buy, but the search great is gonna be to find the best nonfiction books and the dude I'm gonna use is the same to which we learned in the last tutorial to turn seven the Web scraper. So open your crumbs and let's get started. So the first thing we're gonna do is Google and such for by non fiction books more than and again, for the sake of convenience, I'm gonna go to the first result, which appears, and it's It's a simple list, but I wanted to learn the process off. How to get the imagination scraping. That's the idea off this tutorial. So for now, open the same website and you know what to do next. Right, click on your room arousal and people inspect. So you should have the same window again, which is your inspect window. You have to go on last link as usual, which is the Web scraper link. And, uh, this is the the site map that we created for the clutch accounting to export the top founding companies in New York and for this website. It's a new instance, its new project sort of skip new scraping projects. We go, we have a new site map, which is, um oh, for this project. So just go on, create new site map and put a name that you want I'm gonna say reprint nonfiction again Looks you can put whatever you want and I'm gonna select the U. N, which is always the Since we're speaking about imagination, you have to remember that the U. N has with first you are off the page. You know where the scraper has to start the work. And essentially, in this tutorial, when I'm gonna tell my scraper is that hate goto this link Hope goto each item. Remember, the item is each door on this page and then export the names of the books and the names of the authors of the books. OK, and then click on page two and do the same thing again and then click on pastry into this. Do it for route. Pay attention to this because the exact statement I'm going to say is gonna be translated into the into the scraping graph. Go to the root, go to the root, which is this website and scrape all the items. One item is wonderful on this page, okay? And then also scrape those items for each off the pages in the pagination until page 15. And what are the elements from each item. And when you rescript a name, I don't want the Amazon link. Right now, I just want a name off the book and the author. So it's scraped these elements from each item from the route as well as from the Page nation and from each pitch, All right. And that's exactly what we're gonna translate this statement into a map off you to create site map. The first thing you know is to add a new selector, you have to first add the entire element not see. This is the previous website that we scraped the Pledge Accounting website. Everything was in a role like so visually in this case, it's in a grid. However, in your in your mind, when you make the table in the data base, you should have to imagine one rule for one book. Okay, so the you I can be great can be can be single columns rose. But that doesn't matter. It's still in the database. It's always one item is in one drawer in the table. So this is when I do wonder off the table. This is the second row of the table. There table in the database, Not in the user interface and the user interfaces and great format. But that doesn't matter, right? The toad item, the fort item, The fifth vitamins Expect him in all these items. So first I'm going to select the parent. I don't. So I have my route. And now I have my parent item like we always do. We're gonna select two of them, and then it selects just going down and make sure that everything is perfect. It's selected and then eminently condoms on just to make sure it's connect. I'm gonna select element review. We don't need to do a data purview here because it's just the item. And I'm gonna just like we did in the last period when change toe element the type and I d . I can say I don't in this case and will select multiple because I'm selecting the parent items. I always to multiple independent items and then parents electors is gonna be route because inside the room, you have so far So Good Upton here. The process is exactly the same as what we did in, uh, lesson seven. And from here, what we're gonna do is Now we're gonna get into the specific elements of each other, which is also the same process, which we did. And a lesson seven. So I'm gonna select new selector. You can select the first thing, select the name. Let's elect to off them. So it's elects all of that. Just make sure that's okay, but still see, we're on a single page page nation aspect of not coming yet. Okay, So this is the fusion process, which you did last time. Done selecting and you see a dog book name. The element has been picked up by the selector, so it looks good. You can also see the date up your now, actually, the origin of species, which has the original species, which is correct Data element review. It shows all the names. All good. We're gonna say Name off. Since you're inside, we don't need to select mounted below the barren selector. Now, from incident route will be item. Because inside the item, we're gonna scrape this disinformation and save select. Right? So we have item and we have names, so we're gonna go and look at the select a graph. Dude, I didn't and name off the book. That's exactly what we want so far. And now I'm gonna also go back to the root and add new selector for the author name. All right, so it's like the old name once and twice. And then it gives me the selector, a dot or name, which is the element which will give me the auto name. I still keep a text. An essay name off the auto again. The idea can be anything you want. You need to select multiple DNA. The pain sector has to be item because inside the big guy, two years, like the name of safe selector. So, so far, so good. We have the route, which is the speech. We have the item, which is this item the entire rope off the debate. And then inside the item, we have the name off the autumn and name off the book. Okay, so let's make sure this looks good. And for the visual learners, his route, His item is the name of the book and name off the autumn. And so what it's gonna do now, if we don't, the script is gonna go to each item. Escaped the name of the book name of the auto. What's going to stop at the end off page one? Right? So because that's and we haven't given any command to the tool to go to the next pages and scripting X pages. And that's exactly what we wanted to know. So we go back to selectors, we go back to route, and now we're gonna add a new selector in the root. So first, let's look at what we're doing. And then I'll tell you how it makes sense. So we're gonna go to a new selector, okay? And this time we need the page nations. I'm gonna click on select and I'm going to select the pagination you RL's okay, one, two, and it selects all of the presentation. You on it again? I selected two of them. It selected all of the imagination urine. So I'm gonna say, done selected. So now I have the imagination Girondins as a selector and you know what type this is? It's a link, all right. It's a link. I don't need the number I don't need 234 I need links and show you how I'm gonna make the to click on each of them for now, Link and the idea I'm gonna say pagination again. You can say whatever you want, but just for my own convenience. And this is a sort of parents. So I'm gonna say multiple because it's multiple page nations. Just select multiple. In this case, the parent in this case is gonna be route okay and save selective. So now I have item and imagination And let's let's tell you what what's gonna happen here? So route I do and then pays initial, right. So let's repeat the sentence which I told you before I started the process. I said, I want you to go on this page on the route and then exported. I do and from each item, pick up the name of the book and the name of the author in the Excel. But then it's going to stop after the first page announcing, I also want you to go to the page nation and then you want right. There's nothing in this draft after that. I have to tell the tool now that I also want you to go to the Page Nation and then select each item in the page nation and then the name of the book and the name of the author, which means, which means that this branch of the tree can basically be copy pasted next to pagination and could do the same thing. Right? So gonna go to the root and pick up the items and also going to go to the pagination, pick up the items. So to do that, it's really simple. Okay, Ready. Click on. I do edit and make the barren selector, boat route and business. All right, so if you visualize the truth, this is why the visualization was really important. Okay? With a big that branch of the tree and also copy pasted inside presentations. I could pick the item branch of the tree, imagine a baking that plant and also copying it in pagination and I safe selected now to see if this worked. Let's go and select a graph route. I do. This is my branch which I pick up and based inside imagination. So this is the beautiful tree that I have for my scrape. And this is exactly the same statement. I've put it on a tree, former on the two. So what I said is I want you to go on the route, and I want you to pick up each of these items and the name of the book in auto from each item. But I also want you to go in the imagination, which is the links that I just selected and from each page pick of the idea and the names of the book in the auto. Right, So beautiful. It's just how you think. And then you put how you think in the map and you can scrape any website guys. Now it's time to click on the fourth Link and watch Magic happened. This is the time you have to keep your hands off and just watch what's happening. So you see, it loads the first page, but some delay because we've put in the delay. So it's OK, and you can see the original species is the first item on the speech. And, uh, okay, come on, wondering. Right. So slow Connecting. Okay, Now it's a second pitch, right? And, uh, it's gonna now see the second bitch. We still rate. We have prettily, intentionally between the pages and between the low time so that we don't come across and get blocked Now The third page, which is Italian hours, starts with the first item tally numbers. Now this is gonna keep happening. It's gonna keep reloading and keeps giving. It's right. Now it's scraping the data. It's going to root. It's paved all the elements. It's going to the second page newspaper, all the elements. The 3rd 4th 5th will keep doing this under Page 15 and you can raid, chill, have simply coffee and come back. And you have data off all of these nonfiction books from 15 pages off this website. So we basically just scrape this whole website for all the nonfiction books. And imagine doing this for any e commerce company of any website, building all the data elements, and you have enough data of the world where there's so much power in this data. There's so much power in this data, which which probably you will only realize when you start applying it and you can build start ups. You can build data companies. You can build research companies. You can build content companies to compete so much you can create new content. This amazing power in what's happening right now, it's it's magical, it's it's Basically, it's it's it's just a technology, but the very can apply it to different fields to different computer analysis, lead gen and data extraction and market research and content balances. It's so beautiful. And yes, we're back to the page and all of you to do is click new fresh. And there you go. You have all the books and the orders from all the pages that skip. It's also telling you the based numbers this from Page 15 paid 15 16 56362 for it also shows you all the pace numbers escaped from, and in the end, all you have to do is click on Export CSB and let's Look at PC Expo. And that's the beauty of Web scraping eyes. We have all of the books. Someone was crying right now. It's so beautiful. You have all the books you have, all the altars, you have the pages. And that's all the data that you need. And this is the process that you would have fallen basically, for most off the imagination based scripts where you're just going to select the elements, make a nice site map. All you have to do is first, I think the first step in if I have to give you a general tutorial, you know, forget about the methods and processes. The first step is to put into statement what you're trying to do. You know, that's a strategy. And then you translate that statement into the tree, and then you just pressed the button and wait for the data. Could be scraped. So, um guys, that's the That's, um lesson eight. And in the next lesson, I want to in the next a couple of lessons. The next three lessons. I'm gonna show you some shortcuts for Google docks to scrape data without using any other dual. And then you're gonna conclude this, uh, of course. But I hope you get the idea off this lean course of having 20% skills, which I needed 20% methods and skills and tools, which you need to do 80% of off stuff in a field. So data scraping on any other field just needs 20% of the effort. And that's the 80 20 rule applying applied to skill development. And I want to have many more lean courses like these which teacher? Different elements, different ways different methods and basically making a Web scraper in a few hours. So in the next lesson, we're going to scrape the search results off Google for any keyword and the top 100 search results authentic. You will see you there. Thank you for watching. 10. Using Google Docs for Scraping Data: Hey, guys, Welcome back. So now you know how to scrape any website. You can use the tools and actually apply them to websites practice and escape a lot of data . You also understand the fundamental concepts of Web scraping the most important concepts that you need to actually get what you need from the web. The next three trillions are actually just free templates that I'm giving out. And I'm gonna put the links of these with this with the lessons I'm making a common this video, this one video to explain you how all three of them will work. So the next one is basically scraping top Google search results, and I'm gonna follow the methods. Are there different blocks and how to do this? I'm gonna follow the method, which is described in S u thing dot co dot UK. That's the website. I'm falling. I'm gonna put you this Put this link for you. It's a simple matter where they run sort of XML import query on Google sheets and then clean it up and then imported it. Us. You can follow these step by step, However, Like I said, it's a lean course. You don't have to do all of this. There's a link directly to the spreadsheet. You can just make a copy in your Google sheets, and you just have to play the keyboard. It's gonna export the search results for us, so we don't want to waste too much time into this. But well, put the link off this dog and spreadsheets. You can directly get this done. There's nothing really to learn here because it's just a free template. So free resource. Okay, the next one, Number 10 is how scrape data tables from Websites North and you'll have a data in the form out of tables and Web sites, and some people try to copy present tables. But the formatting is not correct. The columns a lot line and the data sometimes get gets messed up. And let's say you're doing some research. You writing some columns about the most spoken languages, in the words. That's the Google search that you want to do. And typically, I love to look at these different tables of statistics on Wikipedia, and I've been doing that for languages as well. I'm interested in learning about this, so now let's say that I want to do some analysis of this and create content out. If it or something like that, I can eat a copy, paste this table in sheets. But the best way to do this the easier, faster way to do this is using this templates again. I'm gonna give you this link off the spear resource, the free template which have already put together on the lesson. How old is actually nothing to do here. It's just a simple cord so important. Ashley Miller is a function that we already have included sheets. And what we're doing is we're leveraging dysfunction to escape tables from different websites, right? It's really as simple as it sounds. So it's equals inboard HTML. These are the three. Battery doesn't be passed on to the function so important. Stream is the function. These are different perimeters. That's all you need to learn that there's no court as such, but the function is important. Female apprentices is where you put in the perimeters. So you have a one table one and what I'm gonna do is to rob dysfunction. I'm just really moved. Of course, double guards an ember. Best enter. And of course, I have to put the link off the website. So let's because it has no deafness. Let's put the link off the website and we're gonna run the function on a one, OK? Because that's the link you are, and that's all. Do you have this this entire table of data which it actually scraped right now from Wikipedia from this link and you can see that it's actually picked up data very clean, and you can filter this and you can use this in any of the analysis. And I think that you want the beautiful thing about this court is which is much more interesting than this is if you have, um, many tables on the on the on the page. So this place, for example, has only one table, I believe one table and some charts. But if you have many tables, you can just for the number of the table you can put to where it's gonna export the 2nd 1 and three, and it's gonna export the third one's whatever tables it finds on the page. It's gonna export and put it all together. So if you have 20 tables on the page, you'll have all of them in this beautiful Google sheet and you can export it and do stuff with it. Right s. So that's the 2nd 1 I'm gonna put a link off the freely sources, which which is just basically the function. You just need to know the function and you can make your own bed sheets of this. The lesson 11 is how to scrape the feed from any website. And this one in this case, when I would do is I want to scrape. I will see what's happening on Reddit. It is always funny what's happening on Reddit on. People are speaking about entrepreneurship and what What are they speaking about? Right, So I need the goddesses, Um, sort of feed or basically any artist is I need the our asses off off the off the threat. So let's search for her harnesses, Freed's and then talked want urbanized speeds. This is where I usually pick up my feet from analysis can be used in different tools to x import exim a landscape XML information. But in this case, I'm gonna pick up the red it Well, so this is about credit RSS feed for entrepreneurs. So this is actually honest. This is like the cord which picks up all of the information on a threat or a block on boats it together in this format, which is used by websites who, like feed, spot another who deliver feeds to your ex. Really? To your phone, your device, or you're or you're next. Eso I'm just gonna copy this goddesses feed, and then I'm gonna go to this grouping sheet. All right? I'm gonna share this Google cheat with you and link. Also, it's another free resource. It has a simple function. Import feed. You are all right. I've already imported this. Let's let's remove this. Let's make a new dove. Okay? Copy the link. Copy the link off the analysis, and it's really hard to use this. So it's import feed. You are also what you want. Type is equals. The function name is import Freed. You can see all of these import functions are basically all scraping functions. That's why you need to think about this import feed and then put the link here and then press enter and that's there you go. You have the names off all the people on Reddit and what they've commented and even the link of the test. So you you have you have just scraped and entire artist since we using just Google dogs. So, yeah, those are some sharp goods guys. And what I want you to the reason why I have these three in my course. I mean, I could have ended the course on lesson eight, but I wanted to tell you that there are so many options now for you to about a next door. You can search for Google. She templates to him to scrape website Gucci temperature scrip, Google search results and Reddit trends and RSS feeds and all kinds of things like that the dour having to write a single line off court. And that's the whole idea off the name of sleeping course. So to conclude, I think I, um I, um, well, put together this sees wanted to put together the CDs for people who don't know how the court, but still get what they need from the Web. And also open up your mind set about the different ways that escaping can happen, be applied and also give you the top to a person, tools, processes and concepts that you need to get 80% off your Web scraping skill set. And thank you for enrolling in this course. You can definitely check out the feet. Resource is it's gonna be there on the course and please go back, see your lessons, practice and stop creating beautiful stuff from the data and applying the data to grow your business or your individual brand eso Thank you so much. And if you have any questions, I'm gonna put my email, I d and you can send it to me. If you like the course, please let me know if you want other lean courses. And what kind of topics are you looking for? I am very excited to make more courses in the city's thank you for being by the scores. Once completed, the scores just send me a mail and I'm gonna send you a certification from the academy from mapping to cut me off course completion. So go ahead and enjoy the new skill off Web scraping. Thank you for enrolling in this court's and see when the next one. Thank you 11. Web Scraping Applications & Ideas: Hello. Welcome back to the lean Web scraping course in this video. I'm gonna quickly talk about the applications and ideas, and I decided to make a new separate video on this so that I can give you new ideas off how you can use the skills that you learn in the scores. All the scripts are that you can do with those skills and how in general can be applied Web scraping So a lot of these things will be able to do immediately after this course and for a few of these things will have to think a little more off how to apply those skills in the right way, and I will give you some direction. So I've made this video because I got a lot of questions about if they want a specific application, how did they get from the lessons of a Blunt learned to that particular application, and I wanted to bridge that gap. So it's only a thinking gap, and it's only limited by imagination what you could do. And you could just sit of scent and make us a good strategy experiment. Executed some experiments, and then you you'll get it yourself but this will give you some push and support to get it faster, you know, to get there quickly. So legion is quite quite a common application of escaping, I should say. And as you've seen in the lesson where I've explained how to scrape company information or how to scrape leads from social media, that's that's gonna help you a lot in terms of legion. So how I I'd like you to think about this is Think about all right. What is the target audience in terms of the leads that I want to generate, and then one of the channels off platforms or directories or blog's or social media channels with this audience hanker typical marketing process. And then you would think about all right, are these places, you know, you make a list of the spaces and say, Which of these places are a scrape herbal? Right, So So you make a nice excel, she to make a list of all the places your target audience hangs out, and then you'd say, Scrape herbal. Yes, no. And then you're gonna pick up those capable ones and start applying your new Web scraping skills. Okay, Next, let's talk about the listing gathering. So there are a lot of these listing websites. It could be restaurant list things or local business listings or educational school listings. Universe feel it's things. And I found a beautiful application in this to help a start up that I was working with. And I was consulting the startup who was looking to make a platform where they would have a restaurant listing a platform for a very specific target audience in a very specific location. And we used a lot of listing scrapes. So we had different platforms where they're stranded already exists and a lot of the state s public. So we would scrape that data, but only for that particular filter which were skating to our target audience and putting all this data from different places would actually result in the start up having all the information that they needed. And we could then dive traffic people to the website. The reason why people were of course not going for all the other directories because they were very generous, They were too generous, and this one was particular catering only to that particular filled up. So I would just go and put a filter on all these websites and then use the scraping methods to get the listings out. And you've seen the example of flip card that I have explained in the e commerce website. So this could also be used for product listings from e commerce websites. So imagine escaping and gathering. I've shown you some examples of how to scrape this from contact pages as zealous from company information pages. You can also look a company directories and scrape this information, however, do remember that in this course I'm only teaching you to scrape information which exists on the website. Of course, there's a lot of scraping that you can do if you get into pages where you're not supposed to get into. Let's put it that way. But that's completely outside of the scope and it's also outside of the terms and conditions off those websites. So not something that I teach in a public course like this. But if you have that information, you can scrape it. So the way I want you to think about this is think about one of the places where I would get this information right so it could be lingering for example, for email addresses. It could be company on listing websites like me. So in the example, and then you would run those skates because it's an example where we actually got email addresses from Facebook. So it could be any channel but not think about one of the places where this image this information that exists. And then instead of manually having to do that, let me run my speaking skills and scrape all that information. All right, so the next does B two b immediacy data scraping. So think about both sides. You might be someone who's working majorly on the beautifully space or in the B two c space . But think about both sides and give it a try. So I know a lot of B two b a legion, said marketing execs who just use my course for scraping business websites and company websites, but lots of fun to try out the e commerce I defend. It's also fun to try out Let's see a simple restaurant listing, for instance, that we discussed holds off under tryout social media scraping like Facebook. Usually the B two B guys would do the linguine scrapes, but skip on the Facebook scrapes. But the reason why I want to diversify a little better because you're scraping skill set is going to improve by trying out different platforms and different websites. And that's what I keep doing when I have time. I'm just gonna go to random websites and scrape them. It's just a hobby, no door, even though it might not be something that my clients are looking for or that I need the data myself because I want to have a variety off scraping experiments. The product reviews scraping is quite interesting because I was working with a client who was making a directory off all the product reviews off one specific product, and he would book put only, of course, the top of the ninth products, Not not something which is like a smartphone, for instance. And then he would scream Amazon and, you know, whatever e commerce websites in his market exists for that particular phone and then put all that data together so that people could see reviews all in one place rather than having to go to all these different websites. Um, and you know, if you're buying from Amazon, you can when you see that on Amazon, but then you can go to this website and you can have all these reviews and then you can also click on those links. And then he would get an affiliate of force for those scents. Pretty decent idea. It's completely curated. Nothing original but works. So that's how you can think about using product reviews. Scrapes scraping for Web, abdomen up Minden Date of his creation. Just like I said Now, a lot of startups that I've helped build their entire databases build an entire APP data just from scraping. So startups usually have this chicken or the egg situation, and you can solve that problem by scraping first. So, for instance, if you're an educational platform and you want to get teachers on board and students on board, but the students are not gonna come, let's let's let's take the example of you did me because all of you know, you Demi simplify things. You want to have courses on board and you want to have be students on wood. But let's say you're starting from scratch. So what do you get for you? Get the students was because, but educators and instructors like me won't come on the back from If no one is gonna see the course, no one is gonna buy the course and students won't come on the on the platform Mondays. They have a variety of beautifully designed and very instructed and logical and, you know, execution ready courses on the platform. So what do you do first? So one thing that you could do is you could have public courses from open courseware, let's say and just scrape all of those courses with permission. Of course, in this case will have to ask the, uh, the instructor. So let's say, for example, if someone comes to my academy directly like, you know, me but a small platform and says Can we take your forces? And I would rather say, You know, why don't you just scream them? I'm giving you permission rather than me having to upload all the files and on your platform again. So that's one way that I could just have a nap. Or you could just have little public listings, scraped them, and then you have the one part of the you know that's the supply side has been sold, and then the demand side can be fulfilled. So that's how a started could actually go with this technique. Social listening. I like this a lot. I would also scrape corrupt threats and tweets for just one understanding how my keepers have been used. And then I would put that all that information into a sentiment analysis off it is it positive is a negative. So those are ways that you can use scraping for social listening for research data. We have a complete course about this so you can refer to that scraping from blogging is interesting. What usually happens is if you're a blogger up, you would have you would either right from the experience or you would write from research . But in all cases, you have to have a lot of data points that you can supply. And a lot of times I've found that the direct graphs that I want to put in my block force, for instance, are not available. The data is not not ready for me because I want to let since point out specifically to a data point which proves the point in my block post and or my podcast or whatever, Ponting. I'm creating But in this case, I'm just mentioning blogging as an example. And then I would just scrape some information, create a nice job Based on that information, I would put it in Google sheets, which I've explained you in the scores. Even I would use that information to have a nice analysis and put it in. The same goes for scraping for analysis as well. So that's the application that you can make with this competitively. Diet is very important. I would usually run the scrape on something like similar web dot com, so I would go similar. Have dot com put in three competitors. Scrape all of that, but everything on a Google sheets. So now you have all the Combinator information together on a cheat Use a sentiment analysis and feedback analysis is something that I would do a lot from my course reviews. Initially, I used to do this and, of course, then once you have the direction, you know how the courses going, and then you don't need to do it any longer, but then you can do it on an individual basis. But let's say I have some free courses. I have 5000 to 10,000 people in the course, and I have 500 to 1000 reviews. Then it's slightly unmanaged me for me to go through every review in the course. I would always go to every of you in the course of a trade course like this one. But for the free course, let's say I have the growth hacking course on you to me, on the on the growth. And of course I have around 5000 to 6000 people and a lot of reviews. A few hundreds, and I would only check it. One survived because I'm not interested in replying to all the reviews and for free courses , at least. So then I would just scrape all that data and that I would put it in the user sentiment analysis, which is a different application letter. Gonna so use it. Sentiment analysis is something you need to learn about data analysis. It has nothing to do with scraping, but then you can look run this data in that machine. So I have that uses sentiment analysis machine, which is basically just a Google sheep. But it has a lot of factors, and in bottoms I don't like to go into Abdel Love meant and according and their bats, So I don't do everything in a lean way, Which is why this lean, Web scraping course. So I have the dire sentiment it answers machine on Google chiefs. I would feed that dies input, which I get from the scraping and then my get my feet back and anuses or use this sentiment forum. Scraping is another interesting one. And the way that I think I would use this and you could also use this is that you would goto different discussion forums and I'll give you an example of Cora because that's pressed meat, something that I use a lot. So I was at some point ranked as the top Ottoman for half ago, tacking. And then I have been used the frequency, of course. But then when I did, is I. I executed a hack where I would see what the top waters in the growth hacking segment and I would scrape all the answers that they everything never right nuances for those questions. But I will make sure that my answer is like 10 times better than their answer, and I would upload that entire thing. Mancora. The upload on Cora thing doesn't happen in one click, so you actually have to answer every question again. But you still have this sheet, which is helpful for it scraping. The interesting thing is you can run these the forum scapes. You can use that for Cuba finances. So now that you have everything in a good sheet, you can. You can do a Cuban anuses. You can count the people frequency in their answers and all those kinds of things. Similar thing off what we discussed on flip card, for instance, you could use that same process to run to scrape the entire product catalog off websites and on all of fashion brands, for example, don't like to give out their product catalogs if you ask them on a contact form. But when you can scrape it, who needs to ask them, anyways, Life. So it's public information. It's all right to escape it and use it for your personal use. Now, social media scraping like we have a complete core video on this in the lesson so so aggregators are essentially the same example that I discussed before about the review. One applies here so I would pick up all these reviews. The other thing that you could do is you could let's say they make a platform just for books. And you could scrape books from Amazon, kindle all those places and then have a website where you can buy the book at the best price. So that's that's That's an example that you could do also off the price comparison website . So that was the price. Combines an example. Scraping for this prevention is a very interesting application, and one of my personal favorites and in fact, scraping sounds like something that you're doing to get the data to use for your benefit. But a lot of times you would need to do scraping for security so you could identify. Let's say, a lot off instances where someone might have used your work outside of your copyright policy so might have used your work outside off. You know, let's say you right of Block or E book, which is a baby book, and the person down does that e book and then just distributes it everywhere. So ever do is I would I would do keywords off my e book and that would run regular scrapes on specific websites where I know that these people might be producing my work so it could be treated, for instance, and then that would prevent my work from getting disputed. Because then I would find all those instances where it's being disputed in a in a wrong way . If the PdF, let's say, is uploaded directly. So what I do is I. I have a list off a book. Websites like Free eBook Download Websites and I would scream them for mighty would search them for my keywords, people the results. And then I would have a link off the person who was posted it in my scraping output and never click on all those links and send a message. You can also automate that process, which is very interesting, but the scraping input is gonna help you do that. First, it's the same thing goes for reputation management. I would scrape threads where people are speaking about me. I would also do this with the course of use on you. That means sometimes you get a negative review, and in most cases people are there is either a misconception or there's something which didn't match that expectations and you can always ask them why the review is negative. So outside of you know me as well a lot of places where I have my content and I would just scrape on the mentions. And I've been fix the things that people have found, um, missing, if at all, any so that the reputation management can happen. So the scraping input is a useful here for s you're dragging in NC escapes again. This is a beautiful application. I can give you an example of Cuba tracking. You could also track your rankings, so not just the with. But you could also scrape search results so that you can see who is ranking where on that gives you a lot of useful information. You can also scrape keyword research websites who don't like to export the keywords directly so you can scrape those websites and then you have all those for ad targeting. I would do this, for instance, on you, Demi, we have ah you Timmy Insights application. There's a lot of websites which have inside stab and there was capable the interest groups from the insights which I can then use import in my Facebook ads, this custom audience did active, so that's an interesting way that you could do scraping for if you're a marketeer. If you're an advertiser, if you're a performance marketeer specifically, this would be very helpful for you. And, of course, you can scrape all kinds of directories. This I found to be by far the biggest pocket off application you do any directory and a directory. Think of directing like a list like a database, and you can scrape all that information and use it the way you want. I would also include another application that you can scrape any tool stack a list of tools , the directory of two of the director of Start Up something that product hand. You could scrape all that information with links and use it for your benefit. So these are the different applications and ideas of Web scraping. I hope you have enjoyed the course, and you also start applying the course and start using different creative ideas for creating businesses for creating startups for building blocks from building content and all kinds of other things that help other people. So thank you so much for watching. That's all for this message