Self-Driving Cars Tutorial: Identify Lane Lines with OpenCV & Python | Rayan Slim | Skillshare

Self-Driving Cars Tutorial: Identify Lane Lines with OpenCV & Python

Rayan Slim, Teacher

Self-Driving Cars Tutorial: Identify Lane Lines with OpenCV & Python

Rayan Slim, Teacher

Play Speed
  • 0.5x
  • 1x (Normal)
  • 1.25x
  • 1.5x
  • 2x
14 Lessons (1h 27m)
    • 1. Introduction

    • 2. Installation - Anaconda

    • 3. Installation - Atom

    • 4. Computer Vision: Overview

    • 5. Loading Images

    • 6. Grayscale

    • 7. Gaussian Blur

    • 8. Canny

    • 9. Region of Interest

    • 10. Bitwise_and

    • 11. Hough Transform

    • 12. Hough Transform II

    • 13. Optimizing

    • 14. Finding Lanes on Video

  • --
  • Beginner level
  • Intermediate level
  • Advanced level
  • All levels
  • Beg/Int level
  • Int/Adv level

Community Generated

The level is determined by a majority opinion of students who have reviewed this class. The teacher's recommendation is shown until at least 5 student responses are collected.





About This Class

In this lesson, you will learn to detect lane lane lines using computer vision techniques. 

Self-driving cars, have rapidly become one of the most transformative technologies to emerge. They are continuously driving our society forward, and creating new opportunities in the mobility sector. 

Learn & Master Automotive-related Computer Vision techniques in this fun and exciting course with top instructor Rayan Slim. With over 28000 students, Rayan is a highly rated and experienced instructor who has followed a "learn by doing" style to create this amazing course.

You'll go from beginner to Computer Vision competent and your instructor will complete each task with you step by step on screen.

By the end of the course, you will be identify to build a lane-detection algorithm fuelled entirely by Computer Vision.

Meet Your Teacher

Teacher Profile Image

Rayan Slim



Hi! I'm Rayan, a full time software developer based in Ottawa, Canada.

I first ventured into development when working on a start-up. Since then, I've built countless web and mobile applications as a freelance developer, meanwhile expanding my repertoire and exploring new avenues in Robotics, Deep Learning & Data Analytics.

In my free time, I love to teach!

See full profile

Class Ratings

Expectations Met?
  • Exceeded!
  • Yes
  • Somewhat
  • Not really
Reviews Archive

In October 2018, we updated our review system to improve the way we collect feedback. Below are the reviews written before that update.

Your creative journey starts here.

  • Unlimited access to every class
  • Supportive online creative community
  • Learn offline with Skillshare’s app

Why Join Skillshare?

Take award-winning Skillshare Original Classes

Each class has short lessons, hands-on projects

Your membership supports Skillshare teachers

Learn From Anywhere

Take classes on the go with the Skillshare app. Stream or download to watch on the plane, the subway, or wherever you learn best.


1. Introduction: welcome to the skill share self driving cars tutorial. In this free course, I will show you how to detect lane lines with open CV and python by making use of various computer vision techniques, as would be done for a self driving car. This is really exciting stuff. But before we dive in, we will need to perform to installations. We will install Python through the Anaconda distribution and then install the Adam text editor if you already have those installed and feel free to move onto the course content. Otherwise I would recommend following along with the 1st 2 electors. All right, I'll see you in there. 2. Installation - Anaconda: welcome to your first lesson. This lesson is quite simple, as all we're gonna be doing is installing the Anaconda distribution. We'll start this off by going over to anaconda dot com slash download. The Anaconda distribution conveniently installs Python, the Jupiter notebook app, which we're going to use quite often throughout this course and over 150 other scientific packages. Since we're installing Python for Mac, make sure to navigate to the max section and we're going to install Python three, not Python to. It's very likely that you're seeing different versions of python and the ones I'm seeing right now. But regardless, since Python to is no longer being updated, it's imperative that you download the latest version Python three, to keep compatibility with future python improvements and to also follow along with this course, click on the download button. No thanks. Once your download is finished, will open this package continue. Keep pressing. Continue. Continue. Agree to the terms and conditions that you have, read the agreement and simply enough press install. Once your installation is complete, press continue again close and move to trash. That is all very easy and intuitive to ensure that this works whatever terminal window you had previously opened, make sure to close its already had one open sol close mine. If you don't have one open, then you should be fine. What we'll do is we'll open up a new terminal window by performing Spotlight Search, which you can do by pressing command and space. Right terminal and press enter. Alternatively, you could have just access your terminal by pressing on a four and performing the search here. But regardless to ensure a successful installation, right, the Command Python three double dash version and I get a version of 3.6 point four. Make sure you also get a python three version that is all hope you were able to follow along. If you have any issues with the installation, feel free to ask me in the Q and a section 3. Installation - Atom: Welcome back. We'll be making use of the atom text editor in the computer vision section. You can feel free to use any text editor you want, like sublime or vim, in which case feel free to skip this lesson. Otherwise, if you don't have a text editor installed, let's get to it. Downloading it is quite simple for both Mac and Windows. I'll be proceeding with the Mac installation by going over to adam dot io and inside of Adam that I owe just clicking download. This is also downloadable from Tech Spot by also going over to Google and then searching Tech Spot Adam download and going into the first link where if you just simply click on the appropriate link and then wait for it to finish setting up the download whether you're on Mac or Windows, All right. Once your installation is complete, it should be inside of your downloads folder. Let us just open up Adam and see what it's like. It's verifying this new application, going to open it, All right, We're gonna close the following, and then what we'll do is click on packages and inside of packages. We're gonna go to settings you and open the settings. If you're using a PC, I imagine the process to get to your settings. You should be quite similar. And now inside of editor, we can modify some settings. For example, the font family I'm quite satisfied with the current front and fought size 16 seems pretty reasonable. And two other boxes I like to have checked are showing the cursor on selection and showing indentation indicators. Indentation is fundamental to python, a code as it distinguishes between different blocks of code, so it will definitely be useful to always have indentation indicators. And finally, one more thing that's pretty important is the tablets. The default tab links should be to, I believe, but I personally prefer using four spaces per tab. And just one more thing I want to do before moving on is enabling auto save. So we'll go back to package. And this will be very convenient than our code, since it would keep us from having to save our code every time we need to run its So what will their was inside of core packages Scroll down till you find auto safe right over here and make sure that you enable it's already have it enabled all rights and that includes the installation section 4. Computer Vision: Overview: Hey, welcome to the self driving cars course, although the main focus of this course is deep learning will be making use of an open source computer vision library in many parts of the course, namely a library notice Open CV. Ultimately, we will end up using open CV, the pre process training data before it is fed into a deep neural network. That being said, what better way to get familiar with open CV and to use its identify lane lines for a self driving car? One thing to notice that will be making use of the python programming language and numb pie . If you're not familiar with some pie or python that, in that case it would be best if you checked out the complete self driving cars course on you to me, as it includes crash courses on both concepts, as well as 18 hours of content where you get the building fully functional self driving car . If you are interested, feel free to use the promo code skill share to get your 94% discount. Otherwise, if you are familiar and just want to proceed with finding lane lines for a car using open CV, then let's dive right in. I'll see you in the next lesson 5. Loading Images: Welcome to Lesson One of this section Going forward will be using a text editor whether it's Adam Sublime. Whatever you choose to work with. If you still don't have a text editor installed, make sure to refer back to the text editor installation tutorial in Section two before proceeding any further. The purpose of this section is to build a program that can identify lane lines in a picture or a video. When you and I drive a car, we can see where the lane lines are using our ice. A car doesn't have any ice, and that's where computer vision comes in, which through complex algorithms helps the computer see the world, as we dio in our case, will be using it to see the road and identify lane lines in a series of camera images. This lesson in particular will be quite simple, as all we're gonna do is set up the initial stages of our project and display an image onto which will be identifying lane lines. You'll start by opening up your terminal or command prompt and navigate Judy Desktop Directory with the Command CD. This top change directory does stop inside of desktop will make a new folder with the command em que de ir make directory finding lengths this folder you just made inside of desktop. You're going to open up with Adam by going to file open desktop Finding Leight's Inside of Finding Lanes Making New Python file called lanes dot p y inside of the Lanes file. We're going to start by running a program that can identify lanes in a J. Peg image. I posted the image on my get hub taxes set. Make sure to go to the following link. And so once you get to this, get her page click on test image dot jpeg, and what we're gonna do is actually download the image. Or, better yet, just saved the image and make sure to save it as a J peg image. It doesn't matter where you save it and make sure this says test image such that there are no extra ones or twos in there. So that you're naming remains consistent with what I have in the videos. All right, save your image. Once your downloaded. Wherever you have it downloaded, make sure to drag it into your project folder like so and now it to display the image we're going to use open CV, An open source computer vision library. So inside of your terminal, we're going to write the command pick. Install will make use of the package Manager Pip to install open CV Baskin Trib Dash python . Once you're finished installing at back to visual studio, we're going to import the library CV to and from this library. For now, we will access to functions Emery did and m show to load our image will first make use of the Emory function by setting image is equal to c V two dot m. Read. And in this argument is where you will specify the image the file name as a string. Ours is a test image dot jpeg and what this function will do is read the image from our file and return it as a multidimensional, numb pyre. A containing the relative intensities of each pixel in the image, we now have our image data in a numb pyrite. The next step is to actually Orender it with the M show function. So we're right CV to dot m show. This takes in two arguments. The 1st 1 is the name of the window that we're going to open up. We'll just call it result. And the second argument is the image that we want to show itself. If I run the code now to my terminal, navigate to my Project folder by writing CD. Our project is named Finding Lanes and Run the Python file Python Elaine's dot p y. Notice that nothing is going to happen. That's because this function should be followed by the weight key function CV to dot wait key and what this function does that displays the image for a specified amount of milliseconds. We'll set a time of zero. What this will do is it will display our window. Our result toe window infinitely until we press anything in our keyboard. If we were in the code Python lanes, the image is displayed and notice our window name results will keep this lesson shortened. Stop here. You learn how to load and display images using the open CV library. In the next lesson will start discussing canny edge detection, a technique that will use to write a program that can detect edges in an image and thereby single out the lane lines 6. Grayscale: Welcome to lesson number two. The goal of the next few videos will be to make use of an edge detection algorithm. The canny adds detection technique. The goal of edge detection is to identify the boundaries of objects with an image. Is, in essence, will be using edge detection to try and find a regions in an image where there is a sharp change in intensity, a sharp change in color. Before diving into this, it's important to recognize that an image can mirror. It is a matrix, an array of pixels. A pixel contains the light intensity at some location in the image, each pixels intensity denoted by a numeric value that arranges from 0 to 55. An intensity value of zero indicates no intensity if something is completely black, whereas to 55 represents maximum intensity, something being completely white. That being said ingredient is that the change in brightness over a series of pixels a strong radiant indicates a steep change, whereas a small Grady INTs represents a shallow change. On the right hand side, you're looking at the greedy int of the soccer ball. The outline of white pixels corresponds to the disk continuity in brightness at the points , the strength ingredients. This helps us identify edges in our image, since an edge is defined by the difference and intensity values in adjacent pixels and wherever there's a sharp change in intensity, a rapid change in brightness. Wherever there is a strong radiant, there is a corresponding bright pixel in the Grady Int image. By tracing out all of these pixels, we obtain the edges. We're going to use this intuition to detect the edges in our road image. This is a multi step process step one being to convert our image to gray scale. Why converted to grayscale? Well, as we discussed earlier images, air made up of pixels, a three channel color image would have red, green and blue channels. It's pixel, a combination of three intensity values, whereas a grayscale image only has one channel each pixel with only one intensity value. Arranging from 0 to 55 the point being by using a grayscale image. Processing a single channel is faster than processing a three channel color image and less computational intensive. Let's start implementing this inside of Adam. We've already loaded and read our image into an array now What we'll do is import num pie as the alias and P. We're going to work with a copy of this array. By setting lane. Image is equal to numb pie dot copy image, thus copying our array into a new variable. It's imperative that you actually make a copy of the Saray instead of the setting lane. Image is equal to image. If we do this, any changes we make. Two lane image will also be reflected in the original mutable array. Always ensure that you make a copy whenever working with a race instead of just setting them equal directly. So what we'll do now is will create a gray scale from the color image will do that by setting a variable. Gray is equal to CV to and from our open CV library will call the function CVT. Color, which converts an image from one color space to another, will be converting lane image. And in the second argument for an RGB two grayscale conversion, we can use the flag CV to dot color underscore RGB two gray very intuitive. And now, instead of showing the color image will show the gray scale image. If we go to our terminal. Python leans the P y. Everything works up accordingly. This was step number one. Step number two of edge detection will be to next apply a Gaussian blur on this image. Let's talk about that in the next video. 7. Gaussian Blur: welcome to us, a number three. In the last lesson, we applied step number one, which was to convert our image to gray scale. Step two is to now reduce noise and smoothing our image when detecting edges. While it's important to accurately catch as many edges in the images possible, we must filter out any image noise. Image noise can create false edges and ultimately affect adds detection. That's why it's imperative to filter it out and thus smoothing the image. Filtering out image, noise and smooth inning will be done with a Gaussian filter. To understand the concept of a Gaussian filter, recall that an image is stored as a collection of discrete pixels. Each of the pixels for a grayscale image is represented by a single number that describes the brightness of the pixel. For the sake of example, how do we smooth in the following image? The typical answer would be to modify the value of a pixel with the average value of the pixel intensities around it. Averaging out the pixels in the image to reduce noise will be done with a colonel. Essentially, this kernel of normally distributed numbers is run across our entire image and sets each pixel value equal to the weighted average of its neighboring pixels, thus smoothing our image. We're not going to go over Colonel Convolution and how it doesn't just know that when we write this line of code inside of our editor, Blur is equal to C v two dot Gaussian Blur. What we're doing is applying a Gaussian blur on a grayscale image with a five by five colonel the size of the colonel's dependent on specific situations. A five by five colonel is a good size for most cases, but ultimately what that will do is returning new image that we simply called Blur. Applying the Gaussian Blur bike, involving our image with a kernel of Gaussian values, reduces noise in our image. Backed our project set Blur is equal to C V two dot Gaussian Blur will apply this blur in our gray scale image with our five by five Colonel, and we'll just leave the deviation a zero. The main thing you should take away from this is that we're using a Gaussian Blur to reduce noise in our gray scale image and now will simply show the blurred image. If we run this code python lanes dot p y. There is are blurred grayscale image. Later on, when we apply the canny method, it should be noted that this step was actually optional. Since the candy function is going to internally apply a five by five Gaussian when we call it regardless, now we know the theory. We've obtained our gray scale, we smooth and it and reduce noise with a Gaussian blur. Now it's time to apply the candy function. We'll do that in the next video. 8. Canny: in the last lesson, we smoothen our image and reduced noise. Now it's time to apply the Kenny method to identify edges in our image. Recall that an edge corresponds to a region in an image where there is a sharp change in intensity or a sharp change in color between adjacent pixels in the image. The changing brightness over a series of pixels is the ingredient. It's a strong radiant indicates a steep change, whereas a small, radiant, a shallow change. We first established that an image as it's composed of pixels can therefore be read as a matrix, an array of pixel intensities to compute the Grady INTs in an image one must recognize that we can also represent an image in a two dimensional coordinate space accent. Why the X axis traverse is the images with and the Y axis goes along. The images heights with representing the number of columns in the image and height. The number of rows, such that the product of both with and height, gives you the total number of pixels in your image. The point being Not only can we look at our images an array but also as a continuous function of X and y. Since it's a mathematical function, we can perform mathematical operation. Which begs the question. What operator can we use to determine a rapid changes in brightness in our image? What the candy function will do for us is performing derivative on our function in both accent y directions there by measuring the change in intensity with respect to adjacent pixels. A small derivative is a small change in intensity, whereas a big derivative is a big change by computing the derivative in all directions of the image were computing the Grady INTs. Since recall, the Grady int is the change in brightness over a series of pixels. So when we call the Kenny function, it does all of that forests. It computes the Grady int in all directions of IRA blurred image and is then going to trace our strongest radiance as a series of white pixels. But notice these two arguments low threshold and high threshold. Well, this actually allows us to isolate the adjacent pixels that follow the strongest radiance if the ingredient is larger than the upper threshold than it is accepted as an edge pixel. If it is below the lower threshold, it is rejected. If Teague radiant, is between the thresholds, then it will be accepted on Lee if it is connected to a strong edge. The documentation itself recommends to use a ratio of 1 to 2 or 123 as such will use a low , high threshold ratio of 1 to 3 51 50 Now that we know what goes on under the hood, we can call this function inside of our project by writing. Kenny is equal to C V two. Duh. Canny will apply the Kenny method on the blurred image with low and high threshold of 50 and 1 50 and now we'll show the image Grady Int instead of the blurred image. Kenny. If we go ahead and run the code, Python leans that p y. And there's the Grady Int image, which clearly traces an outline of the edges that correspond to the most sharp changes in intensity. Grady INTs that exceed the high threshold are traced as bright pixels identifying adjacent pixels in the image with the most the rapid changes in brightness. Small changes in brightness are not traced at all, and accordingly they are black. As they fall below the lower threshold. That's it for the candy method. We used it to outline the strongest radiance in our image. Now. This was a lot of explanation for just three lines of code, but it's good to have a grasp of what's going on under the hood before we proceed any further. Now that we've computed the strongest Grady INTs in the next few videos will apply the huff transform method to detect our lanes. 9. Region of Interest: in the last lesson. We used the candy function to outline The strongest radiance in our image now will focus on how we can identify lane lines in the image before doing that. What we'll do now is specify a region of interest in our image that we're going to use to detect our lane lines as currently shown before proceeding any further. What will do first is actually wrap our code inside of a function by defining a function. Kenny, which takes in an image and what we'll do is copy the canny algorithm code and son of the function and specify Kenny as the return value return Kenny and what we can do now is simply set can be equal to the return value of our candy function. Passing in the initial RGB color image and re running this code back into our terminal Python leans that P y everything is still intact. What we'll do now is will specify this area as a region of interest before doing so, instead of showing our image creating with open CV will use theme at plot led library to better clarify how we're going to isolate this region. You should already have in that plot live installed, courtesy of the anaconda distribution. So what we can do now is imports Matt plot lib, and we're going to need the sub package from APP lot lib called pie Plot as an alias. Plt conveniently pie plot contains the function and show so we can just replace CV to with plt and in this case, no need to specify a window name. Just the image the equivalent of open sea visa weight key function would simply be plt dot show figure about gonna run the code in our terminal python lanes that p why we get the same image along with X and y axes. Notice how the Y axis starts from the first row of pixels and then goes downwards with our axes were going toe limit the extent of our field of view based on their region of interest which ultimately traces a triangle where the verdict use of 200 along the X and 700 pixels along the why which would simply be the bottom of the image 1100 pixels along the X and once again, the bottom of the image 700 pixels at the Why the very bottom and the last. Vertex will simply be 550 pixels along the X and 250 traveling down the why, ultimately tracing a triangle that isolates the region where everyone identify the ley lines. So the goal of this video will be to create an image that's completely black, a mask with the same dimensions, our road image and fill part of its area where they triangular polygon first and foremost will revert, showing the image to being done with open CV rather than Matt Plot lib. And what we'll do now is defined a function deaf region of interest, which also takes in an image. And what this function will do is pretty self explanatory. It will return the enclosed region of our field of view and recall that the enclosed region was triangular in shape. So we'll set a variable name triangle is equal to, and this polygon this triangle will declare is a numb pie array and p dot array inside of the Serie is very specified Vergis. He's recall that while limiting the extent of our field of view, we traced a triangle with Vergis Is that go 200 along the X and vertically until the extent of our image until the bottom, which in this case is the height we can get the height of our image by setting height is equal to image dot shape at the index zero Recall from the numb Pie Crash course that the shape of an array is denoted by Tupelo vintage er's. Since we're dealing with a two dimensional array the first and injure corresponding to the number of rows the Y axis traverse is the images, heights and height. Accordingly is the number of rows. So you can already assume that this value will be something very close to 700 since that's what we saw in my plot lib. So I will simply set the height to complete our first of Vertex, the 2nd 1 being 1100 pixels along the X and vertically once again, right up until the extent of our image being the height, the vertical extent and lastly, the last Vertex was 550 pixels along the X and 250 pixels along the Why this polygon. We're going to apply it onto a black mask with same dimensions as our road image, so we'll sent. Mask is equal to numb pie that zeroes like image recall that an image can be read as an array of pixels. Zeros like creates an array of zeros with the same shape as the images corresponding array . Both arrays will therefore have the same number of rows and columns, which means that the mask will have the same amount of pixels and thus the same dimensions as our candy image that we're going to pass it in a bit, although it's pixels will be completely black as all of them will have zero intensity. What we have to do now is fill this mask. That's, um, black image with our polygon using open Sea V's Phil Polly function that is CV to the Phil Polly. We will fill our mask with our triangle. The third argument specifies that the color of our polygon, which we're gonna have be completely white. So what we're gonna do is take a triangle whose boundaries we defined over here and apply it on the mask such that the area bounded by the political contour will be completely whites. Well, we'll do now is return our modified mask return mask. And instead of showing the Kenny image will be showing the return value of our function region of interest and the image that we're gonna pass It is simply going to be the canyon itch. Let's run the code. That's a run python Elaine's dot p y. And it would throw an exception. And that's because Fell Polly the Fill Polly function fills an area bounded by several polygons, not just one. Even though you and I both know that we're dealing with only single polygon will rename this variable from Triangle two polygons for consistency, and we'll set it equal to an array of polygons. In our case, an array of simply one polygon change this from Triangle two polygons. If we run this code, there is our mask. And inside of the mask there is the enclosed the region, the polygon with the specified vergis ease. Now you might be asking yourself, Why did we go through all of this? Well, this video has gone on long enough, so let's talk about that in the next lesson. 10. Bitwise_and: previously, we created a mask with same dimension, deserve a road image. We then identified a region of interests and have road image with very specific verdict is along the X and Y axis that we then used to fill our mask, the image on the right. Why is it important? While we're going to use it to only show a specific portion of the image? Everything else, we want a mess. So it's understand how we're going to use this image to mask our canny image. Toe only show the region of interest traced by the triangular polygon. You'll require a basic understanding of binary numbers. If you're already familiar with binary numbers, feel free to skip the next two minutes or so of this video. Otherwise, I'll introduce it now, and I'll do it very quickly. Commonly, when one thinks of Bynum representations, they think of zeros and ones well. More specifically, binary numbers are expressed in the base to numeral system, which uses Onley two symbols, typically zeros and ones. What does that mean? For example, the number 23 it's a binary representation is 10111 How did I obtain that number? Well let's imagine eight placeholders eight boxes As we're dealing with a base to numeral system. Each box represents a power of two. Each numerical place will correspond to an increasing power of two. The first box two to the power of zero. What simply equals one, then two to the power of one which equals two all the way up until two to the power of seven. Up until 128. Welcome back to this. Now each box can Onley except one of two values zero or one. So we want to put in binary the number 23. We wish to represent this number in binary format. So what we do is we start with the highest value 128th and ask ourselves, Is this value in the number 23 120? It is clearly not in the number 23. It's way too big. So we leave that 0 64 is not in 23. We also leave that a zero 32 is certainly greater than 23 were. Leave that a 0 16 This goes into 23 so it's assigned the number one and so far we've used up 16 23 minus 16 Equal seven There. Seven left. We have to account for this. Eight going to seven? Nope. We leave that a 04 going to seven. Of course, that takes a value of one. So now we've used the 47 minus four equals three and there's three left that we have to account for is to go into three. Yeah. Now there's only one left. Does one go into one? Indeed. And there is the binary representation of 43. Cut off the zeros in the beginning. And it's just as we said earlier. 10111 All right, so why did I just read only start talking about binary numbers while the image on the right I went ahead and printed at its pixel representation? I resize the array simply because it was too large. But never mind that. Notice how the triangular probably gone translates the pixel intensities of 2 55 And the blocks surrounding region translates the pixel intensities of zero. What's the bind? Every representation of zero. While none of these numbers go into zero, so we leave zero for each placeholder, leaving us with a binary representation of 0000 What about 2 55? Well, for that we'll need eight placeholders, 220 up until two to the seven. And so, if you do the math we just talked about, you realize that it's buying. Every representation is all ones eight ones, as all of these numbers add up exactly to obtain to 55 as a side note. If we think of this in terms of bits where each bit holds a single binary value at a bits form one bite 2 55 is actually the maximum represent able value by an eight bit byte, so we can conclude. Since the surrounding region is completely black, each pixel with a value of zero than the binary representation of every pixel intensity in that region would be all zeroes 0000 As for the political contour whose region is completely white than the binary representation of every pixel, intensity in that region would be all one. Why is this important? Well, we're going to apply this mask onto our Kenny image to ultimately Onley show the region of interest the region traced by the political contour we do this by applying the bit wise and operation between the two images. The bit wise and operation occurs element wise between the two images between the two arrays of pixels. Now both of these images have the same array shape and therefore the same dimensions and the same amount of pixels. By applying the bit wise end, since it occurs element wise than we're taking the bit wise. End of each homologous pixel in both the race and the way bit wise end works is, let's imagine to binary numbers 011001 which, if you do the math you saw earlier, you'll realize, is the number 25 and 110010 which would be the number 50. This is a pretty standard example, but regardless, let's take their bit wise. And And what end Will dio is? It puts a zero unless both pairs are once in the first pair. Since one of these zero we put a zero and the 2nd 1 both of them are ones. So we put a one and this 11 of them is zero. So we put a zero and we keep doing this until Eventually the resultant and operation would yield 010000 What if we took the bit wise end of all zeros with any other value? Well, no matter what, during the operation, you're always going to have at least 10 which means the result of the end operation will yield all zeros, no matter what value would choose to operate against it. So going back to our two images, the black region was pixels of intensity values which correspond to the binary number. We just talked about 0000 by taking the bit wise and by operating it against the pixel values in the corresponding region of the other array, the result is always going to be a binary value of 0000 This translates to the number zero , which means all pixel intensities in that region will have a value of zero. That's what the results gonna be. They will be completely black, thereby masking the entire region. We know the operation occurs element wise, so all the white pixels in this region of the array will be operated against the corresponding region of the other array. Well, this region will remain unaffected. Why, you might ask. Well, we already concluded that since the political contour is completely white than the binary representation of each pixel, intensity in that region would be all once. If you take the bit wise and of the ones with any other binary value, it's not going to have an effect. We could try this out as we take the bit wise. And of these two values two ones. So we put a one here, same case here, here and here. And we keep doing this step to in the following results. Which notice is the same as one of our values, meaning that taking its bit wise and with ones didn't have an effect. And so in our image, taking the bit wise end of these two regions would also have zero effect, which means we've successfully masked our canny image to ultimately Onley show the region of interest the region traced by the political contour. We can implement this by setting mast image is equal to C V two dot bit wise and and will compute the bit wise end of both Decani and mask a race. This universal function implements the python operator and finally will return masked image and we'll just set. Cropped image is equal to the return value of region of interest will pass in the Kenny image, as we did in the previous video, and we'll show the cropped image instead back to our terminal python lanes that p Y and everything worked out. Accordingly, we isolated the region of interest and masked everything else. The final step of lane detection will be to use the huff, transform technique to detect straight lines in our region of interest and thus identify the lane lines. 11. Hough Transform: So far, we've identified the Anderson our image and isolated the region of interest. Now we'll make use of a technique that will detect straight lines in the image and thus identify the lane lines. This technique is known as Huff transform. We'll start by drawing a to d co ordinate space of X and Y and inside of it a straight line . We know that a straight line is represented by the equation. Why is equal to MX plus B? Nothing new so far. Just simple math. Our straight line has two parameters. M and B. We're currently plotting it as a function of X and y, but we can also represent this line in Parametric space, which we will call Huff Space as be versus M. We know the Y intercept of this Linus to and the slope of the line is simply rise over. Run the change in y over the change in X, which evaluates to three given the Y intercept and slope, this entire line can be plotted as a single point in huff space. Now imagine that instead of a line we had a single dot located at the coordinates 12 and two. There are many possible lines that can pass through the Stott each line with different values. For M and B, you could have a line that crosses it with them and be values of two and eight, three and six, four and four, five and 26 and zero. So on and so forth, I noticed that a single point in X and Y space is represented by a line and huff space. In other words, by plotting the family of lines that goes through our points. Each line with its own distinct M and B value pair, this produces an entire line of M and B value pairs and huff space. What if we also had a point at eight and one? Once again, there are many lines that can cross this point. Each line with different values for M and B, all of these different values for M and be represented by a line in Parametric space. The point being whenever you see a Siris of points and we're told that these points are connected by some line, ask yourself this question. What is that line? As previously mentioned, there are many possible lines that can cross each point individually each line with different slope and wider sub values. However, there is one line that is consistent with both points. We can determine that by looking at the point of intersection and huff space, because that point of intersection represents the M and B values of a line consistent with crossing both of our points, which in this case has slope and why Intercept of four. Suppose there is one more point than our image space at the 10.0.16 and three. This point is also represented by a line in Parametric space. Each point in that line the notes, different values for M and B, which once again correspond to different lines that can pass through this points. But notice that there is another intersection at the same point. Which means that the line with the following slope and why intercept foreign four crosses all three of our dots. Why is this relevant? Well, this idea of identifying possible lines from a series of points is how we're going to find lines in our ingredient image. Recall that the Grady Int image is just a series of white points which represent edges in our image space, you and I can look at the very Siris of points in our image and automatically assume these points belonged to a line. This series of points belongs to align so on and so forth. But what are the lines? What are their parameters? How do we identify them? Well, take these four points, for example, in our image space which correspond to the following huff space. What we're gonna do is first split our have space into a grit each been inside of our grid corresponding to the slope and y intercept value of a candidate line. For example, What if I told you these points belonged to align? What is that line? Well, I can see that there's points of intersection here. There's some here, and some here is well, will all of these points of intersection are inside of a single bitten for every point of intersection. We're going to cast the votes inside of the bin that it belongs to the been with the maximum number of votes. That's gonna be your line, whatever m and B value that this been belongs to. That's the line that we're going to draw. Since it was voted as the line of best fit in describing our data. Now that we know the theory of how we're going to identify lines in our Grady in image, you would think to yourself. All right, enough talking time. The code. Well, not so fast. There is just one tiny problem. We still haven't taken into account of vertical lines. Obviously, if you try to compute the slope of a vertical line, the change in X zero, which ultimately will always evaluate to a slope of infinity, which is not something that we can represent enough space. Infinity is not really something we can work with anyway. We need a more robust representation of lines so that we don't encounter any numeric problems because clearly this form wise equal to MX plus B cannot represent vertical lines . That being said, instead of expressing our line with Cartesian coordinate system parameters, M and B will instead express it in the polar coordinates system row and data such that our line equation can be Wrana's row is equal to X Coast data plus y sign data. If you're really interested in how this equation is derived, feel free to ask me in the Q and a But the main idea is that this is still the equation of a line, but in polar coordinates, if I am to draw some line in Cartesian space, the variable row is the perpendicular distance from the origin to that line, and data indicates the angle of inclination of the normal line from the X axis, which is measured in radiance clockwise with respect to the positive X axis. Let's look at some examples. Suppose I had a point with Exposition five and Wise equal to two. As you know, this is a point, and many lines can pass through this point, including a vertical line we used to define lines passing through our points by their slope and why intercept and and be. But now they will be the fine based on a row and data if we want to measure the perpendicular distance from the origin to the top of the line. The angle of inclination of the normal line from the access is simply zero, which works out to a distance of five. Another possible line that could pass through our point is a horizontal line. The perpendicular distance from the origin to our line would correspond to an angle of 90 degrees, which in radiance it's pi over two, which ultimately works out to a distance of two. This line, therefore, is characterized by an angle theta of pi over two and a distance, wrote of two, just to strengthen our knowledge. Another possible line is the following, whose perpendicular distance from origin toe our line corresponds to an angle of 45 degrees from the positive axis that is pi over four radiance, which works out to a distance a row of about 4.9. The point of all this being is that previously a point. An image space represented a line in Huff space, whereas now, with polar coordinates for a given point by plotting the family of lines that go through it , each line with a distinct value for theta in row, we get a Sinus Auteuil kerf. This curve represents all of the different values for a row and data of lines that pass through our points. This might look a bit intimidating, but the concept is the exact same. Because imagine, instead of one point we had 10 points, which in turn results in 10 Sinus total curse as previously noted. If the curves of different points intersected, huff space, then these points belong to the same line characterized by some roe and data value. So this, like before a line, can be detected by finding the number of intersections between curves. The more curves intercepting means that the line of represented by that intersection crosses more points. In our case, all 10 of our curves intersect that a single points, which means that there is a single line with some Moreau and data value that crosses all 10 of our dots. We can look at a more specific example with three dots in our Cartesian, which represent the following signer Total curves. All three lines intersect at the same point, characterized by a theatre value of 0.92 radiance and perpendicular distance of about 9.66 So that's the idea finding which line best fits our data. In our case, it's the one with the following parameters. We can also apply the concept of voting that we discussed earlier, such that our half space is still in the form of a grid, and obviously this been would have the maximum number of votes and just like before the been with the maximum number of votes. That's going to be your line, whatever theta and our valley that this been belongs to. That's the line that we draw, since it was voted as the line of best fit in describing our data later on, when we start implementing, this will talk about the concept of thresholds. But for now, that is all for have transform. We're just trying to find the lines that best describe our points, and that's what we're going to use to find the lines that best defined the etch points in our Grady INT image. Let's start implementing that in the next video. 12. Hough Transform II: previously, we looked at the theory behind detecting possible lines from a series of points by looking at the been with the maximum number of votes that is the maximum number of intersections inside the bin. We discussed that the been with the maximum number of votes is the line we draw through a series of points, whatever theta in our value that this been belongs to. That's the line we draw. Since it was voted as the line of best fit in describing our data will start implementing this by detecting lines and the cropped ingredient image by setting lines is equal to C V two that Huff lines p. The first argument is the image where you wanted to tuck lines, which would simply be our cropped image. The second and third argument specified the resolution of the Huff accumulator array, the half accumulator array. Previously I described as a grid for simplicity, but it's actually a two dimensional right of rows and columns which contained the bends that we're going to use to collect votes with each been representing a distinct value of row in data. The second and third arguments are really important as they specify the size of the bends Row is the distance resolution of the accumulator and pixels, and data is the angle resolution of the accumulator and radiance. The larger the bins, the less precision in which lines are going to be detected. For example, imagine every been an hour rate was so large. This is way too course, in the sense that too many intersections air going to occur inside of a single been. We need our bids to be sufficiently small. The smaller the row and degree intervals we specify for each bend, the smaller the Benz and the more precision in which we can detect that reliance. Yet you don't want to make your bins too small, since that can also result in inaccuracies and takes a longer time to run. So what we'll do is we'll specify a precision of two pixels, accompanied by a one degree precision that's needs to be in radiance. 180 degrees is equal toe pie Iranians, So one degree will simply be pi over 80. That is, numb pie, the pie divided by 1 81 radiant to demonstrate the effect of this early on, here is a sneak peek of the end results when we finally detect our lines. In this picture, the been resolution was a row of 20 pixels and five degrees, whereas here it's two pixels with a single degree precision. Clearly, this one is much more precise in its output. So that's it for resolution. The fourth argument is very simple. It's the threshold to find and displayed the lines from a series of dots. We need to find the bends with the highest number of votes, right. Once you find that Ben, you take its data and Roe Valley and plot the line, However, how do we know what's been is to choose. What's the optimal number of votes where we can say, OK, draw the line that corresponds? Did this been well? That's where the threshold comes in. Threshold is the minimum number of intersections needed to detect a line, as previously mentioned in a series of points, the points of intersection and huff, space representative data and row values of lines that are common between a series of points for every point of intersection of votes cast inside of the bin that it belongs to, which represents a line with some value for our own data. There's five intersections here, so five votes and we assign a threshold of three. The number of votes in this case exceeds the threshold, and it is therefore, accept it as a line that describes our series of points. If we assign a threshold of some large number, let's say 12. In this case, we don't have sufficient intersections in our bin to see that the line belonging to this been describes our data and is therefore rejected. In the case of Iraq, radiant image will have a threshold of 100 which I found to be an optimal value, such that the minimum number of intersections and have space for a been needs to be 104 to be accepted as irrelevant line in describing our data. The fifth argument is just a placeholder array, which we need to pass in. So just declare an empty your right, no much to it. The sixth argument is the length of a line in pixels that we will accept into the output, which will declare is a key word argument. Men line length is equal to 40 so basically any detected lines traced by less than 40 pixels are rejected, and lastly, there is the Max Line Gap keyword argument, which we're going to set equal to five. This should be capitalized. This indicates the maximum distance and pixels between segmented lines, which we will allow to be connected into a single line instead of them being broken up. That's all guys we just set up in algorithm that can detect lines in our crop to Grady int image. Now comes the fun part, which is to actually display these lines into our real image. What we'll do is we'll define a function death display lines, which takes in an image onto which will display the lines as well as the lines themselves. And before giving this any logic, we're gonna go right back here and set line image equal to the return value of our function , which we're going to specify momentarily. But we'll just set it equal to the return value. For now, display lines pass in our lane image as well as the detected lines backed. Our function, similar to what we have in region of interest, will declare an array of zeros. Line image is equal to n p dot zeros like with the same shape as the lane images corresponding array. So it will have the same dimensions as our image, although it's pixels will be completely black, as all of them will have zero intensity. And now all of the lines that we detected in our Grady INT image will display them onto those black image. This is a three dimensional array. The check if it even detected any lines we have to check. If the array is not empty, that is, if lines is not. None, if it's not empty, will loop through it. Four. Line in lines. If you print each line that we iterated through print line back to our terminal run python lanes that P Y if you print each line notice. Each line is a two dimensional array with one row in four columns. What we're gonna do is a reshape every line into a one dimensional array. Such that line is gonna equal line reshape, and we're going to reshape it into a one dimensional array with four elements. And you know what instead of setting line is equal toe line that, reshaped for weaken, simply unpack the array elements into four different variables. X one y one x two y two. And now what Will Dio is. Take each line that we're iterating through and draw it onto our blank image. Thanks to open CV, we can write CV to dot line. This function draws a line segment connecting two points. Well drew our lines on the line image, the black image we just created. The second and third argument specify in which coordinates of the image space that we want to draw the lines. So the second argument will be the first point of the line segment, which is simply x one y one and the third argument as the second point of the line Segment X two. Why, too? All right, So we've specified the coordinates in which we want our lines to be drawn with respect to the image space. The next argument is what color we want the lines to be, So we'll specify a B G R color of 2 55 zero and zero. This should result in a blue color, since the red and green channels will have zero intensity and finally the line thickness, which is going to have a value of 10. Obviously, the higher the value, the thicker the lines. And that is all all of the lines we detected in the Grady INT image in our cropped image. We just drew them onto a black image, which has the same dimensions is our road image. Now let's just go ahead and return the line image return line image. And over here we're going to show the line image instead of crop damage. And back to our terminal will rerun our code python lanes that p Y. And as expected, it shows the lines that we detected using half transform, and it displayed them on a black image. The final step is the blend this image to our original color image. That way the lines show up on the lanes instead of some black screen. So what we'll do is we'll go back to our code and we'll set combo image. I'm sure I could have come up with a better name than that Combo images people two CV to dot add waited. And so what we're gonna do with add waited is take the sum of our color image with our line image. What should now make sense to you as to why the background of line image is completely black, since that would signify pixel intensities of zero and by adding zero with whatever pixel intensities air inside of this image inside of lane image, the pixel intensities for that image would just stay the same. It wouldn't change zero. Plus anything doesn't really make a difference. It's only when we add, when we blended the pixel intensities of our lines to the original. That will see a difference, since they actually have a non zero picture intensity value. Anyway, First argument is going to be the Elaine image, and we're taking the waited. Some between the arrays of these two images will give our Elaine image a weight of 0.8. Basically, that's going to multiply. All elements in this array buys there a 0.8 the creasing their picture intensities, which makes it a bit darker. It will be Maura parents Why we're doing this momentarily. The third argument is the second and put a rate of the same size, and we know what's the same size because we gave it the same shape as our Elaine image right? But it's all zeros and stud, but anyway, let's put our line image Ray, and we'll give that a weight of one, multiplying all elements in this array by one. And now, when we add these to a raise up, this one will have 20% more weights, which means that the lines will be more clearly defined when they blend the two images. This image is going to be a bit darker there by better defining the lines that we're blending it into. Finally, there's the GAM argument where we can choose some value that will add to our some. We'll just put a scaler value of one. It won't really make a substantial difference. And that is all we detected lines in the Grady INTs placed these lines on a black image, and then we blended that image with our original color image. So Free replaced this with combo image run the Code Python lanes that P Y indeed blended both of our images such that the lines are displayed right on top of our lanes. That is all for identifying lane lines. You learned how to identify lines inside of a Grady Int image, but the huff transform technique, and then we took these lines, place them on a random black image, which has the exact same dimensions as our original road image, thereby by blending the two we were able to ultimately place our detected lines back onto our original image in the next video will further optimize how these lines are display. 13. Optimizing: in the last lesson, we detected lines from a series of points in the Grady Int image using the huff transform detection algorithm. We then took these lines and place the money blank image, which we then merged with our color image, ultimately displaying the lines on Tara Llanes. What we'll do now is further optimize how these lines are displayed. It's important to first recognize that the lines currently displayed correspond the bins would succeeded the voting threshold. They were voted as the lines which best described our data. What we'll do now is instead of having multiple lines, we can average out there sloping. Why intercept into a single line that traces out both of our lanes before getting into it? It seems that there is some inconsistency in the code. This should be image so as to properly referenced the argument, not lane image and the same thing here. Make sure to reference the argument image, not the global variable. Kenny. It shouldn't have made a difference, since they both correspond to the same value the same case for this one. But it's always good to be consistent so as to avoid bucks also not a good habit to reuse variable names. So we'll rename this to can image and change it over here accordingly. All right, we'll start this less enough by going over here and setting averaged lines is equal to the return value of some function. I will declare later on average slope intercept that will be our function name, and we'll pass into it. Our colored Elaine image, as well as the lines that we detected and now will simply define the function right on top Death Average slope intercept with argument, image and lines, and what we'll do first is will declare to empty lists left fit his equal to. And if you list right fit is also equal to an empty lists. Left fit will contain the coordinates of the average lines on the left and intuitively right fit will contain coordinates of the line, which will display on the right. What we can do now is looped through every line as we did previously. Four Line in lines and reshape each line into a one dimensional array with four elements. Line Donna reshape for and now we'll unpack the elements of the array into four variables where x one y one x two. Why two will equal the four values in the array, respectively. Nothing new so far. These air, the points of a line when you're given the points of the line is very easy to compute the slope bike, allocating the change and why over the change in X subbing that into our equation. To then determine the Y intercept well to determine these parameters and code. What weaken Dio is set parameters is equal to numb pie that Polly fit. What Paul, if it will do for us, is it will fit. A first degree polynomial, which would simply be a linear function of y is equal. Gemex will be. It's going to fit this polynomial to our X and y points and return a vector of coefficients , which described the slope in Why intercept? The first argument is where you will place the X coordinates of your two points x one x two . The second argument is where you replace the why coordinates of your two points. Why one and why to and will fit a polynomial of degree 12 hour X and my points. That way we get the parameters of a linear function if you go ahead and print parameters and to our terminal. Parthenon leads the P Y for each line that we iterated through. It prints the slope and the Y intercept the slope is the first element in the array. The Y intercept It is the second elements, so what we can do is set Slope is equal to parameters. Index zero and we'll set. Intercept is equal to parameters at the next one. And now for each line that we iterated through, we need to check if the slope of that line does it correspond to a line on the left side or a line on the right side to determine this. It's important to note that our lines, the pending on which side they're at, are all roughly more or less going in the same direction. All the lines here are slanted a bit to the left, and all the lines here are slanted a bit to their rights. Here's our image displayed with X and y axes noticed that in the image the Y axis goes downwards along the rows. And if you remember from basic high school math, a line has a positive slope. When Why always increases as X increases, so these lines would have a positive slope as they're changing y over. The change in X would result in a positive value, although for these lines since as X increases why decreases their slope value would therefore be negative. Since they're changing y over, the change in X would result in a negative value. Final conclusion Being lines on the left will have a negative slope. Lines on the right will have a positive slope. So weaken Dio is backed our code weaken right If slope is smaller than zero if the line we're iterating through has a negative slope value will upend it into the left lest left fits that append and will upend it slope and while intercept as a to pull slope intercept otherwise else well upended into their right list right fit dot append slope and the y intercept If we print the results outside of the four loop print left fit Print right Fit backed are terminal python leans that p y. Now we have a list that contains all of the slopes and why intercept of the lines on the left side and another list that contains all the slopes and why intercepts of the lines on their right side? What we want to do now is average out all of these values into a single slope and why intercept backed Our code will do that for both sides, but we'll start with the left side. Will set left fit. Average is equal to numb pie, not average. And we're going to average out all the values of our left fit, and it's really important that you specify and access is equal to zero. Imagine this was an array of multiple rows in two columns. What we want to do is operate vertically along the rows to get the average slope and the average Y intercept respectively. Do the same thing on the right side, right? Fit average is equal to numb pie dot average. We'll do that for right. Fit with an axis is equal to zero. All right, we'll go ahead and prints left fit. Average print, right, fit average, and we'll go ahead and just label them for clarity. Who left and rights. Dr. Terminal. We get back to erase this array represents the average slope. And why intercept of a single line through the left side and the surrounded the average slope. And why intercepted a single line through there right side right now, out of the woods yet we have the slopes and why intercepts of the lines that will eventually draw. But we can't actually draw them unless we also specify their coordinates to actually specify where we want our lives to be placed. The x one y one x two y two for each line. So what we'll do is we'll define a function death, make coordinates with argument, image and line parameters with some return value. This return value is going to the note X and Y coordinates of the line so back here will set left line equal to the return value of May. Coordinates will pass in the respective arguments the image as well as the slope and why intercept of our left line left fit average. Same thing for their right line. Right line is equal to make coordinates. Image right fit average back to our function. We're going to unpack this list of two elements. Whichever one that's being passed in the slope and intercept into two variables. Slope intercept is equal to line parameters. We'll start with Y one. The initial vertical position of our lines. This one is pretty obvious since we want our lines to start at the bottom of the image. Before we do this. If you go ahead and print the shape of the image print image dot shape into your terminal will rerun this code it prints twice since. Clearly we're calling the function twice, but regardless recall that the shape corresponds to your arrays dimensions. In this case, this represents the images heights with an number of channels. We're only interested in the heights, which also corresponds to the bottom coordinate of our image, as demonstrated by my plot lib. A little counterintuitive that the highest value is on the bottom of the Y axis. But another way to interpret this is to have the Y axis going vertically downwards from 0 to 700 which makes sense since images air simply a raise of pixels, an array indices are read from the top down will set why one is equal to image dot shape that index zero since that represented the heights. And now why, too, is going to equal why one times 3/5 will make this into an integer type. And so essentially, this will be 704 times 3/5, which is going to evaluate too. 422 Which means that both of our airlines will start from the bottom at 704 and goes 3/5 of the way upwards up until the coordinate for 20. All right, now, X one can simply be determined. Algebraic Lee, We know that Why is equal toe m expose be so acts is equal toe. Why minus be divided by M? If you re arrange the variables so we can simply set X one is equal to why one minus the intercept divided by the slope. We'll make this into an integer as well and the same thing for x two, except that we have to replace this with X two. And why to Now that we have, all of our coordinates will return them as a numb pyre. A numb pie dot array, X one y one x two y two. So both of our lines are going to have the same vertical coordinates. They're both going to start at the very bottom, but they're going to start at the bottom and go upwards 3/5 of the way up until the coordinate for 24. But they're horizontal coordinates are obviously dependent on their slope. And why intercept, which we calculated right here? We finally have our lines. We can return them as an array return num pie dot array, left line and right line. And now comes the fun part, which is? Instead of our line image being populated by the huff. Detected lines were going to pass in the average lines. If we show the line image instead, run the code. This looks a lot smoother. Instead of many lines, they were all averaged out into a single line on each side, back to IRA code. Obviously, we're still blending the line image with the color image. So let's show that instead, combo image bucked are terminal. We'll rerun this, and it displays our two lines on our two lanes. We took the average of our lines and displayed one line on each side instead. This looks a lot smoother than earlier. One more thing before we end. This lesson is that previously we were passing in a three dimensional array. The huff lines into our display lines function. But now we're passing in the average lines that we created, and we know that when we generate through over the slides array, each line is already a one dimensional array, so there's no need to reshape it into one. It already is one dimensional, so feel free to remove their reshaped. For it's just extra code doesn't really make a difference whether you have it or nuts. And better yet, weaken. Simply unpack each line and 24 variables over here and delete that, and that is all. Let's rerun it to make sure we didn't make any mistakes and everything still works. Set accordingly in the next lesson will use the code that we currently have and take it up a notch by identifying lane lines in a video 14. Finding Lanes on Video: welcome to your last lesson of this section. In the last lesson, we finally finished our line detection algorithm and identified lane lines in our image. What we'll do now is use that same algorithm to identify lines in a video. This is the video and will use the algorithm. We currently have the detect lane lines in every single frame. This video you can access from the following get hobbling, or you can feel free to just type out the link. It's up to you now anyway. Once you're on this page, click on test to that and before and and downloads all right, and upon downloading it, make sure it's says, test to dot and before so as to stay consistent with the videos. And once you download it to wherever you have it downloaded, drag it into your project. Finding lanes. If you are using the Adam text editor, don't try and open the video with Autumn. Otherwise it will freeze. Alternatively, we could have placed this video in desktop or some other directory and just referenced its path. But we'll go with this for now just to keep things quick, regardless to capture this video in our workspace. We need to create a video capture object by setting a variable name cap is equal to C V two dot video capture, and we'll capture in the video test to dot mp four. And while cap dot is opened, this returns true. If video capturing has been initialized, will enter into a loop where we will first use their read function. Capta read to decode every video frame. And what this returns is two values which we can unpack. The first value is just a 1,000,000,000 that we're not currently interested in, so leave that is blank. The second value is the image, the frame that's currently being projected in our video. And it's the current frame of our video where will be detecting lines. So what we'll do is actually copy and paste the code. The algorithm we already have for detecting lines. We're not gonna do it all over again. And wherever it says lane image, we're going to replace that with the current video frame over here, here, in here as well. Make sure to get all of them. Otherwise, your code will not make any sense pretty easy. All we did was apply the algorithm we already implemented, but instead of a static image, were doing it on a video. Since we're not working with images anymore, you can go ahead and comment up this code or deleted. It's up to you. I myself will comment it out, and now with him, show we're still showing the current image that's being processed. As for weight key, if we leave this that wakey zero, you're going to be waiting infinitely between each frame of the video. So your video, it just frees up. Instead, we want toe wait one millisecond in between friends to them this way, the next one. Now, before we add any logic to break out of the slough Pless just to run the code by going to our terminal and running python lanes. Doc, P. Y. And all right had identifies the lines in every single frame of our video. That's pretty cool, and this is nothing new. It follows the exact same process as how we detected lines in the image. We're still applying the canny algorithm to get the Grady int detecting lines with have transform and then averaging them out. Except now it's being done in every single frame of the video repeatedly inside of a while loop. That is all for this section. Now, just to finish this lesson off, if I rerun the code and try to close the video, it doesn't work. We need a way to actually break out of this for loop and not just wait until the videos complete for it to dismiss, so we'll go back here upon pressing a keyboard key. We want the video to close, so we'll put this inside of in this statement such that we're still invoking the weight key function. We mentioned that it waits one millisecond in between frames, but what it also does that returns a 32 bit integer value, which we can compare to the numeric and coating of the keyboard character that we're going to press the new American coding we can obtain from the built in function. Oh Weir D and the keyboard character that we're going to press will be que, and we'll just set a comparison operation between the two. Ultimately, when we pressed the keyboard button, cue the comparison of these numbers will evaluate the true and once they are equal, will break out of the loop. And once we break out of the loop will close the video file by calling Captain Release. And we'll also call CV to the destroy a windows to destroy the window that work currently on to close its where you run the code python lanes that p y press cute and everything works fine. If the keyboard action did not work out for you, a common trick is to apply a bit wise and operation with the eggs of decimal Constance zero x f f. Just know that this operation it masks the integer value we got from weight key to eight bits, which ultimately just ensures cross platform safety when making our comparison, it's still the same concept was earlier, since when we press Q. This comparison still evaluates to true breaking us out of the loop, rerunning the code. Everything should still work as expected. That is all for the section. We used candy to convert our color image into a Grady it image and from the greedy int image we detected the most relevant lines, averaged out the lines and then displayed them on the image that's currently being processed in the near future. Will be using a more advanced technique to the tech lanes. But for now, this is pretty awesome. Great job on making it this far. If you ever have any questions, feel free to ask me in the Q and a section.