Mastering Bash Scripts

1. Introduction: Stop wasting hours on manual tasks and repetitive commands. I am a nun. Hello, I'm Tom. And in mastering bas scripts, you will go from Linux newcomer to command line Ninja in six action packed modules. First, we'll cover essential Linux commands and how to write your very first bar script. So we'll be comfortable navigating any server. Then you'll master loops, automating repetitive jobs on dozens of machines with a single snippet. Next, we'll dive into conditionals and K statements, giving your scripts the power to make decisions based on environment, user input, or system state. After that, you will unlock the true magic of functions, packaging your code into reusable modular tools that simplify even the most complex workflows. You will see how to pass parameters, return values, and build a shared library of helper functions that you can import into any project. We'll also explore file operations, reading, writing, and processing files programmatically. So you can automatically manage logs, configuration files and data streams without lifting a finger. And because no script is perfect on the first try, we'll teach you professional debugging techniques from set minus x tracing to systematic error handling, you will learn how to find and fix bugs faster than ever. By the end of this course, you'll be delivering rock solid automation that saves you and your team hours every week. You will write cleaner, more maintainable code, prevent silent failures, and deploy with total confidence. Ready to transform your DevOps workflow and roll now in mastering buzz scripts and start automating like APO. We'll see you inside. 2. Parameters: Well, in this chapter, we will talk about penance. Let's clear the scream touch. And the name of the flame, we create empty file. In this case, run a stage. And first slime. Because I want to use the Benbsh I will write this lin and Mm. So it turns into run the beans the bash inside the bin folder. Echo stop. Echo will show the string after it. So in this case, you will see stout when you run it. The word stout. Then see the command with the S or LinoTO When LS show me the files and LSnoLP we show me the files in the directory with all the details with the promision and date and time. Okay, now I try to start the script. But you have permission problem, I need to fix it for this command. See model. Close X and the name of the script. We give him permission, you say RW x. So that means to use a bomb, X is execute. So I can execute the script. Because of the sing you want to close X to the nail of the fine. Okay, now I run it the script, and you see the start string because of the command. Clear is to clear the screen, clear man. And we went clear and then run the screen. So stop commanding the same time. We creen the scream and run. Let's define parametal V one equals zero. And let's print it. We print it with dole sign in the name of the parametal. And we sign when we sign it, we don't need the dot out. Okay, let's echo it. You see it with the brock slip. Dollar broklp. I we define another parametal my equals Toml and we print it. And we write string and palmetal like this. My name is Thoma. Great. Well, let's erase it. Let's define palmetalname, equal Joe, let's print it like this. A this example I will show you how I can use the palmeta that is the name of other parameta and if I want to print it and I don't want to print the name, the string name. I want to print the parametal name that the pointer tell me the parameta name. So I will show you what you need to do in this case. Here, you see, it's only printed the strained name, and I don't want it. I wanted to know that this is the name of parameter. And now I need to change. I need to add exclamation M. To the pointer before the string pointer. So now you see he tell me pointer equal jo why the extension mark and the palmetaPointer, will tell him that this is the name of the parameter name is the name of the parameta. So in the background, he will do dollar name when you saw when you do explanation and prime t. Okay, param one equal A BBCc. Now I want to show you if you want to use patata, for example, part one and to replace punt of the string with different string. For example, in this case, if I want to replace DB with TT or other string, it's not metal. Or il what slash and the string you want to change and slash and the new string. In this case, I want to replace BB with DD. Now you see when we print it, you will see AA DD CC and not AA DV CC because of this replacement of partal replace Okay, now let's define Now I would call it pmt. Pmt equal p one with this replace of one string with the other string. Okay, now let's say do a list of the Oh. Varlet swap with raise. Let's call it fruit, puffin Bexley. And D I will put some strings. This case, fruit, apple, banana, orange with space between them, and close fights. They print it like this. Zero is the first tametl in the lane. So you're seeing on me atm. Let's delete this. First bimetal April. I brought zero. I brought dollar breaks it through open break it, zero, close break it. So zero is the firstmeta in the array. A pomeal is like this. Instead of a number, I would do it fine. It's a shift too. This will show me all the pomeals in array. In this case, apple, banana and orange. Okay, now to see the last phenometal I will write minus one. Let's say orange, 'cause orange is the last para metal in daylight. Let's delts lines. Now let's talk about dictionaries. Declare minus A funds. This is how I declare it. Tuns left braless. Dog dog is the key, and the bug is the value. Dictionaries wop with T and value. Or here I declare first pair. Dog is the key, bug is the value, db equal b. This example, I will write sound for each animal. Okay, let's declare another pill. The key will be cow, and the value, in this case, the sound will be more. It's printed. I call DL and the name of the dictionary. And I want to see the value of Dong. So I will write inside the bracket the name of the key, and it will print it the value of this key. If I want all of them, I will do, like the eSign shift too. And it will show me a bulk space for all the values. Now, if I want to see the kids Sano value word to it. You say you are free because it's free tails in the ditchin This is the kids. This is the variance. Number of elements. I hope you enjoy this chop down above Amtus kind. See you in the next one. 3. Loops: Hey, guys, and welcome to Chapter three of our Mastering B Scripts course. In this chapter, we are going to explore one of the most powerful features in bass scripting Loops, Mnan we're excited to show you how loops can transform your approach to automation and task management. So far, you have learned about Linux basics and how to create your first scripts. Now we are going to level up your scripting toolkit by introducing structures that can repeat actions, saving you countless hours of manual work. Let's dive in. So why are loops essential in devops? As a DevOps engineer, you'll frequently encounter situations where you need to perform the same actions multiple times. For example, deployment tasks running the same deployment steps across multiple servers or maybe file processing, performing operations on every file in every directory, or maybe log analysis, scanning through log entries to find specific patterns. Without loops, you would have to rewrite repetitive code or manually execute commands for each instance. Loops solve this problem elegantly by letting you define an action once and repeat it as many times as needed. Let me give you an example. As you probably saw before, this is the Sebenk with it, we begin every bur script. Now, I could write this code, echo item one. Echo iteration two, echo, iteration three, and so on and so forth. But we have a much easier way to do that. We can do that for item in one item two, item three, and the following. Do echo processing item. And finally, we write the word done. That way we are doing something called a four loop. We put an iterator, the item with it, we count. We put the items over which we count, in this case, item one, item two, item three, and run the echo command. And if we do it, we'll just make it look the same. We'll first put the Run command, the CH mode plus x on the Chapter three file to allow us to run it. And then we'll just run it. As you can see, we just put running stuff again and again and again without writing too much code. Now, let's see another example. How about the following? Yes can you guess what this code does? Pause the video and think about it? Okay, I hope you have enough time to think about the code. But generally, this code backs up all the files at the folder ETC, asterk.com, meaning all the config files at the ETC folder, and backing these up into the BAK extension. Let us run this file and see how it works. Okay, so it ran over the files, as you can see, but we got permission denied. Any idea why that is? We'll talk about permissions soon enough. Now, after you have talked about the code that you have just written, we can see all the stuff that we have done and learn some file. So far, we've learned how to create four loops. Four loops allows an iterator to run over a set of parameters. For example, item one, item two, item three, or in this case, all the configles in a folder. Think what could you do with four loops in the future, how much time it will save you. Another loop I want to talk about is the y loop. Y loop works very similarly and executes Always until the condition you set for it becomes false. So while the condition is through, it runs. This is perfect for scenarios where you don't know in advance how many iterations you need in contrast to four. Now, let us slide the loop. This i loop got the counter. The counter is set to one initially. Now, as long as the counter is less than or equal to five, the loop will continue to one. Let's see an example. Okay. As you can see, it shows iteration one, Iteration two, Iteration three, Iteration four, and iteration five. While the counter is less than or equal to five, it continues to count. So far, we've seen pretty simple examples of walking with loops. Now, how about we see a real world application that monitors an actual service? Let us see an example about it. Now, this is an example of a real bold application where we monitor the HTTP server bolt on our local host. Let's see how it turns. As you can see, it is waiting for the server to start. In this case, it will never come up. But if the service does come up, it will echo the line services up and running. This is an example of a real world usage of the yoop. Another loop I would like to talk about is the until loop. Let's see an example how it works. The antelope is the exact opposite of y loop. It continues until a condition becomes true, while it assumes the initial condition is false. As you can see, it began with the count zero as we set it, and then one and then two, and then three, and then four. Another thing I want to talk about is the loop control, breaking and continuing the loop. You will use it many times while you need to break your script from running once you found the condition. What this loop does? Pause the video and think about it. Okay, I hope you have enough time to think. Now, in this loop, we are running a four loop, which we have learned about previously. We get an iterator number and a set of numbers one to ten. If we find the number equals five, we break the loop from running anymore. In other cases, we continue to run. Let's see an example. Passing number one, passing number two, posing number three, passing number four, and we found the five. We are breaking the loop just as we said the follow up to do. All right, guys. So far, we have seen examples of different stuffs and how it works. Now, let's do a practical DevOps example a server Earth check. All right. The following script gets a list of servers, google.com, yahoo.com, and binning.com. For each server, it iterates over with the flop. If a pink to the server succeeds, then the server is reach. Then we try to check disk space and check the CPU usage on the server. Now, this is assuming we have SSH access to the server. It these cases google.com, yahoo.com and binning.com. We don't have an access naturally. Now, we can run and see what happens. Google.com is reachable. Wait. Let's continue. Now it will try to get a Sage access. Obviously, we won't let it accessing this specific server. This script shows a complete example of using four loop to actually manage your day to day DevOps tasks. This is one of the most basic DevOps task, monitoring, health check ECOS servers. Instead of writing the same code again and again and again, you just write it once, put the list of servers you want to check, and that's it. It just works. Finally, I want to talk about some best practices for using loops. First one, always test with a small subset before running on a loop on thousands of items. Second, use meaningful variable names that clearly indicate what's being looped over. Thirdly, consider performance. Extremely large loops may need to be optimized. And fourth, add safety checks to prevent unwanted behavior, especially in loops that modify files. Finally, let's talk about our conclusions from the lessons. Loops are the backbone of automation beh scripting. They allow you to execute repetitive tasks efficiently and handle dynamic scenarios with these by mastering the different types of loops and understanding when to use each one, you are building essential skills for your Devops toolkit. In our next chapter, we'll explore conditional statements, how to make your scripts even smarter by adding decision making capabilities. Remember to practice what you have learned. Try creating loops for your own automation tasks and experiment with different loop structures to see which works best for your specific needs. Thank you for joining us in Chapter three. We'll see you in the next lesson. Oh, one last thing, as a bonus exercise for this lesson, create a script that uses a loop to find all log files in directory that contains a specific error message. This will combine the 4. If and Case: Hi, guys. Welcome to Chapter four of our Mastering Bash scripting course. In this chapter, we are diving into conditionals and case statements, the decision making components that make your scripts to intelligent. I'm a nan. And today we'll show you how to add logic to your scripts so they can respond differently based on various conditions. After mastering loops in our previous chapter, you are now ready to make your scripts even more powerful by teaching them how to make decisions. Let's get started. So why do we even need conditionals in DevOps? In the world of DevOps, environments are rarely predictable. You will frequently encounter situations where your script needs to adapt to different environments, behave differently on development versus production servers, or maybe Ns gracefully, check if a command succeeded before proceeding. Or maybe process user input. Your user will provide input and you need to validate and respond to different types of user input. And finally, make operational decisions. Determine when to alert, when to try or when to proceed. Conditionals, give your script the intelligence to handle these vit scenarios automatically. Let's begin by talking about the statement. The basic decision making process. The most fundamental conditional in Bash is the I statement. It evaluates a condition and executes code only if that condition is two. Let's see an example. In this case, we wrote a very simple script that says, I some condition, then we will run commands. And finally, we'll end by writing the letters FY, which are the reverse of I Let's see a practical example. Imagine you are deploying an application and need to check if a configuration file exists. Let's do it. I minus F, which says, if the file exists, configure dot YML. We'll write. Then Echo configuration file found proceeding with deployments. Finally, we can run our file Deploy dot a sage. We'll assume it exists. And finally, we'll write the letters FY. What if we want to handle a different scenario? What if we want to handle a case where the file doesn't exist. Instead of writing FY, we'll write HeLS. And then we can do the following echo configuration file not found. Exiting and we can even write Exit one, which means return to the operating system and AL code. This simple script checks for a critical file before attempting deployment, potentially saving you from fail deployment and service disruption. Let's go over some different options in the comparison operators. What we can do with comparison operators, we can do the following. We can write the equal operator, which we write like equals or double equal, which means equals two. Alternatively, we can write not equal. In that case, we can write it like that. We can also try check if a string is empty. How we do that. We'll write the minus Z, and alternatively, we can check if the string is not empty by writing the minus N. Now, what if we want to compare numbers? We can do minus EQ to check for equality of a number. We can also write minus NE to check if the numbers are not equal. Alternatively, we can write minus GT for checking if the number is greater than or maybe checking if the number is less than by writing the LT. Finally, I want to talk about two different options, the minus GE, which checks if it's greater than or equal to, and finally, minus LE, less than or equal to. Now, what we can do if we want to check files? For files, we can do the minus F. Check if the file exists. Alternatively, we can write a minus D. Check if the directory exists. And finally, we can do the funnel checks, minus X. Check if the file is not empty. And finally, the minus X, check if the file is executable, meaning we can run it on our terminal. Let's talk about more complex conditions. The end which is written like that and the O operator, which is written like that. Now, what we can do with the end operator? We can do the following one and then zero. What would be the result? So one and zero, meaning if it's true and false. Naturally, this cannot happen. So the result will be false, zero. Now, in the case of O, we can do one or zero, meaning true or false. That would be the result is true. Let's do the following. We can do checking if the user is root user, and if the system is a Linux system, how we can do that. Do you have any idea? Pause the video and write it up. All right, guys. By now, I hope you had a chance to see if your code was correct. Let's do it I The following script does exactly what we talked about. It checks if the user has an ID equal to zero, meaning root user, and if the operating system is identified as a Linux, if that is the case, meaning both conditions are true, it writes an echo. You are the root user on a Linux system. Alternatively, it writes a different situation. You are not a root user or you are not an Linux system. Let's talk about more complex scenarios. Let's talk about the LI and clauses. What we can do with that? For more complex scenarios, you can use the LI I and s to handle multiple possible conditions. For example, the following, we can check if the memory usage of our system is a value, then we get it. If the memo usage is greater than 90, we'll write an echo. Memo usage is above 90%, else, we'll write a message. Memo usage is below 90%. Alternatively, we can write the following Checking if the memo usage is above 80% and then echoing MMO usage is above 80% message, meaning we tested both the 90% scenario and the 80% scenario. This code gives us much more flexibility in checking our systems. Finally, for the end of the lesson, I want to talk about the K statement and elegant multi way decisions making. When you need to compare a single variable against multiple values, the K statement is much cleaner than multiple LI branches. Let's see an example. That case, we can do an echo one development or echo two staging or echo three, production. Now, how we can utilize it, we'll write the following case, choice. In. Option number one, then we'll echo deploying to development. On case two, we'll write deploying to staging. On case three, we'll write deploying to production. And finally, if we get no acceptable value, we'll go to the default, Aster, meaning invalid choice. Finally, we'll write the reverse of case. We'll write it as EAC, which is the reverse of case. Now, what I want you to do as a final example is to put all the concepts we talked about in this course into a practical deployment script. I want you to get environment variables and then use the case or the I clauses to echo to the user their choice. Now, this is a more complex scenario, but you will use it daily in your DevOps journey. Finally, I want to talk about best practices for conditionals. As you start working with conditionals, keep these best practices in mind. Number one, always quote your variables to prevent eros with spaces and special characters. Two, use double brackets for advanced tests, which support additional operators. Number three, add O messages that clearly explain what went wrong. Number four, test all branches of conditionals to ensure the work is expected. And number five and the last one, use meaningful variable names to make your conditions more readable. Alright, guys. Let's talk about conclusions from this lesson. Conditionals and as statements are what transform your scripts from simple command sequences into intelligent tools. They allow your scripts to make decisions and their eros and adapt different situations. Essential capabilities for any DevOps automation guide. In our next chapter, we'll explore functions, how to organize your code into reusable blocks that make your scripts more modular and maintainable. Thank you for joining Gas in Chapter four. We'll see you in the next lesson. Bye bye. 5. Functions: This chapter, we are exploring functions, the building blocks that will take your scripts from good to great. We have the function, in this case, demo function. We write a comment commands to execute, and some echo function is running. Now, how do we call the function? We just write the name of it. Simple, write. Let's see a more basic but usable example. Let's write a function called welcome. With it, we'll do an echo, beginning with some nice separation and then echo. Welcome to deployment tool. Another echo, which can be interesting by writing out the current date, which is basically just calling another function named date and another separator at the end. Now, we can call this function by just typing out, welcome. Amazing, right? Let's run it. And as you can see, welcome to deployment to with the current date. Amazing, right? This might seem trivial, but even these simple functions saves you from writing the same four lines of code every time you need a welcome banal. Now, let's talk about functions with parameters. Functions become much more powerful when they can accept parameters, such as the following. Here, we're introducing a new pomeal called name and another Pametal called all, which will get this data from the user. Now we can do the following echo Hello. Name a role which we got from the user. Now, let's try the following Grit Tomel and DevOps engineer. Now, let's run it. As you can see, it printed out exactly what we wanted. Hello, Tomel. You are a DevOps engineer. Funny, right. Let's try another theme. Let's call it grid like that system administrator. What will happen? Now we expected to print out two greetings. Let's test. As you can see, it printed out exactly what you wanted it to print. Hello, Tomer, you are a DevOps engineer, and also hello Anan. You are a system administrator. Parameters are accessed inside the function using position variables, dollar one, two, et cetera, just like command line arguments to your script. Now, let's talk about returning values from functions. Unlike other programming languages, Bash functions don't have a traditional return statement for values. Instead, there are several ways to return values. The first method is using Echo. Let's see how it works. Let's see how it works now. Let's call it by using the following. Okay. So now it printed out what we wanted. It couldn't find the folder, so it printed nothing. Now, let's talk about using return status. Functions can return functions can return status code zero into 255 to indicate success or failure. Let's see an example. Now, let's see what happens when we run check service with HTTPD. We've got nothing, but how can we test the result? Maybe we should add some echo. Now, this one says if the result we got from the function equals zero, meaning success, will echo HTTPD is running, else, will output HDDPD is not running. Let's see how it works. And we ran it and we got HTTPD is not running. Now, another thing we can do is using global variables. For complex return values, you can modify global variables like the system information. Let's see an example. And now we need to call the function, right? Let's see how it works in real time. And as you can see, we got a value, the Os Name, the kernel version, and the uptime. Amazing, right? Local variables in functions. Let's talk about it. To prevent variables in your functions from affecting the rest of your script, use the local keyword, meaning the variable is only for your current function. In contrast to the global variable, which modifies global value in your system. Now, we can do much more with functions. We can create a library of functions or we can utilize external functions created by other people. This is very useful when utilizing complex DevOps scripts. Now, what I want you to do right now is pause this video and write a script that gets palmetals from the user and notifies teams and other people in your organization. This is something which can be complex, but we'll go over it soon enough. Pause the video now and do it. All right, guys. By now, I hope you had a chance to write some code. Let's go over our version of the code. Now, we'll go about talking about function library. In this case, checking the environment. We'll get a variable from the user ENV, we'll utilize the case we talked about. And if we got one of the valid values, Dev, stage or pod, we'll return zero. Otherwise, we'll output in valid environment and showing the user the relevant values. After that, we'll go about backup in the database. This is something that you'll do very often in your DevOps world journey. So we get the ENV from the user, and then we echo to the user that we create a database backup for the selected environment. Now, we won't go into the process of backing up a database because this is much more complex, but we'll talk about it soon enough. Eventually we'll output a backup completed message. Now, let's talk about deploying the code. We'll get two variables from the user. The first one is the ENV, the environment, and the second one is the version. We'll echo the message to the user, deploying version, dollar version to Dollar environment. We won't go into the deployment logic here, but we'll walk about it soon enough. And eventually we will run deployment completed message. About the Teams notification, we'll get two erbers from the user. The first one is ENV and the second one is status. The first one will be dollar one. The second one will be dollar two, and then we echo sending Dollar status notification to Team about dollar and V deployment. Finally, we'll run the main function. This is the entry point to every application, either in dash or in other platforms. We'll get two vibers local enviroment, which will be, again, dollar one and local version, which will be dollar two. Then we will utilize the external function, check environment with the dollar environment. If it's not valid, meaning the return code was not zero, we will exit application with a one status code, which means arrow. Now we will get if we checked the version variable. If we got a version variable, we continue. Otherwise, we will output an arrow and output to the user, the usage and outuse our application. Eventually, we'll call the functions Dup database with the variable, and the deploy code with the two variables. Once we finished all steps, we'll call the external library, run tests with the variable and then notify Team, which we wrote earlier with two variables, environment and success, meaning we notify the team about our success. Hm. If the Run test exit code was not successful, we'll notify the team with the FLO and exit with Exit code one. Finally, we'll call the entry point main with all the scripts arguments. We do that by inputting Doral and the E sign. That means passing all the script arguments into the function. Now, let's talk about conclusions and best practices. Best practices for functions. Number one, give functions meaningful names that describe what they do. Number two, use local variables to prevent unintended side effects. Number three, document your functions with comments explaining purpose, parameters, and return values. Number four, keep functions focused on a single task. Follow the D one thing well principle. Number five, enter errors in side functions and provide meaningful return codes. And finally, number six, test functions individually before integrating them into larger scripts. Now about conclusions. Functions are what separate basic scripts from professional grade automation tools. By organizing your code into modular reusable functions, you make your scripts more maintainable, reliable, and powerful. You have now learned about variables, loops, conditionals, and functions, the core building blocks of bass scripting. In our next chapter, we'll explore working with files, how to read, write, and process files effectively in your scripts. Thank you for joining us in Chapter five. We'll see you in the axtsen. Bye bye. 6. Files: Hello, guys. In this chapter, we will talk about walking with the logs, you will use loops. Let's see the log log C log. It's a log of the Linux. Let's look at it. It's a big log. Let's use the third on the last 400 line. Okay, look at this line and this line. Let's say our task is to look at the lines. Let's say the last 500 lines of this file, this log file, and search for this strand and also info or arrow. And if you see info, you will do one task. If you see arrow, the other pass. For example, if you see O, show me the other lines after. And if you see info, only count the lines of lines in the local that this string and also info with capital letters, count how much lines. And if you see O, save in other file this string come after the L. Okay, so this will be the base command in our script 500. Now, let me show you a writing here in. Stain clear and run the spile. So this one clear of the screen and run. Okay. This is easy. Now we want to create a lot. So we will do something like this. While while M til the gala. So this is the start of the loop. This is the end of the loop, and this is the loop. This string will show last 500 lines because I did tail -500. And this is the log of course, the log slab. And then on this output, we come in this type of this command. This command, tell them while read, and let's turn it the my line. Low line. So every lime would be inside this tamal every time the loop grabs each line, okay? So let's do this, the We Line, love lime dal line. So this each lime, let's see. You said the log line and then the line of the log. We keep telling here, this will show me the all five other lines, but our mission to filter only with Amazon is the same age this. So we copy this and we will use dregPi so amazing. Like, it is. So this will Striilt all the outputs. Show me only lines that have this string. Okay, let's take Okay, you see now, it's only line with our zoning system agent. Now we want to see only info of Evo. You know what? Let's say a one of info of. So how we do it. We will use grip. A grip, you can tell him you do a taste, you will tell him what to see them. Some options to faster. For example, info, then you do like this type. Lo. Wow. Okay, save. That's funny. Now, we see on lines with this strand and also here info or EO or war, like warning. You see? This is the egret. You give him the choices with the type inside. And then this is the safe for this string. That was the same age. Okay, now we know that the lines inside this look the lines we want. Now we can ask. We can do something like this. In E C dollar, BfletEch. With this one t time. This will count if the line L O with capital letter. This command, WC M L will count lines. If I if I write Plata t instead, you can show me one because this is one line. If I do WC minus C into account the chow. But when it chow you C five, and this is only one line. So I will use line. And here, I can ask then level. Then the th. So in this I section, I'm asking if this is an RO kind because it's bigger than zero. So it's one. If it's one, we know that inside this loop, we're talking about O line. So let's say our task is its O to print the string after the O. So let me show you I want to test it. I will only copy this this example. A cold. Okay. Now if I don't invest, the blood soil out. Okay. That's why based my lime. Okay? That's hit my lime. Now I want to I know this is a yellow lime, and I want to take these twins. Okay? So what I can do is I can use PL minus. Tal minus, it's transfer. Spell, for example, space to backslash, BexsN is new line. I want to take this line, and every time you see space, it will change it to new lime. Let's do it. Okay, you see, every section of the line is new lime. Now I can tell him that grab A, B, and I can use with the grab minus 99, for example, show me the 99 lines after after the arrow. So you see we see only the arrow and the fines after. Okay. Now, I want to replace it or change it to one line. So I will do MN, changes big sesh N go space. So you change it back from few lines to one line. Okay, let's copy to our scape So if it's kind, show me the love, the love line. Change it to every time you see space, change it to a new line. The pass only the error in the 99th world after the error and then take it back to one line. Okay. That's what the ball. You see, we see only only do with the phase after. We can do like this to put the and a new life. Let's take it to a. Yeah, this is theta. Okay, so you see EO and the string after the O. And then you in Let's see if I do I have breakfast and et cetera. It's, this is good. Okay, let's test this. Okay. This is good, but I said this at the end. Okay, what do we do echo space here. So this is place and enter, of course, with regular echo. So this will fix what we want with bunny. Okay, we see each line arrow and the arrow after Very good. Now we want to check the Ifokyd. So I will copy this docuate this Ifokine And I would levels leave. Fine. That's No. Okay, this is about and it ends if it's Eli. And then I ask is checking, if in the log line, we have info with capital letters, count lines. So if the lines is greater than zero, then write info info and the line. We can do only only this and line we have the info strain inside. Let's check. It's the easy tory. Okay, info and then we see the line of the info see the real info string that you create. And this is my string info. Okay, so you see, in this case, I dot count the info, but I show you the info and then all the line after the info. And then in LO case, I only show you the string EO and the lines asteri because the original line of info have other information like dates other styles Avazon SM agent. So in case of O, we only show the Evo string and the text asi, on. Okay. This is what we learn about the walkway log s which hides the tail. Show me only the 500 lines, GregEGreg with the choices ben Wil and inside the wine we do to kind of check if it's evine or if it's in info cry. You can ask if it's warming, you can do another cl for it Wow. Warning without the line, only the warning text. Let's see if it's walking. You say this is warning and it's not showing us the warning. Now, if I want to save it on a file, I can do lo. For example, I run FLP by the g This will delete the log. If it's not exist, it will not show at all because the mins f. Okay, so this file is deleted. And here I can do if I want only the info. I'm doing this for example. My install low my info log a TPO, and this will add last each line will come to the last line of this log only with the info. Let's say I want only the info lines to be fine. So then I will delete it without MO if it's not exist. And in this case, I will edit the bottom each line to the bottom of the log. Monsan now let's see PNP. But BNP. Why info you say in ten my inflop, you will see only the infi and also another string of each fine because of this. That's it, folks of understanding. See you in the next chapter. 7. Parse Files: Hello, guys. In this chapter, I will show you we will continue to parse log files and using grab with regular expression. Regular expansion, or you can call it eggs. It's a sequence of characters that specifies a match pattern in text. So if you want to filter text like log file, let's say this log. You can use Ex. In this example, I will want to search for an IP like this. You see? I will want to I will want to use gec to fill down this IP and get on a guy. Okay. I I don't come logo, you want see the content of the logo. Now we want the IP. The IP is free dig it, dot, pre digate dot, pre dig it, dot, pre dig it. So in this case, what we can do is like this. Bread. In hg, you can do like this. This means you want to grab only one heat that is zero to nine, so it's a number. Okay, so I want three of them. I will do this. Then I want a dot socks dot. Then let's start for this. It took the red and the red. Okay, so let's continue. We'll give him another empty heat here. Okay. This is the red. It's clear. There you go. Okay. We took the red one, so let's continue metal gi. And remain big. Okay, too. Now, the grab if you enter the grab binds, it will not return on the line with the string you wanted. I will show you only the string you wanted. So instead, this is the IP. This is what we wanted. Now let's say the IP is on the left side, like this. Okay? And there is another IT Yeah. And you want only the IP that comes from the beginning of the file. In this case, it will take both of them, but I want only this. So we can do like this. B tell m B character like an d, tell them that I want the numbers to be at the beginning of the line. If I wanted the end of the line, I will do. So it's end of line. Okay, so I want like names, band nus all, I want the I Okay, now let's bake it. So the script. What did you may have? I think. Ay core. I think I missed. This IP took the command inside the peck slate. This IP took this command, and this is the print. Okay, thanks loves. Hope you understand it. I'll see you in the next chapter. I 8. Debug: This chapter, we're exploring the bugging techniques, essential skills that will help you troubleshoot and improve your B scripts. I am a nun. And today, we'll show you how to identify, isolate and fix issues in your scripts, making your developed workflows more reliable. Bash provides several options to enable debugging. The most common ways are using the set options within your scripts. Let me show you an example. The following are the options. The first one is the set minus six. Print commands and the arguments as they are executed. The second one is the set minus E. Exit immediately if you command exits with a non zero status, meaning an arrow as a cured. The third option is the set minus U. Treat unset variables as an arrow, and the fourth option is the set minus O pipe fail. Return value of pipeline is the status of the last command to exit with non zero status. Let me show you an example. As you can see, we put all the options, and so all the code was outputted, meaning the set, and then the echo, then the actual echo has appeared on the screen. And then the variable set name equals DevOps engineer, and then the echo and then the actual printing on the screen. Now, what happens if we add back these lines, set plastics, meaning disabled debugging for a specific section if needed. So we put set plastics for this echo, and then we put again set minus six and the echo debug mad enabled again. Let me show you an example. As you can see, this command won't be printed in the Bug output. There was just the actual printing to the screen. And then we added back the set minus six and we got Echo as the command and then the printing on the screen. Another option is running scripts with debug flex. A we do that, we do that the following method. We can put it like that. And then as the following, minus six, Chapter seven dot Sage. As you can see, we got the full writing, the set plus X as written, and then the actual printing of the code, and then the echo and then debug mode enabled again printing. We can also put the following minus X V to get the more detailed debugging. As you can see, we printed all the code, including the Shibang we have some options. The common debugging flags we'll talk about the following. First one, minus X, meaning X trays, displaces commands and the arguments do execution. The second one minus V, verbs, displays shell input lines as they are read. The third option, minus E, meaning AO exit, exits the script if any command returns non zero. The fourth option minus U, no unset treats unset variables, meaning the following. And nothing printed. Sane. Amazing, right? Now, let's talk about strategic echo statements. One of the simplest debugging techniques is placing strategic echo statements. Let me show you an example. If we put the following, you can see we written a code, a function actually called deploy application. We put the local variables up name and version, and we put a strategic echo placement, meaning we ritten echo and a debug message, starting deployment of up name version dollar version. Now, following that, we written an I section. If directory exists, then we printed a debug message with a strategic echo message, otherwise, also known as se we printed a strategic echo message with a message creating application directory. Following that, we put another strategic echo message called Debug deployment completed. Now, we can go into a further level of debugging using the following. A full debug function, meaning we set up a variable local level and another variable local message. And we print out if the debug is set, then we print out debug messages with the type debug, the thin level, and message. That is a complete debug function you can use in your DevOps journey. So we use that. We use it in the following way. We set up the debug equals one with sum script. Let's call it MScrip dot Sage, in which we enable debug output. And then we call the function debug with the variable level, in this case, info and the message starting Bu process. Another usage is the level detail and the message Pametals received. Now we can also trace variables and function calls with the following function. We can put calculate storage with the variable directory, and then we call the function from previous with the level trace, and the message entering calculate storage with directory equals dollar directory. And then we put another variable size, and then we print a debug message with the level trace. Calculated size equals dollar size. As you can see, we can use the function debug in many ways, which will help us improve the debug process and with it the usefulness of the DevOps journey. Now, we can also debug full functions and return values. Let me show you an example. We can put the following function, validate input. And then call it afterwards. So how can we debug this function? We can put the function calling debug with the level FNC, which is an acronym for function. So in that way, we show to our users, which in this case, can be R&D teams, QA teams or stakeholders that our code runs and works as expected. So we put the cooling debug and the acronym for functon and the message. Validate input coded with the input we provided. And then we can debug it. If we got a valid input, then we print debug funk as the coning for function and validate input returning success. Otherwise, also known as debug funk aconym for function, validate input returning FLO. Finally, outside of the function, we call it I validated input, dollar user input, then we echo input is valid. Otherwise, known as s, we put input is invalid. Let's talk about the trap command for debugging. The trap command can help you catch Script zero. Let me show you an example. But the following does the following. The function O and LR in which we just echo rocurd on line one, meaning the parameter which failed. We written here the word Trap. We set it up to catch arrows. So we do trap arrow and LR and then we pass the variable line number to show where we got the arrow for our users, so they can open a bag for us so we can fix it later. That's the full DevOps journey. Amazing, right? Now, we can also utilize the PS four to enhance debug output. Let me show you an example. Okay, default PS four is just the letter plus, and we can set a more informative PS four. For example, pot PS four plus B source and line number and funk name and fun claim again. Now we can enable debugging and let's see how it works. Let's run it. As you can see, it got out the full text, the script, and the line number, and then echo. This is a test, and then what printed into the screen. Amazing, right? We can also isolate issues with conditional debugging. Let me show you an example. This is the following script is backing up a database. So we call it by setting up. Debug backup equals one and calling up the script. In this script, there is this backup database. We set up the local debug backup, meaning we check if we got a flag debug backup equals one, which enables debugging only for this function. If that is the case, we set the minus six. Otherwise, meaning we didn't get debug flag. We don't set it. Now, if we got it, we set the minus six. We run our function, and then we disable debugging after we are done with the function code. This is called conditional debugging. All right, guys. Now, I want you to take the following script and debug it. Let me show you the script. All right. You can check out this script in the course notes. Please pause the video and try to debug it. Later, we'll go over it. Pause the video now. All right. I hope you had enough time to debug it and you found the issue. Later we'll show you in the course notes the full solution to the debug. By examining the debug output, we can quickly identify why deployments might fail. Finally, I want to talk with you about best practices for debugging. As you develop your debugging skills, keep the following practices in mind. First, start small, debug the simplest case first. Second, isolate the problem, test components separately. Third, check variable values, confirm they contain what you expect. Fourth, review messages. They often point directly to the issue. Fifth, use version control, make it easy to revert changes that don't fix the problem. And six, document fixes. Note the root cause and solution for future reference. Nobody remembers everything. Alright. Let's draw some conclusion from this lesson. Effective debugging is what separates season develops engineers from beginners. By systematically tracking down and resolving issues in your scripts, you ensure your automation remains reliable and robust. You have now learned about variables, loops, conditionals, functions, and debugging, core skills for bash scripting mastery. Thank you for joining us in Chapter seven. We'll see you in next lesson. Oh, one other thing. As a bonus exercise, take the script that isn't working correctly and apply the debugging techniques from this chapter. Use set minus six, add strategic echo statements and implement the debug function. Document your issue you found and how you fixed it. All right, thanks a lot guys. Have a nice day. Bye bye. 9. Final Scripts: Hey, guys, and welcome to the final chapter of our Mastering Ba Scripts course. In this chapter, we are bringing together everything you have learned to build a complete DevOps automation solution. I'm Henan. Today we'll guide you through creating a comprehensive SAVA monitoring system that you can actually use in production environments. Throughout this course, you have mastered variables, loops, conditionals, functions, debugging techniques, and file operations. And now it's time to apply all these skills to solve real world problem. Monitoring the health of your servo flit and responding to issues automatically. Let's put your new skills to work. All right, so let's talk about the challenge. Our challenge today is monitoring an automated server flit. As a DevOps engineer, one of your core responsibilities is ensuring system reliability. This means continuous monitoring of critical system metrics, proactive detection of potential issues, automated responses to common problems, clear alerting for issues requiring human intervention, and historical logging for analysis and improvement. Manual monitoring doesn't scale and inevitably leads to missed issues. Our solution will handle these tasks automatically for any number of servers. Alright, let's talk about our solution today, a complete monitoring system. We are going to build together a modular configurable monitoring system with these components. One, configuration management, easily add or remove servers and customize thresholds. Two, core monitoring engine, collects and analyzes metrics from all servers. Three, response system takes automated actions based on configurable rules. Four alerting system, sends notifications for critical issues, and five logging system, maintains historical data for analysis. Let's start by looking at the overall structure. Then we'll examine each component in detail. If you want to take a challenge and do it yourself, pause your video now and try to do it yourself. Alright, guys. Let's start by looking at the overall structure. So, we have the main file, the main dotsge that would be the entry point into our application. We'll begin by the Shiban the Bin Bash, and then drill down into the files. The first file is the source configuration file, the monitor config. It exists right here. After that, we have the setting up of the logging. We have the log file variable, which inputs into Log Deal monitor undercoe the date with the specific format and the log file. All right, we are creating the directory and touching the initial log file. After that, we'll source, meaning input the different functions, the login functions, the metric functions, alert functions, and action functions. And so begins the main. We are logging the details using the log function, we hold in login functions and initialized monitoring. Looping over all the servers in the servers array, and of course, logging. And so beginning collecting any analyzing server metrics. After that, we are running all configured checks for the server and eventually slipping until the next check interval. Finally, this file, we are checking if the debug flag is at one, so we enable debug logging with the minus X. Okay, guys, let's drill down into the first function, the component one, configuration management. We'll go into the file monitor on fig and we can see the configuration. We set up a servers array that inputs all the servers DNS names we want to monitor. We input the check interval in seconds, 300 seconds, and the threshold when to alert. Dig threshold, for example, load threshold, memory threshold, and pause to check. Of course, you can modify to match your own environment. Finally, we have the response configuration, what to do automatically when we detect an issue. For example, restarting services, and how much times do we start before we escalate, meaning alert. Also, we deal with notification, email alerts, slack alerts, and where to input. For example, the alert email or the Slack web finally, we talk about the log in configuration, the log deal, how much there is to save the date, meaning log retention days, 30, and the debug flag. A, this configuration file demonstrates the power of variables we covered in earlier chapters. By changing these values, you can customize the system without modifying the core code. All right, let's go into the second component, the monitoring functions library. This file resides here in metric functions. Okay. So what this file does? These functions demonstrate our use of conditionals and functions for chapters four and five. Each function has a single responsibility making the code modular and maintainable. Let's go over some examples. The first example is the check Silva connectivity. What we do is a simple pink check to the DNS. If the pink is successful, we return zero, meaning success. If the pink fails, we return one. In many cases, the pink command will be blocked in real world scenarios. You can find as a challenge, a different command that helps you check in connectivity for the server. The second function we'll go over is the check disk usage. This function, SSg to the server and checks if it has enough disk space. If it is below the threshold we set up in the ult configure, we put an alert. All right. We can also check the load the CPU load on the server using the SAG or the memory usage. Or maybe check that critical processes are not running and alerting or if we want to run automated escalation, we can restart the services. If the restart fails, we can put an alert. And finally, we will run the wall health check suit using the check disk usage, check load average, check memo usage, and check critical processes. If all passes, we log it as all health checks passed. If not, we can alert one or more health checks filed for the server. Now, you can try to write this code or you can use our own repository which you will find right in our Github channel. Let's go over the response system. The response system tells us what to do when we need to respond to several issues. These functions demonstrates the use of associative arrays and advanced bash features to tac state and take appropriate actions based on several conditions. Let's go over some example. So we began by initializing the monitoring. We declare the variables as we learned in previous chapter, and so we begin with monitoring it. We run the update server status function to endle state transitions. What happens when a server state changes from healthy to unalPy or the other way around. If a server, for example, is recovered, we can send recovery notification. Amazing, right? Also, what you do when a server is unreachable. We can update the server status to be unreachable, meaning nLPI and tiger and alot. What you do when we need to restart the service, or what to do when we need to enter some other stuff. We can always log and alert. Okay, let's talk about one of the most important functions as part of your DevOps journey. How to do an alert, sending notifications as escalations, alerting your teammates when something needs to be done quickly. So these functions demonstrate using ase statements from Chapter four to handle different alert types, providing appropriate context for each situation. Let's go over some example. We put the local names type, server, resource, value threshold and message, and we utilize the switch case. Case, for example, disk type alert. We put a message, alert, I disk usage on server and resource. Maybe you have a memory issue, so we put the messages alert, I memo usage. Eventually, we log the alert and send it to the log file. If we set up to enable email alerts, we can send email alerts with the message. Or maybe we put enable Slack alerts, so we put Slack alerts. When sending email alert, we do that and use the mail command with all the details. Of course, this is just a simplistic mail. We can utilize much more advanced content using HDML email to show the monitor essay with colors and stuff that alerts. Of course, we can use Slack alert. Using the call command, we can utilize the Slack webbook and send a message to a specific channel. Finally, we can send recovery alert recovery alert, meaning the server return to HeLFI status. That's something we want to see when we do automated monitoring. Alright, let's talk about the login system. The log in system is the most important part of your journey. Every system needs logs. Let's see what we can do with login. We wrote the log function, which gets parameters level message and the timestamp. Formats the log entry in the way we want timestamp, level message, and goes over different stuff. So we print to console, if not in background mode, and we utilize the colors, red for errors and alerts, yellow for warnings, green for info, or sin for debug. That shows colos to the log file and to the console. Clos alerts us and puts our attention at most. Finally, we have a function, cleanup logs to find log files that are older than the log retention days variable, in that case, 30 days and deletes the files. Of course, we utilize the Is statement to show up an in case of failure to clean up log files. Just a reminder that all this code is found in our Github, which you can see right in the course nodes. All right. So how do we run this code? To use this system, we first create the directory structure. We make deal with lib files and log files, S which component to its appropriate file, the main script, the configuration, deliraries and make the main script executable. CH mode plus X, the main Dot sage. Finally, to run the system, we run the following command, which you can see right behind. We do. At one and re run Min Sage. We want on it now because we need to define the service we want to test, but you can run on your system. For production use, you would typically set this up as a system D service to ensure it starts automatically with the system and doesn't need manual intervention at all. Finally, let's talk about how we can extend the system, giving some further ideas. This monitoring system demonstrates the power of bas scrapting for DevOps automation, but it can be extended in many ways. Web dashboard, for example, at a simple web interface to view server status, historical metrics, store metrics in a database for trend analysis, additional checks, add checks for service specific metrics, integration with other tools. You can connect with PomtusGrafana, or other worldwide monitoring system, and maybe autoscan response, integrate with Cloud APIs to add resources when needed. So how about debugging the system? Remember, debugging techniques we covered in Chapter six, here's how to apply them to troubleshoot the system. We can put the flag, debug equals one, and then made a sage. We can run with Trace mood. How we do that the following minus six, and then main 30 sage. Also, we can check logs for specific issue. For example, arrow on the file valve, log server Monitor dot log, which will show you all the rows you can find in the log and many more. Now, this is the final chapter for this course. Let's talk about some conclusions. First, congratulations. You have now built a complete production ready DevOps automation system using pure bass scripting. This project incorporated all the key concepts from our course, variables and configurations, command execution and file operations, loops, conditionals and key statements, functions and modularity. And debugging techniques. The power of bash scripting lies in its obiquity and simplicity. With the skills you have learned this course, you can now create sophisticated automation solutions that will save you time, reduce errors, and make your infrastructure more reliable. Thank you for joining us throughout this course. We hope you'll apply these techniques to create your own automation solutions and continue your journey into Devops Excellence. W the final challenge. Sure, customize this monitoring system for your own environment. Add at least one new check type and one new alerting method. Test it on your servers and observe how it helps you identify and respond to issues more quickly. Thank you so much for joining. Bye bye.