Programming Graphics III: Painting with Sound

1. HYPE update: to record a quick little video of an update. When I first did these courses some four years ago, processing to was was with the thing. Um and in conjunction with using processing to hype and how I described hype in all of these classes was also a dot PDE file. So, um, and in these three classes, you had a hype dot pt hype dot pd contained the entire hype library. And that also meant that you you could use things like processing J s at the time. Right? But now lots changed. Its four years later. Um, processing Js really isn't a thing much anymore, but P five Js is kind of the new new right. And of course, we have processing three. Well, uh, since I was doing mawr on an application basis and not really using the browser anymore, um, we actually hup updated hype to a proper processing library. So what I've done is I've gone through all of these classes that I had previously done four years ago, using hype P. D. E, and processing to and updated them so that they were using processing three and hype the proper processing library. So I just wanted to show you really quick. If you go on to get hub dot com slash hype, you will see that there is hype processing right here. This is the Master Repo. However. I just want to show you that if you click branches and inside of branches, there is lib. Staging lib Staging is always the most current. It's where we're pushing the most stuff, the most changes making adjustments. It's always like kind of the work and process before it actually gets pushed to master. So because we want the new knew the secret new new uh, let's go ahead and go into lib staging and you're just gonna click this green button that says Clone or download, and then you're gonna say Download zip. Now what this is doing is is downloading the entire hype library, including the source code and everything. But we want to use it inside of processing, right? So you'll notice that if you go to your documents file processing, there's a folder called Libraries. And in the case of this particular example, uh, my folder is empty, so I am going toe unzip the zip that I just downloaded and you know I should should get this guy. And what this has is as reference and examples, and resource is really the only thing that you need to look at is the distribution folder. The distribution means this is the current build that you can push inside of your documents . Process in libraries holder. So go into distribution and you will notice that there was hype dot jar and hype dot zip. Let's go ahead and unzip hype dot zip, and it will create this folder called Hype, and you're just going to drag that folder to your processing libraries folder. Right? And so inside of this hype, he should have example, library reference and source. Right? And if you look at examples like, Oh my gosh, there's a ton of examples here Now, if we were to launch processing the I d. E, you would notice that if you came, it's probably tell me to update. No, Yes, no. Let's go to sketch import library. Look, you should see hype there. So now hype is a proper processing library. So the question is, does that mean can I use this in a browser? No, you can't use this in a browser. If you wanted to output your stuff to drop a script, I don't. I suggest using this environment all. Go ahead and look at the P five J s project That's happening, Which which is processing for the browser in Java script. But if you want to update all of these class files that I've done over the past couple of years, I moved to this system. Now, where you're gonna be using processing three and then you're using hype the processing library. Right, So cool. So we've got this inside of, um, inside of processing and just for the heck of it, if I was Teoh, move this aside. And here is some of these skill share and gum road classes that I have been working on and you will notice if I go into programming graphics one. Uh, I have now included in this zip all of the updates to this code to use processing three and to use hype the library. Right. So if you go to, um, let's just say drawing visual assets and you were to go and look inside the folder, you'll notice that hype dot PTE is now removed completely from all of the projects. And if you click on build PDE, you will notice up at the top here that we've got a new structure, which is now that it is a proper processing library. You have Teoh import hype as an external library. So at the top, you can see I've said import hype dot asterix, which means loaded in if you want to use other things. So if you just are importing hype, you kind of get the bare bones that's needed in order to Teoh to use hype with processing. Three. If you want extra stuff like ham using a color pool arm using a grid layout, then you have to also manually import. Um, the class that handles would do whatever it is you're trying to use. The best thing to do is to go into it examples and say, Hey, I want to use an oscillator. So if you look at all sailor, you'll notice I've got import oscillator. That means that this keeps the hype library very thin, whereas before, with hyped up PTE, you were importing everything, even if you weren't using at all. So this is lean. This is mean. It's much more efficient a zey proper processing library. You import the classes that you want to talk to. Eso again. Look at the examples folder. It will show you how to structure things. In addition to that, if you look at this zip file, you will see that all the files air there just like they were before. But they all have been updated to support this new structure. I'm gonna do that for all the classes. And I'm obviously going to use this structure for all new classes going forward. Um, hope this helps chat soon. 2. INTRO: All right, Uh, thought I would record a little intro video. Um, this is programming graphics three painting with sound. And I am super excited about this class. It ends up that I have actually two new classes I actually wrote first all of these files for texture, mapping and many controllers because I thought, you know, I'll show you how to do kind of these visuals that I do for for bands when they go on tour . And I wrote all these files and then after it was done running his files, I thought, Well, crap, we should probably cover sound first, because sound is going to play a role in that class. So I took all that could set it aside, and actually route 42 files for this class, which is called Painting The Sound 42. The answer to life, universe and everything. Coincidence? Probably. No. Um, I desperately tried to get 42. Um, and I nailed it. Nailed it. Um, awesome. Uh, listen, in this video, I'm just gonna cover some housekeeping stuff, which is, um, I work on this library called hype and hype. Is is just a tool to help you do things quicker, easier and so on. But check it out. We actually don't cover hype until build 17. So if you're just using straight processing code, of course, this class is gonna be super helpful around build 17. We'll start to introduce some of my hype library, and things will just even get more awesome. Um, so great. Um, awesome. 42 examples all covering sound. Now, um, we're going to be using a library here called Minimum. And, um, I just wanted to point out that something is happening with processing right now, which is the stable version of processing is processing 2 to 1. And right now, processing three is in beta. Okay, of course, these files are gonna work in both environments, but there is a little bit of a shift, which is in processing 2 to 1. The sound library that we're going to be using called minimum, was actually built into processing 2 to 1. So if you look at my documents folder and you look at processing 2 to 1 and you come to libraries, it's actually empty. You could actually write minimum code and fire it. No problem with the transition of processing three, they've actually pulled minimum out of processing three. So you'll notice if you look at documents processing three libraries. I actually had to import minimum, and I did that by coming. Teoh the processing three I d. E. You could come to sketch. You could come to import library. And of course, here you can see I've already added minimum, but you could say add library and this pulls up the little processing library search tool and right here under filter, you could actually write minimum and install it yourself. Now, since there are still some people who have not migrated over to processing three I'm actually gonna be writing all of the code with processing 2 to 1 in mind. I will show maybe a little bit later how processing three makes things just a little bit more smooth. So, ideally, we want to get everybody to processing three. But again, the code for these 42 examples will work in both versions of processing. Okay, great. There is going to be a video in every single one of these builds on, and that specific video will be talking about the code for that build. And, um, I hope that this will be super helpful because I have found that most of the examples um specifically with minimum aren't very helpful for visual artists. You know, how do you tap into sound to trip or something, or to move in animation or two to do stuff that us visual artists want to do? So I found that some of the minimum examples could be a little bit daunting. If you're tackling sound synthesis for the first time, Um, cool, we're going to be working with minimum. Specifically, we're going to be working with F f T, which is fast for your transform eso again. This wraps up this little kind of intro kind of things that you should think about before we get started. Um, while there has been enhancements to the processing three i d. E. My personal preference is to write code and sublime text. So that's just the environment that I'm most comfortable with. By all means, you can write code in any environment that you're comfortable with. Um, so I'm gonna be writing code in sublime text to we're going to be used minimum. We're going to be using FFT. We're going to be using hype. And hopefully, through this process of these 42 files, you'll find a way to actually use sound, um, to your visual advantage. All right, I will see you in the video for build one season. 3. Build 01: who? Who? All right, build one. Um, Okay, now build one. It was only 30 lines of code, and this probably could be the easiest sketch. Ah, for us to run. But I kind of want to explain some of the reasons behind what I'm doing. So we're gonna be working with minimum, and we're gonna be working with FFT fast for your transform. And what that's going to do is it's gonna take audio right, which is waves, and it's gonna convert them into, um, numbers, highs and lows. And that spectrum of audio is 256 bands. OK, so it's a number between zero and 2 55 zero being the deepest A base and 2 55 being under no dog music. Uh, right. So you're deep bass and your high tweets, your high pitch noises. Um, so, up at the top here on line one. I've said that the my audio ranges is 256. Okay, Now, I'm actually gonna run this sketch because as I talk through this settle, it'll start to make a little bit of sense here. So here, I'm gonna I'm gonna run the sketch and again, there's there's not much happening, and I'll explain why. So the audio range is 256 and I want to visualize that band of 256 sounds, Uh, and I want it to happen in the center of the screen, but I want to like a little buffer space around around my sketch here. So check out Line five. I say that the stage margin is 100. So if you look that's putting ah, 100 pixels above 100 pixels below 100 pixels to the left and 100 pixels to the right. So I've just got 100 pixel breathing room around where I'm gonna visualize this information . OK, now, uh, what? I'm visualizing these 256 bands. I want a view. Each band is like run pixel and then, like I want one pixel breathing space and then the next band, then one pixel breathing space and the next band. Right So online. Six. On creating basically the sketches stage with and stage height, right, nu Uh, before we do stage with let's just go and do stage height because it's it's it's easy stage height. This is 300. Why is it 300? Because it's 100 of the top 100 of the bottom. And then that visualization. I want to stick within that 100 pixels in the middle. And I'm gonna talk a little bit of more about that in a bit. Now, stage with I've said Okay, I want my audio arranged times too. So it's taking 256 times, too. And that's because I'm gonna do one pixel space, one pixel space, one pixel space and so on. So I'm taking my audio range and times it by two. And then I'm saying, plus stage margin times two, which is 100 pixels on the left and 100 pixels on the right. Okay, Line 11. I set the background color and then 15 through 18. I'm constructing the set up. This is the sketches, constructor, and I say, Hey, sizes that stage within stage height that you just calculated up above. And then on the draw again, I'm setting the background color again. And here I'm just setting some lines to visualize the margins in the space is okay. Now you might be wondering, Ok, that's cool. So you set this up here is the left edge. Here's the right edge. Here's the top edge and here is the bottom edge. But that bottom edge I've changed the color to orange. Okay, Now my goal is is that in this box right here in the center eyes where I'm going to visualize the audio. And again, it's one pixel art, one pixel space, one pixel art, one pixel space. So this is 2 56 times too. And I want individualization to start on this line right here, right. So I'm basically gonna draw a bunch of lines or rectangles I haven't decided yet, which which pushed down. Now, the reason why I turned this line oranges because my goal is to look at these 256 bands and understand that this line is kind of the zero point. So if if the band is on this line than it's probably zero hence, that band is not broadcasting any audio information, OK? And my goal at the end of the day, once we get through some of these files, is is my goal is to get a number between zero and 100 right? So this orange line right here represents ah 100 pixels beneath where the start, uh, is for these boxes. So you'll see a little bit later as we start visualising some of this audio, I'm gonna be looking for that audio to break past this orange line, and then we're gonna do something called Normalization and normalize the audio across the spectrum. And so the goal is is is that I want to listen toe audio, and I want to take that spectrum and basically get numbers between zero and 100 right? And so again, this line is going to represent the zero axis, and this line is going to represent the 100 access. So I'm gonna be looking at that spectrum going up and down, and I'm gonna look for it to be hitting this orange line. So that's why I have colored this orange. Okay, so this build one again, we're not We're not doing any audio stuff, but I'm just showing you how we're mapping out this sketch. And once you moved in to build two, we're going to see the process of this unfolding. The sends this video. Hopefully you'll see how this is all gonna come together over the next couple of sketches and I'll see you in built to 4. Build 02: Okay, here we are and built to, um and let's just walk a little bit farther down this path. Build three is where we're actually going to start to load in some audio. But before I do that, I want to build these boxes. And these boxes are the things that I'm going to be expanding to. Visualize the audio range. Okay. Ah. So let's look at some of the changes here in the code. Audie arranged to 56. Still the same stage margin stage with stage height still the same. Okay, so line 789 haven't changed. But now I introduced this new interred your online five where I say rectangle sizes too. Right. So I want these boxes that I build to be two pixels, Okay, and then online's 11 through 13. I am going to specify where these bands start on the X axis, where they start on the y axis. And actually, what is their spacing? So line 11 I say x, start this stage margin. So basically, it's going to start at 100 on the X axis and on the why start is also stage margin. So basically it's going to start drawing that very first band band zero of the spectrum at at 101 100. Okay. And then I say, Well, the X spacing is rectangle size, so the spacing is is two pixels. So it's gonna draw a to pixel blocks than space two pixels. That a to pixel box in space. Two pixels. Okay, remember I said I wanted, like, a space in between. I've decided to do that with a stroke, so I'm gonna use rectangles, but because the rectangle is going to have a black stroke around it, you'll see basically one pixel Phil. And then you've got the stroke, which will be that kind of space between the pixels. Okay, set up remains the same, but the draw Now, I've removed some of those line visualizations and let me go ahead and run this sketch before I start talking about line 29. So, um, first is is actually, let's look at lines 34 through 35. Um, I just clean up the line a little bit. That orange, the orange line of death again. That line represents where I want these things to push past, right. I want mine for numbers between zero and 100 I'll show you in a few sketches how we're gonna get there. So again, I've said that the stroke line is lunch, and I just put that line at the margin. I do that, Um, 2 56 times, Two for the for the width. Here and again, I end, um, 100 pixels off of the right hand side. Now, let's go in and look at this. I've got now 256 boxes that are two pixels in width and height right now, in order to do something a set number of times. Of course, this is a four loop. So I go ahead and say, um four loop run while I is less than 256. So I know that this four loop is gonna run 256 times. I go ahead and set. Um, you stroke and Phil. Now some of you might be asking, Why in god's name would you put the stroke and fill inside of the four loop? Why not put it outside of the four loop? You'll see in some later sketches? I'm actually going to do some color coding, so I'm gonna look for specific bands in the spectrum and color them specifically because I think that that's where audio action is happening. So I'm putting actually the stroke and fill inside of the four loop because in some further sketches here, you'll see that we're gonna mess around with some color coding. So rather than having it outside the four loop and then tell you that I'm putting it in the four loop in other sketches, why not just keep it consistent? So again, I set the stroke to black, and I set the fill toe white. Now let's look at this rectangle the rectangle here online 31. Of course, when you're setting a rectangle that's you can use for attributes X y width and height. So I've said that the exposition is X start right, which is 100 plus I times X spacing. So I times, too, because the X spacing Aziz Rex eyes wreck sizes to now. Of course, when this runs for the first time zero times +20 So we we know that the first rectangle is gonna be at 101 100. The next one is gonna be at 1021 to the next one's going one for 14 And so on is it runs down the 256 items. Okay, So, um, why spacing is why start? Why start is 100. So again, we know that it's gonna be 100 pixels off of the top and then look it for with and high ice . Simply pass it repped size, which, of course, is to So it has a width of two and height to now. Looking at this, this is great, because now I've got 256 rectangles. Okay, they're all two pixels by two pixels. And again over here, this is rectangle zero. So that would be the deepest of base. And over here, we're starting to get into dog music. This is those really high pitched sounds over here on the right hand side. So great this ends built to build three. We're going to look at, um, importing minimum, looking at an audiophile, loading in that audio file and playing the audio file seeing built tree 5. Build 03: so before, Um, we actually getting high, Satan. Let's try that again. Hi. How are you? Uh, build three. Um, before we actually write the code, um, toe load in some audio. I need to send a huge amount of love, um, to my German brother from another mother, Ben Boysen, who also goes under the moniker. Heck, H e c Q Um, and he has actually been super awesome. And he donated a bunch of his way files for me to listen to so that I could, uh, give you a way file to use in all of these sketches. Okay, so, um, definitely send him some love because that was super rad. I tried to pick away file that I thought had a lot of good range to it. A lot of base, a lot of high tones that would be good to work with. And we're gonna be using this over the next, you know, 39 files. Okay. So, um, Ben, thank you. You are awesome. And actually way have matching tattoos. It's what you do. It's what you do get matching tattoos with people that you love. Um so awesome. Check out his work. H e CQ dot e And we're going to be using a track called with angels off of the Avenger album. So So check that out. Super, Super cool. Um, Okay, Ben. Thank you. Eso Look at build three. Bill three, We're going to load in an audio file. And I noticed that that audio file is a wave file. Okay, you'll see on build eight. We're gonna talk about the difference between wave and MP three and what? That what that means, uh, from from audio. So, look, we've got a data folder and inside of that data folder, we've got a lovely little text file saying Then I love you, bro. Right? Super awesome that he let us have this way file. Of course, there's the way file itself again. It's called With Angels. This is actually a remix of The With Angel Song by Trifon IQ. Again, it's it kind of really is fits from an audio point of view, kind of all the different things that we want to hear across the spectrum. So I think it's a pretty good, pretty good file on, Of course, I put in a little of injured J peg, which I just think is radical. Well, OK, so most important is inside of that data file is we have this wave file and this way file. If you look at the information down here, the sample rate is 44,100 and the bits per sample of 16. I did notice that minimum has problems with playing back any audiophile. That's 24 bits per, uh, sample. I noticed that some of the files that Ben gave me they were 24 bits per sample and minimum just basically shits the bed, so I had Teoh convert them down toe 16. So again, sample rate 34,100 bits per sample. 16. All right, let's look at the code here. So I'm just gonna move this over about the top online. One you're going to say import gdf dot minimum dot Asterix Asterix means pull in most of the base classes for this library online's three and four line three. I create an instance of the minimum library. So I say minimum is minimum notice that it is a capital M in a lower case M and then I want to create an audio player object in that audio player object is what's going going to allow us to load in an audio file and play back that audio file. So notice online. Four, I say audio player is my audio capital A, uh, all of this other business has remained the same. That is unchanged from build too. So I'm just gonna skip rate over that Now look at lines 30 through 33. Online foodie. I want to fire off minimums library constructor. So in order to do that, you say minimum equals new minimum, and then you point it to this. OK, so that basically is going to fire the minimums. Libraries Constructor line 31. I'm gonna say OK, well, I'd like to load in on audio file. So because online four audio player is a my made that object called my audio, I say my audio equals minimum dot load file. And here is the name of the file that I'm trying to load. Notice that if it's in the data folder, you don't need to specify data slash and then the name of the way file processing will automatically go into a data folder to look for content. So notice that the load file is just the name of the way file, and it's past is a string in quotes. Now online 32. I just show you some different play options. Line 32 is my audio dot loop, so obviously that is repeat play. So once the song is finished, it will actually start to play again infinitely. Right? So if the song is one minute at minute minute, he'll actually replay audio over again. Man Hi, Line 33. If you call my audio notice, I have this commented out. If you call my audio dot play that it's going to play the file just once, so it will start. Play that and then after it's done, it will not repeat play. So I did put those two commands in there. Notice that I'm gonna go ahead and use Line 32 which is the my audio loop. So it'll just keep playing over and over and over again rather than stopping once it gets to the end. Now, um, the draw Nothing has changed lines 36 through 46 or exactly as they were in build to Nothing has changed there. Now minimum does want you to put this stop function here at the bottom. You might notice this in some of the examples when you're scrubbing through the minimum examples inside of your process in the library's folder. And this just simply says, If you close the movie, let's go ahead and and shut it down, Right? So you're saying my audio dot close so it actually closes the audio player. It tells minimum to stop doing audio related stuffs, and then we also call a super dot stop. So you'll see in most of the minimum examples. They do want you to put this function at the end, which just shuts everything down If and when you close your movie. Okay, so now nothing is happening visually in this file. We're just literally loading in an audio file and playing it. But nothing is changing visually in our sketch. So if I were to go ahead and run this, uh, again, you'll notice in the sketch that nothing is happening visually. But we have, in fact, loaded in that audio file, and we're actually repeat playing. We're moving it. So this file, we will continue to play and play and play until we actually close it. And again if you listen to what's kind of happening audio spectrum wise, some good sounds awesome. Deep bases. We've got a nice snare. You've got some really nice. All right, let's move on, Teoh, build four and let's actually tie this audio to something visually on screen seeing built for. 6. Build 04: build for, um, build four is where we're going to start to tie these two things together. So we're going to be looking at that audio. We're going to be playing that audio. We're going to be using FFT fast for your transform. And then we're going to tie that audio to our little squares up at the top. So we are in build for data file. Nothing has changed, bill dot pd. So let's go ahead and open this and you'll notice that, um I've written a lot of comments in here. Okay, So in this particular file, we're gonna introduce something new, which is fft fast for your transform. So in order to do that, we're gonna have to include a new import. So you notice online to have said import gdf dot minimum dot analysis dot Astra's now again dot Asterix is going to import in all of the class files that are associate ID with analysis. Right. So we have to add a new import to start doing this fft business. Uh, lying four and five. Still the same. Same as before line six. We introduced something new, so we say fft um we're gonna create this FFT object and we're gonna call it my audio fft. Okay. Now, um, I've tried to include some comments here so that you can look at fast for your transform. There's some really great Wikipedia information so that you can start to look at what FFT does and essentially is your taking, um audio? That's happening over time. It's way file and we're converting it into numbers were converting it into some numbers that we can start Teoh attached to some type of artwork visualization on screen. All right. So again, if you're new to FFT, by all means hit the Wikipedia links, there s so that you can read up on what FFT is all about. Okay, Um, of course, nothing changes here. All of this is still exactly the same you can see here. Online's 58 through 60. This is also from the previous file that is loading in the way file and line 60 is We're looping that that that way. File. Now online 62 63 we're going to start Teoh, listen to and set up our FFT business. So online 62 I say that my audio fft is a new fft. So I create a new FFT object and it wants to know to arguments that wants to know buffer, size and sample rate. So I said that the buffer size is we'll just go look at the audio. Um, go look at the audio and get the buffer size from it. And the second argument says, Well, what's the sample rate? And again, I also say, Hey, go to that my audio object and get the sample rate yourself right? So when I'm creating this FFT and I am asked four buffer size and sample rate, I'm simply pointing to the my audio object and getting the buffer size and sample rate from the audio itself. Now line 63. There's a couple of different things that you can do here, and I like Lynn averages, which is linear averages. There also is log averages, which is logarithmic averages. But Lynn averages, I think, is gonna suit us just fine. So I say my audio fft dot Lynn averages is, uh, my audio range. Okay. And, uh, my audio range, as you know, is 256 from appear on online eight. Okay. And so you can see a little comment here that this is going to calculate the average is by grouping frequency bands, literally right. But eso here's right now we're not really doing any groupings were actually looking at each individual band in the spectrum, right? You'll see where this is gonna come into Play in some later sketches is maybe I don't want to listen to each band individually, like Band zero band, one band to band three. Maybe I want to group them so, like group the basis together, group the mid tones together and group the high tones together until so forth. So you're going to see where this Lynn averages is going to change in some later sketches. Right now, we're listening to every single band in the spectrum, so it's getting giving us the linear average for all £256. All right, let's go down into the, uh, into the draw function, and you can see right before our four loop Um, we run a forward on our FFT. So Line 71 says my audio FFT got forward my audio dot mix and again I put a little comment here. I just said perform a foreword fft on the samples in the audio mixed buffer Note that if the audio were amano file that this would be the same is using audio dot left or audio dot Right, Okay, so we're gonna call a forward on the FFT before we actually do our little visualization here. So great. Now we've got this for loop. This world loop is where we've been attaching all of our little boxes up at the top. And the difference is is that we're now going to create this this new flute and that is a temp index average. Right? So sensible. Your average Joe's group Equally numbers of adjacent frequency bands weaken simply pretty. Calculate how many pixels with high each average ellipse should be in again. I didn't use the lips, so I'll change this to wrecked. Um great. So online indie, I say temp. It's afloat. Temp index average is equal to my audio. FFT get the average of sub item I so get the average of zero. Get the average of one, get the average of two all the way up to 255 Now I've made a little adjustment to the rectangle right. So the exposition is still the same. The Y position is still the same. It's whip is still the same. It's with the still wrecked size, which, of course, up above you'll see it's que What has changed is the height of the rectangle, and the height of the rectangle is now this new temp index average. Okay, so that is now tying the height of our rectangles to the linear average of each band of the spectrum. So let's go ahead and run of this sketch. Okay? So what do we see here? Way, obviously. Now, see, the rectangles are expanding their height based on, uh, what it hears from the audio. I'm just gonna let this run for a little bit, so that, especially when it's a little quiet here in the beginning, and now it's gonna build up. Well, really Watch. So great. Um, not what I expected, right? It ends up that I'm just gonna lower the audio a little bit here. Um, not at all what I expected. In fact, you know, I was I was kind of really hoping that all of this audio was going expand past this orange line, but it really looks like only the first couple of bands in the spectrum actually go past this, this orange line. So, uh, it ends up that when you do, uh, on fft Lynn averages and you get the average of each band, You you're going to see this kind of slope where the very first part of the spectrum, you're gonna get some big information and that it actually tapers off. And it seems like almost nothing is happening on this end of the spectrum. Uh, So what I need to do now is I need to figure out a way to amplify the audio so that when I amplify this audio, um, I'm seeing a lot more action push past this orange line. So I need to figure out how to address that that perv and again, you'll see this in the next couple of sketches. And now I need to figure out how to amplify this audio so that the number data that I'm getting when I say amplify I'm not talking about the actual audio that you here. I'm talking about the members that I'm getting. I need to amplify the numbers so that I see something a lot more drastic than this. So notice that if you don't do any amplification, if you're just literally, uh, talking to FFT getting those Lynn averages and then getting an average Actually, the number data is quite small. So I need to figure out how to amplify that audio information. And I also need to figure out how to deal with this this curve of the FFT number information, But great, at least in this boat four sketch We have audio plane way have FFT running, and we have FFT giving us number data, and we're assigning that number data to the height of directing. So we're seeing something, but it's not right where I need it to be. So let's move on to build five and keep walking this progress the next. 7. Build 05: and build four. We discussed that. The number information that we're getting is very tiny is very small and build five. We're going Teoh, uh, think about this idea of amplification. And so if we going to build five, uh, let's go ahead and open up. Bill PDE here. And, um, a lot of this is still the same. Uh, you'll notice here. Um, I online 10. I've introduced a new float, and I've said my audio amp is equal to 20. Okay, so whatever number information is coming in, I want to multiply it by this audio amplification. And again just to reiterate, I'm not talking about the actual amplification of the audio. You hear? We're talking about the amplification of the number information that we're getting from FFT . Okay, so line 10 my audio amp for amplification is equal to 20 again. The rest of these variables remain the same. The set up, nothing has changed in the set up. This is exactly the same as it waas. In the previous build where we're loading in the way file were looping the way file. We're creating an FFT object and getting the lin averages of the FFT object now in the draw . I've introduced, uh, a couple of new things. The first thing is, is that I'm just gonna skip past the four loop here for a second, because you'll see that, actually online 56 57 online 56 I change the stroke color to a teal blue and online 57 I create a busy a curve. Now, uh, this, I hope, will help visualize what I was talking about in the previous file, which is F 50 has a slope to it. It has a busy a slope to it that there was a great amount of good numbers in the beginning . And as it proceeded across the bands of the spectrum, they slowly tapered off. And that whole back apple, like nothing was happening at all. So I thought I would create a little busy a curve there just to help us visualize like what the F f t information actually looks like. All right. Now, another change is line 49. So in the previous file, that float for temp index average was just my audio fft dot Get averaged on sub item. I write so 01234 Here's where I'm saying, Multiply that by my audio amplitude, which in this case is 20. So it's going toe Take that number data that we're getting and multiply it by 20. So we're gonna get some magnification of the numbers Now, watch what happens when I run this sketch. The first thing that I want to address is this blue line. See, this is what I was talking about. If you're looking at those white boxes, you can actually see that there is kind of a curb slope to that number. Information, right. It's really robust in the beginning. And then it just slowly sort of exponentially starts to slope off into, uh, not good number data towards the back. Right, So, so cool That move wine at least just helps us visualize what that FFT kind of slope looks like. OK, but now check it out. Now we're getting a much better representation of of audio by adding that audio amplification by taking that number information and multiplying it by 20. And when I do that, I'm able to see a lot more of the information. And I'm also seeing that a lot of these lines are breaking that that orange line that's 100 pixels off, Right? Because again, I'm gonna reiterate this again Eventually. I want to look at the spectrum and get a number between zero and 100. So the fact that I see some of these breaking past 100 is actually kind of a good thing you'll see in the next while we're gonna constrain it so that we only get a number between zero and 100 right now, I'm just checking to see if if that number date is actually exceeding past that warms line , which in a lot of cases it is, but way haven't even addressed that slope yet. And we're gonna address this in a couple of sketches down the road where we're gonna try to normalize so that we're not actually getting a slope were actually breading kind of the same sound information entirely across the spectrum. So there isn't a curb. There's the same kind of audio information happening in zero that could also potentially happen in 2 55 Right? So cool. We're just getting a little bit further down this process, this progress. Um I will see you in the next build where we'll address constraining against that 8. Build 06: this is build number six. And, um, we're gonna do something and build six. That I'm gonna immediately break in build seven because we are crazy like that. It's just what we're gonna do. But build six at least introduces an idea that again, we're gonna break it, build seven. But we're gonna come back to later on. So we're going to introduce it in six. We're gonna talk about why it's awesome. And then we're gonna break it in the next couple of files only to return back to it. Because ultimately we want We want to get to this idea that we introduce into to build six . So, uh, let's go ahead and open up. Build PD e. And almost everything is identical, except for two changes on and the changes are down here on, uh, wine 51 52. So online 50. We introduced this audio amplification, right? We said, get the f f t A data, get the average for each one of the bands in the spectrum and multiply it by 20. Right? So Line 50 was amplifying that number data that we were getting. But remember I told you that our goal Our ultimate goal is to get a number between zero and 100. And that orange line represents that 100. Yeah. So we saw that a bunch of those rectangles were exceeding past that orange line, which was cool. We could look at that file and go Awesome. I see that the amplification is cool because a lot of those rectangles are exceeding past that orange line. So online 51 we introduce a new float, and we say, um, going Actually, that's not true. Yeah, I added one line. Appear at the top, my audio, max. And again, I'm trying to get a number between zero and 100. So I put that my audio max equal to 100. Yeah, and this is gonna come into play down on that line 51. So my audio max is equal 100 because my goal is get is to get a number between zero and my audio max, which is 100. All right. Going down the line 51 cause I forgot that one variable. Uh, line 51. I introduced a new float, and I say a temp index Khan for constrained, and we're actually gonna call processing is constrained. And what constrain allows you to dio is past three arguments. The first argument is what kind of information is coming in. And then the next two arguments are What are the men in the max? So little my constraining This too. Right. So I understand some audio and some audio information is coming in, and, uh, those audio numbers might be larger than I need them to be. And maybe I want to smash them or constrain them Two men and a max. So I say, Cool. Let's constrain the thing that's coming in. Is this temp index average, which is the FFT? Get average times 20 times That, Uh uh, that amplification. Okay, I know that the number is gonna go past 100. Okay, so I'm saying that the men is zero and the max is our my audio max of 100. Okay, so now that information is coming in. But now I'm smashing it to a number between zero and 100. So, yes, I do know for a fact that those numbers go past it, but I'm forcing a ceiling onto that number information. All right. And then what? I changed online. 52 is is the height of the rectangle is no longer temp index average. It's now temp index con. So again, I know that that rectangle is going to be a height between zero and 100. So now if I run the sketch, you should notice that it's it's almost identical to the previous build. Except for as those rectangles are giving us number information. Uh, I'm saying that they cannot exceed, uh, past my audio max, which happens to be 100. So I do. This is kind of ah, window into where we're going to get to later, which is this is awesome. I can load in a way, file. I can amplify that number information and then constrain its min and Max. So in this case, um, um, I'm getting numbers between zero and 100 now, uh, we haven't addressed the slope. You'll notice that that blue line is still in there and again, you're still seeing that the rectangles air actually adhering to that slope. So in the next file, I want to talk about normalization. How do I get this FFT number information in How do I amplify it? And then how'd oh, I try to eliminate that slope? How do I try to make it normalized? So that zero is giving me a number between zero and 100 sub item to 55 is getting me a number between zero and 100. How do you normalize the numbers so that they're giving me the kind of same consistent number across the entire spectrum. That's where we really want to get to. But again, in this next build, we're actually gonna break constrained. We're gonna We're gonna We're gonna get rid of it. But I'm introducing it here because you can see in some later sketches once we work out normalization, Um, we're gonna want to constrain it again. So it's good that maybe talk about constrained. Now break out of it. Work on this concept of normalization, uh, only to introduce constrain later on. So again, this is kind of a nice window into where we're trying to get, which is listening to audio, getting that number, data, amplifying that number data and constraining it to a minimum axe. And then later this would be so easy to tie to animation to attach to a whole wide a range of properties because we're number mining. Three entire spectrum on. We're looking at getting a number between zero and 100. Okay, this ends this build hopefully again. You can see the progress and I will see you in the next build. 9. Build 07: build seven. Uh, k This this one, there's a lot of stuff that we got to cover. This is probably gonna be a long video build. Seven has been tagged green, Um, here because this kind of represents a point where Oh, my gosh, all of the components that we need or kind of in place in this in this one file and this build seven also represents kind of like my go to file when I'm loading in a new audio file . I'm probably gonna load it into this build seven because this built seven kind of helps me see where all the number information is at and just helps me see it efficiently. So that one I'm going to do something later on down the road. And some of these sketches build seven is kind of like that perfect footprint of of, of where I need to be with a specific audiophile tons to cover. There's also some P and G's in here, which are going to showcase window ing, which we're gonna talk about. That's sort of the end of this, the end of this discussion. So, man a ton to cover in this sketch, okay? And the first is this idea of trying to normalize the audio, right? Have been talking about it now for a couple of sketches. Couple builds. We put him that busy a curve to kind of represent what the FFT number data looks like. And I want to do the reverse of that. So in the beginning of the spectrum, I wanted to be kind of mellow, and then I wanted to get more amplified towards the back. So I almost want to do the reverse of that Busy a to the FFT data so that I'm getting kind of the same amount of numbers across the entire length of the spectrum. So how am I going to do that? So, love at the top, I have introduced three new variables. Okay. So, essentially, what I want to do is is I want Teoh, um, kind of reversed that that busy a curve. So, yes, I understand a number is coming in and I'm amplifying it times 20. But I also want toe amplify each individual band. So that way, not only my amplifying the overall spectrum on incrementally amplifying each band so that that number gets larger and larger and larger and larger, larger across the entire range of the spectrum. All right, let's do this. Uh, line 12. Where to start? So I create this thing called my audio index online 12. And I say, Hey, my audio index, let's start with a number like 0.5 That's the starting point. Okay? And then I have this next float online 13 which is my audio index. Amp, I'm gonna amplify and notice. I say, hey, immediately, go get my audio index. So my audio index amplification is also equal to 0.5 But as you can imagine, we're going to increment uh, my audio index amp Well, what are the steps that we're going toe amplify my audio index? Amp. And that is actually the next flu. So Line 14 says that the my audio index step is 0.25 So basically, I'm going to be multiplying, um, the step incrementally on to the amplification. Right? And again, you're going to see down in the sketch. I've actually included a little thing so that you can actually visualize how we're amplifying across the spectrum. You'll see it a little bit farther down um, I actually changed line 22 Stage height State right Was 300. I wanna make it 700. So I've got a much greater space at the bottom of my sketch so that I could actually see what's happening. Because, as I told you in the previous build, we're going Teoh, turn constrain off. I actually want to see what's going on before I constrain it. I'm gonna constrain it later in a couple of sketches, but I actually need toe to really see what's happening. So I opened up the stage height to be a little bit larger so that I can actually visualize what's going on now. Inside of sit up. There is one change, and, um, I'm gonna address it. Ah, a little bit later, and that's running a window on your FFT data and window is like a filter. It's like filtering the audio, but we'll come back to this, so please skip over this, But this is in addition. But the only addition to set up is, um, the FFT window call. Okay, so let's go down into, uh, draw. All right. Um, one thing is is that I'm gonna make some changes to how this thing paints. So notice online. 66. I comment out the background color. I actually don't want to clear the screen. I want to keep painting to the screen so that I am seeing, uh, time right. So instead of clearing time, I'm actually leaving sort of a fingerprint of time. And that allows me to see what happened to the audio over the operation of the audio across the entire spectrum. So I'm not going to wipe background, so I comment outline 66. That also means that I'm gonna make just a little adjustment to my fill color. Um, and what I'm doing here is is I'm saying that the fill color is white, but it's Alfa is only five. So it's going to be very minimum white. And because I'm not wiping the background that will slowly build up and slowly build up slowly build up, and so it will get whiter and whiter and whiter in areas where it keeps, you know, painting over itself. So I also changed the fill color just toe have the doubtful of white rather than 2 55 so 2 55 would be white to 55 is Alfa. So I'm saying to 55 5 for the Alfa. Okay, um, I'm gonna skip over line 73 because I'm going to show you online. 74? Yes. I actually commented out constrained. I don't want to use it in this particular sketch. That also means in line 75 that I revert back to using the rectangles height to temp index average back to this number rather than constrained. Now, let's take a little quick look here at a temp index average because I've modified this a bit. I've said I've wrapped in parentheses this first bit, which is, um, my audio fft get average, multiply it by our amplifier, which is 20. And then I say, Cool. Let's also multiply it by my audio index amp, right. So I'm just gonna scroll back up to the top again, and you'll notice that it's this number right here. My audio index amp, not the step, not the base, which is my audio index. It's my audio index, AMP which yes, for the first time, it 0.5 Cool. Okay, so, uh, this temp index average online 73 is now multiplying the index the individual index Amplification. Okay. Now, uh, I'm gonna skip over some stuff because, uh, down here online 81. Ah, here's where we're stepping up the index amplification. So I say my audio index amp Plus equals, uh, my audio index step step. That means itself. Plus that, uh, audio index step. Now that audio index step is incremental each time by 0.25 So as you move down, the index right step is is adding itself onto each one of the sub items. So it's just getting bigger and bigger and bigger and bigger and bigger as it moves across the spectrum. So not only you amplifying the spectrum as a whole, but you're individually amplifying each index as it goes further down the index. Right, Because we wanted to get router in numbers towards the back, right towards 2 55 quieter towards the front zero. Right, Because we had that that curve that already showed that it was kind of loud in the front and tapered off in the back. Okay, So rocket rate down here online 81. Uh, I'm plus equaling my audio index step now, after this is over, I reset right, because it needs to build up, build up, build up, build up. But it does that each time in the draw. So online. 83 I say. OK, yeah. I got to start over again. So my India, I'm trying my best here my audio index amp is equal to my audio index so resets itself back to its base. Setting in that base setting is 0.5 So it amplifies it in the four loop. And then after the four loop is over, I need to reset it back to its base. Because if I didn't do that, that plus equals it would just keep getting exponentially bigger and bigger and bigger and bigger and bigger every single time that draws running. No, I only wanted to do it in the four loop, and then I wanted to reset itself when draw runs again so that the for loop is basically getting the same exact a single index amplification each time. So again, after this four loop here is finished, I reset the index amp back to its base, which is that 0.5 So my audio index AMP. Is equal to my audio index. Okay, um, now I've done some other things on here. That again, I'm gonna that have to do with the window ing. Let's back up all address it in a minute. So now you understand why I put the stroke and fill inside of a four loop Because, uh, here, online 77 through 79 I wanted to help you see that step happening, right? So I put in a new stroke. The stroke colors green. And I said, No, Phil. And I'm going to draw some lines. And again, this line 79 is huge. But what it's doing is is creating lines that air two pixels and with, But they're attaching the y axis in relation to the amplification. Okay? And it'll this will best be illustrated when I actually run the sketch. So let's go ahead and run it. Now, let's forget about the window wing and all this other stuff. Um, it's nice now because, uh, what that grew line now represents is where amplifying the audio across the entire spectrum . But we're also amplifying each individual index across the spectrum, and so you can see in the beginning here, uh, I'm not adding any much application at all. But then each time that index keeps growing and growing and growing and growing and growing so that I'm basically doing like a reverse curve. Now, the reason why I have turned line 66 off well, so that I could see what's happening to the audio over the course of its duration. Right? And now check this out. The fact that this kind of, uh let's say it looks like rain this streaking this rain. Um, yes, there are some instances where there are certain indices in the spectrum where it just goes bananas. Look at this one. This one jumped all the way down here. However, my overall feeling about this is awesome because that sort of sheet of rain, that sort of sheet of green, it's cut. It's kind of the same because look at this. Like if I were to start right here and just kind of draw my mouse across, that's rad. That's read that that all of these lines, how much are pushing past this orange line, and they're doing it somewhat amusing that loosely, somewhat evenly, right? So I'm seeing this kind of vale of white a crossed evenly. Someone not perfect again. Somebody's gonna jump out more than others. But all in all right. The fact that if I If I just move my mouse caught through this area right here, Yes, with the exception of back here, the very, very end and that has to do with the way of file There just there just isn't a whole lot of audio information all the way at the back end of this of this spectrum. But all in all, this looks great, because the audio information is staying pretty consistent all the way across. And it it kinda is giving me a sheet that is somewhat normalized. And again, this this green line is representing that normalization. It's representing the amplification of each index across the spectrum. Awesome high fives, high fives. Everybody 0.2 people wink. So what celebrities do. Ah, that's great. So we're getting in the FFT information. We're amplifying it all is a whole times 20 But then we're individually amplifying each embassy of the spectrum as a greater number as it moves across. Cool. Now let's address some of the other stuff. Um, appear on line 47. I put a nice link to Wikipedia for win doing. And if you look at the Wikipedia page window is great. It allows you. Teoh basically kind of do like what I'll say is like a filter on the audio. And there's a ton of different windows that you can that you could look at and they're all named differently. So you've got, like, a rectangular window, a triangle window. Welch hand Haughn, however, it hamming homing tomato tomato. Let's call anyway. Great link toe look at, um, window function. Okay, and it ends up that minimum. Has some window filters built in right? So online. 62. In this particular case, I'm saying, Do not use a, um, window filter on the audio, but you can see I put some links. There's, um, bar DeLaet. There's Blackman. There's co sign. There's Ghazi in hamming Hawn, triangular so on and so forth. So you can actually apply a ah filter to the audio, which will change very minimally. It will change the kind of FFT numbers that you're getting. Yeah, now again, I've said to do to do none. Um, but I did do some stuff down here, which is I wanted to be able to save out these images. What I wanted to do is is is, uh, subscribed to a filter. Let the entire audio play, and then after that, audio is done playing basically send out a saved image that shows the representation of the entire song over the entire duration of the song and save it as a za PNG. So, um, one thing that you'll notice with this is is that online's 88 through 91? Um, I just put in some processing text that would just spit out what filter that I'm currently listening to So you can see here under text, I put window fft none. And then look at this online 93 through 97. I put this Well, let me let me just back up before I talk about this. I did make an adjustment here online 40 where I'm not looping the audio anymore. I'm just playing at once, and when it's done, it's done. And the reason why I did that is for down here at the bottom online 93 through 97. I put in if statement that just said If my audio dot is playing returns false. And that's what the little exclamation point or bang is in front of that function call is, while the audio is playing, it will be passing true. And then, actually, when the audio is done, it will pass false. And if it passes false, I know that the auto audio is no longer playing. I then go ahead and say, Save the frame. I gave it a specific name and then I actually exit or quit the sketch, right? So if you were to play this all the way through, let the audio play all the way through, it will actually save out of PNG and exit your sketch. Now I'm going to minimize this really quick because I actually saved out P and G's for all the different types of window filters that you could actually apply to this audio. And if I just open this up here, you'll see watch. Vory This sort of vale of white changes, right? So there's none right. And as I run through these, I can see how I'm sort of filtering or compressing Um, that number information. And so in some cases, Ghazi in might be perfect again. Look at that Yes, I understand that some of these peaks breakthrough, but again, like, look at where I'm moving my mouse like great here. That's awesome. Because I'm getting essentially the same kind of, uh, peaks, highs and lows across the entire spectrum. Because we did apply this green line, we did apply this amplification of each individual, um, band across the spectrum. Wow. Okay, a lot to take in. But build seven is awesome because it really represents kind of this fingerprint of the audio during its duration of playing. And that means that every audiophiles different some audio is gonna be more bass heavy. Some might not have a lot of base and some might have some higher pitched sounds. So number seven really is kind of. This is why I tagged it green. It represents a file where I can I can load in a piece of audio. I can sort of define what I want that audio amplification to be to normalize the audio and then get it to a place where I can look at this and go Yes, that really looks normalized across the entire spectrum. So So build seven again. It's it's tagged, it's It's it's highlighted because this is a great place to start to experiment, loading in different types of wave files. Okay, I'm gonna shut this down. Bill Date is really going to address this idea of why way files. Why are we using way files? So it is wave versus MP three and understanding what MP three compression does and what MP three compression does specifically to our FFT number data. So build seven. Loving it. Um, see you on building. 10. Build 08: So here we are. Build eight, Which is Ah, rave files versus MP threes. And, um, why are we using way files? Um, why aren't we using MP threes? Well, uh, an MP three is It is a compressed audio format. And we say that and, you know, you hear people talk about, like, you know, MP three, you know, quality is not as good is like a record player or un compressed audio files. Eso were We've been using wave files because really files are un compressed. Um, audio files. Now, in this build, it's more about the images that I've generated than it is the actual sketch itself. Um, because what I've done is is I've shown you the difference between I'm compressed and, uh, compressed audio. Okay, so, inside of the data photo folder, you'll see there's our initial wave file that we've been using. But there's also, um, five MP three files, and they are taking that way file and rendering them to, uh, to MP three and a specific bit rate. So you'll see here that we've got a new MP three at a bit rate of 128. Ah, a bit rate of 1 60 a bit rate of 1 92 a bit rate of 2 56 and a bit rate of 3 20 Now it's important to realize that a lot of default settings on ripping CDs to MP three is actually 1 60 That's actually in a lot of cases. Is default for a lot of, um, CDs that you've probably converted to MP three unless you went in and changed it to a higher bit rate. Most of the time, at least for me. A lot of the CDs that I've burned over the past couple of years I've noticed that that default bit rate is actually 1 60 that's that's kind of an important issue here. So let's check this out again. Really? These p and G's are the focus of this build. Um, this is our way file, okay? And you'll notice that, um, the way file is un compressed audio. Okay, so we've got bits per second, and you can see that looking at this song over the duration of the, um um over the operation of the song itself, you can see we're getting a lot of information happening visually. Okay, Now look at what happens when we take the same exact sketch and we have it. Look at the MP three with a bit rate of 1 28 which, of course, is not good. Okay, so check it out. You'll notice that I've drawn this little green line, and I've marked it that this is the 128 bit rate. So with MP three does is is it actually compresses the the audio file. And in addition to that, in order to make MP three small right, cause that was the goal was un compressed audio where larger file sizes. So in order to make MP three is really easy to throw around. Look what they did. They actually chopped off the back end of the spectrum. Right? So there's really no audio information at all coming from the back end of this. Thats audiophile. In fact, it's safe to say that added 128 bit rate. You're losing 1/4 of the audio. You're actually losing this part of the spectrum because they figure out Yeah, that's dog music or that stuff that you're not gonna here. And so we're actually gonna chop that off and we're actually gonna compress the audio so that the file sizes smaller. So of course, not only does the audio quality suffer, but you're losing actually that data. So this becomes problematic because with our code, um, we're really only getting number ranges in this In this in this area here, we're actually not getting any audio information from FFT when losing 1/4 of the spectrum. Now, let's go ahead and look at a bit rate of 160 you can see that in this image. Okay, we've opened up a little bit, right? So on again over the past couple of years, and I've been converting CDs to MP three. Look at how much audio information I was actually losing. Um, and so it's nice to see these two lines. And again, I mark this one blue because for me, this was default for years. This is what I was ripping CD's at, um, is this 160 bit rate? So look at what happened. So we go from 1 28 toe 1 60 we just gain a little bit of the spectrum back. Let's go ahead and jump upto 1 92 So this is a bit rate of 1 92 So now I've marked where 1 28 is where 1 60 is and where 1 92 is, and you can almost start to see the pattern here. It's like, Oh, okay, I get it. Maybe. Right. So this was how much I opened from 1 28 to 1 60 And it seems like from 1 61 92 I actually doubled the amount of spectrum that I'm that I'm getting back. I'm still losing a nice chunk of the the the information here, but I can see what's happening. It's it's it's gonna double, right? Look at what happens when you jump to 2 56 Right? So this is a bit rate of 2 56 You actually only get a small sliver back. So what I was expecting actually didn't occur at all. I thought it was gonna double each time. Like I was just gonna get more of the spectrum back. And actually, I kind of thought like off I get 23 20. Maybe I'm back to the amount of information that it would get in my way file, but as you're going to see, that's actually not the case. So jumping from 1 92 to 2 56 we actually only get a small chunk of the spectrum back, and it's actually less than the jump from 1 28 to 1 60 And believe it or not, if you actually go to 3 20 the sliver actually gets even smaller. So even at that 320 bit rate, we're still losing this back end of the spectrum. So, really, this build eight was just a exercise and showing you how much information you lose by using MP three is your audio source. So I understand that maybe MP three is might be the only thing that certain people have access to, and that's cool. But just understand that obviously, this is gonna affect this environment because we're really on Li like again. If I were to jump back Teoh 1 60 for example, it's like I'm losing 1/5 of the audio spectrum. So if I'm if I'm using this spectrum to control animation or do stuff on screen, really, all the action's gonna happen in this front end of the spectrum. and there is a part in the back that I'm actually going to be, uh, going to be losing. So really, the purpose of this building was just a look at this difference between wave un compressed audio and MP three compressed audio. And actually, how much of the number data you're going to sacrifice? So again, if you go into this build and you actually open up this build PTE e most of the code is is identical to like it was in the previous build. Except for here. I'm I'm loading an MP three rather than wave. And I put in all of these other markers just draw lines to show you again where the drop off point is for MP three and down here at the bottom. I also commented this out, but I was also doing that saved stuff. So after the image was over, it would actually save out my PNG. So really, this, uh and again, like, if I were to do this, run this sketch right now just for the hell of it. Um, you know, again, I'm showing you these markers, but really, this is just a exercise to show you that you actually lose spectrum numbers by using MP three's? A supposed to to weigh files. Okay, so this really ends this build eight, which is like, Hey, man, if you can get access to un compressed audio, you're just going to get better results because you're going to be getting, ah, lot of diverse numbers across the entire spectrum. If you're using MP threes based on the bit rate in which those MP threes were created, there is a portion of the spectrum that you're not going to be getting any number data at all. All right, um, we're gonna move on to the next couple of builds eso we're gonna We're gonna jump into build nine right now where we're going. Toe add constrained back. So remember we abandoned constrained in build nine. We're going to implement constrain back into our file. So see you in the next sketch. 11. Build 09: here we are in built nine and in a lot of ways builds nine through 13. The next five sketches, these videos, they're going to be pretty short. They kind of clean things up. Um, so build seven again. Kind of got tagged as this. This, uh, this moment that I wanted you to focus on, You know, I'm saying, like, Oh, this is kind of like, you know, um, the perfect file to be benchmarking audiophiles against build nine. After we've had the way versus MP three discussion build nine. Just kind of packages everything up. Super nice. All right, so let's go ahead and inside of build nine. Let's go ahead and open Build PD. And there really isn't anything new here. Um, a lot of this code is is the same as it was before, but we re introduce constrained. So down here online 54 if you remember, was temp index average. That was getting the average of the spectrum the Indus e each into sea of the spectrum, multiplying by anam vilification Times 20 and then multiplying it by index amplification so that each individual index was, uh, getting amplified. So that was great. And um, build seven. Let us see everything that was happening. So now it's kind of time to reintroduce constrained, so you'll notice on line 55. Uh, temp index Khan has been uncommon. Tid and again. It's saying, Hey, the input that's coming in is that temp index average, but confine its to a number between zero and 101 100 being are variable. My audio max appear online. Nine. Okay, so now we know that this float is running across the entire spectrum, and it's really going to give us some number information between zero and 100. Okay, so of course I make the modification to the rectangle. Um, I I put back this temp index con, whereas before, um, way reverted back to tempt index average. All right, so, uh, I'm gonna go ahead and run this sketch. And three important thing to see at this point is the heavens have aligned, you know, the seas have parted, the constellations have aligned in a way, that's exactly where I want to get to, which is sort of normalize the audio across the spectrum. Um, so the information on the far right is has the opportunity to be Justus, uh, showcased as it is on the left hand side. But by using constrain right now, we're only getting numbers between zero and 100. And again, it's if I watch this, you know, I'm really getting a nice fingerprint of the entire spectrum. And I can see that that in a lot of ways, I'm getting some really nice movement all the way across. Everything is reaching that orange line. And to me, this is this is exactly where I want to get. Okay, So I'm gonna go ahead and close this, and I'm gonna go ahead and close this build because Well, actually, no. Let me bring this back up. Let me set up what we're gonna cover next, which is the next couple of files are just gonna be really short and sweet. Uh, what happens when we change my audio range? Right. My audio range right now is saying 256 which is every single in to see in the spectrum. But, uh, maybe I don't want to that maybe I want to actually start to group things. Um, so in this bill nine it's 256 is the audio range. We're looking at every single independent in to see in the spectrum. All right, so I'll see you in built 10 where we start to reduce, uh, the variable for my audio range. 12. Build 10: we are in build 10. And I have actually recorded this video a couple of times because I am desperately trying to find the best way to vocalize. Uh, the reduction of the variable, my audio range. Let's try this again. This is very frustrating for me because I want to, um, you know, when I teach these classes, I want to make everything that I'm doing crystal clear, uh, and build nine. There was 256 objects on screen, and each one of those objects was listening to each individual in to see in the spectrum 0 to 2 55 But realistically, right? So if we're looking ahead, you'll notice. Build 13 is where we want to get to build 13 is taking that spectrum and reducing the spectrum down to a set of averages of 11 objects. So let me explain this to the best of my ability. Let's pretend that my audio range was 1 28 Okay, 1 28 is 2 56 divided by two. So what you're saying is, and again here, my audio range is 100. But let's pretend for a moment that my audio range was 1 28 Um, rate here where it says online 43. My audio fft dot Lynn averages my audio range. What that's doing is is saying I've got this block of spectrum 256 from 0 to 2 55 by changing my audio arrange toe. 1 28 That's half of 2 56 Basically, what you're doing is is you're saying two indices are one object, right? So, uh, in two c zero, an industry one would be grouped together, and you're would be averaging those two indices together. Um, in two. C three and into C four would be grouped together as one object. So you're again if my audio rangers won 28 you're taking that to 56 splitting it into groups of two. Yeah. So you're still getting the entire spectrum. But you're just saying I want a group it into a series of blocks. Yes. That is exactly the words that I wanted to come from my brain to my mouth hole clear. You're still getting the entire spectrum. But working with 256 is crap. I just want to get the essence of the audio. And that means that I'm gonna need to reduce the my audio range down to a set of numbers that I feel best represents. The audio that I'm hearing right off. That's the base. That's the snare that somebody singing, Um, this are the high tweet sounds. You know, I want to reduce my audio range down to a number that I feel best represents The audio to 56. It's too much. Too much information. Okay, so with these examples, right? Specifically these these next ones that recovering, I'm tryingto trying to find that Holy Grail. Number of pairings. You can see him Go, 13. I'm gonna get this shit to 11. Is 11 the right number for you? Have no idea. Because I don't? No. The kind of audio that you're using And maybe your sweet numbers. Five. Maybe your sweet numbers 25. Okay, but we can all agree that 256 is just It's too much information. Let's group some of this stuff together. Okay? Now, another problem occurs. OK. Here on the left hand side is build mine, which is looking at all 256 indices. And you can see our audio amplification was 20 are index started at 0.5 and our step was 0.25 Now look at what happens and I've rendered out these p and G so that I can show you the adjustment of the bars if I reduce the my audio range down to 100. And let's just ignore what I have here from 11 through 14. If I were to keep the same numbers from my 256 this is what you would actually get. Okay, um, it's not enough amplification. It's not enough amplification across the whole thing. And it's not enough ample amplification for each individual indices, so you can look up here at the top. I've changed the range to 100. I've kept the amp 20. I've kept this START index at 0.5 I've kept the stepping at 0.25 And look, so much of this data never even makes it. Nan did back So much of this data got my mouth. Sometimes my brain is talking faster than my mouth can accommodate. My problem Certainly not yours. Hi. Where were we wait. You can see clearly from this PNG that these settings won't work because some of this some of these indices don't even make it past the Orange Line. So oh, my God, If I reduce the number of range right, if I start to do these groupings and then I'm doing a linear average on these groupings across the spectrum, that's ultimately gonna affect, um needs to change the amplification and stepping and so on and so forth, Right? So let's try something new here, so you'll see that the next image I said, All right, let's change some of the settings. Let's change the amplification from 20 to 25. Let's do the Start Index at 0.5 Let's up the step to 0.5 and again, it's like a lot of information happening. Um, it seems like they all kind of moved past the orange. But let's look at this grouping here for for what will call into C zero, which this range here, like the base kind of really just barely gets past the Orange line. But the rest of it kind of really aggressively does make it past the Orange Line so maybe again, I just need to massage these numbers a little bit more. So I say, Okay, awesome. Let's let's just up the amplification on here upping the amplification to 30. I'm upping with the start index to 0.75 and I'm gonna go ahead and keep, like, the last file that step of 0.5 The window is none. And again, Okay. All right. So now I'm getting that kind of sheet of rain. I'm getting something that's all moving past the orange line, and but it seems a bit aggressive. So here's what window Incan come into play. So this Ford file I just said this fourth PNG. I just said a look. You know, if I were to apply a window filter of hamming that just kind of softens the filter on the on the audio spectrum. Yeah. So as we reduce my audio range, you are going to have to massage your amplification numbers because as it starts, toe linear average these new groupings across the spectrum, you're amplification probably isn't going to be enough. You're going to need to adjust it as we start to reduce that range. Now uh, I'm gonna go ahead and look at, um, my PDE here and again. These air kind of like, Hey, if I were to change my audio range 200 to me, it seems like I massaged these numbers to a good place, which is 30 0.75 and a step of 0.5 I also changed the rectangle size because, um, if I'm reducing the range, then why not up my rectangle size to be something a little bit bigger so that I can really start to just see the rectangles a little bit more prominent. So here I changed the rectangle size to be to be five. Um, And again, I've commented some stuff out, but there's some you know, there's some some bits here, um, that you obviously can uncommon, meant and start to start tinker with if I go ahead and run the sketch. Um, again, I'm just I'm watching the audio. I'm watching my amplification numbers. I'm sort of giving a sense. I want the lows, and I want the highs. And I want those highs to go past the orange and I want those Little is to settle back up to zero. So I'm really looking to see how this flows. Um, with these new amplification numbers and, um, awesome. This this This looks good. So we're gonna move on to build 11 and we're gonna move on to watch what happens as we keep reducing, uh, the article range. How do we have to change the amplification numbers to accommodate the reduction of the audio range and trying to get to that sweet spot? That sweet spot. That's really for this particular audio. Um, use a completely different audio. I'm I may have to change my numbers. Maybe, uh, production needs to be smaller. Means to be larger. Maybe the amplification needs to be greater. Smaller. Maybe. I need to apply a different window ing or know little wind doing filter. Right. So it's it's it's really just fine tuning and massaging these variables so that, aesthetically, uh, the audio is giving me something that I'm going to tie too, um, to animation. Great. Let's Ah, let's wrap up this build and I'll see you in Bill 11 13. Build 11: great. We are in Build 11 and build 11 is now reducing my audio range down to 50. Now, if you take 2 56 and divided by 50 it's about 5.1. Teoh bans per grouping, right? So it's like thinking the 1st 5 bands and the spectrum are actually grouped together Now, uh, So again, with this one, let's look at these p and G's. Um, if I were to open up the first PNG here, um, this has the settings, like in our on our build nine, which is the 256 right? Look at what happens when you if we were to keep the amplification of 20. The index at 0.5 The step it 0.25 window none, right? Oh, my God. Nothing, uh, reaches past the orange line. So again, we're if we're opening up. Ah, the groupings, right. Got about five bands per group. Now, we're obviously going to have to keep drastically changing our numbers to accommodate where we're trying to get to, which is passed. That one a lot. So here I made the amplification. This is the second PNG I made the amplification. 30 That index 0.75 And I said, Hey, let's step 0.1 now, right, And again, like looking at this. Look at that base. The base, like, just barely gets past the orange, and and this doesn't even make it past it all. And who knows? It may not. So in that foot, PNG. I'm gonna change the variables a little bit more. I said, Okay, maybe amplification doesn't need to be 30. Maybe it needs to be 25. I can start the index at 0.125 and step 0.175 And then I put a window filter of Gaussian and again, like there is enough spread across the entire spectrum that they all reach past this past this orange line. And again, it's like slowly fine tuning, massaging those variables to accommodate what I'm trying to get as I keep opening up, Um, the groupings for my audio range. So now my audio range is 50. So let's go ahead and open up the build. Let's go ahead and fire the sketch and, you know, again, if we just watch this for a few seconds thing is stuck. Um, again, if we watch us, I'm looking for, um, good lows and the good highs. And I'm getting a nice amount of movement again across the entire spectrum with these particular amplification variable settings with my audio ranged knocked down to 50. Okay, um, cool. Let's move on to the next. Let's reduce it again down to 25 build 12. So let's end this, and I'll see you in the next bill. 14. Build 12: build 12. All right. You understand what's going on? Right? We've just been slowly progressing this down again. Here is just another snapshot. We're going to reduce my audio range to 25. Right? Um, so let's do that to 56. Divided by 25. It's like 10 bands in a group. Yeah, Um, cool. Look at this, P and G again, this is the base settings. See what I'm saying. The I'm gonna do this, Like, for the next five sketches. Massaging the variables, right? If I kept that amplification of 20 that index of 0.5 in that step of 0.25 Yeah, it's no good. So, uh, here again, I'm just trying to figure out what looks best. Um, I did an amplification of 40 and index of 0.175 a step of 0.175 and window filter hamming and again. You know, I'm just I'm just gonna keep doing these until I get that kind of that kind of footprint that I'm looking for. So again, um, I brought the amplification back down to 30 uh, moved to the index to 0.17 and moved the step to 0.225 Window, window, window e o would be Oh, window window filter Ghazi in. Ah, and good. It's This is what I'm looking for. I'm looking for a pretty consistent, uh, line past this orange again Because of the nature of the audio, some of them are going to break past, um, awesome. So again, if I just do a little sketch and I forgot to mention in the 1st 1 you know, I've been upping the wrecked size have been opening that up. I think in the last one rep size was 10. And now, obviously it's 20. So as I'm reducing the my audio range, I can actually open up my rectangle a little bit more. Um, okay. So again, if I run this sketch on, but we'll just set it off here to the side again, I can I Can I just check here These my settings 30 17.22 Yes, I can kind of watch how this is gonna move to make sure that I'm getting my zeros getting my hundreds right and getting good movement between those zeros. Hundreds and again. I understand that some of them are gonna push, really passed, but all in a long. I'm just trying to get this nice movement across the spectrum with this new grouping. Okay, I'm gonna go ahead and close this, Uh, because again, I was I was gonna try to keep these things short. Boom, Bill 13. We're gonna reduce it down to this beautiful variable of 11. My only arranges 11 and I like 11. Everything should go to 11. A few people should have left that reference. It goes to 11. Uh, build 13. We're gonna reduce my audio range down to 11. Which again is just this great for me for this particular piece of audio. It's just a great set of groupings that really gives you a feel, um, of all the different sounds that happen across the spectrum. Um, cool. Let's close this and I'll see him. Bill 13 15. Build 13: sorry. The Miss this build 13. This is a good one. Uh, obviously, we have been reducing my audio range down to something that we're going to fall in love with. And for me, uh, my audio range 11 is the perfect the perfect variable. Of course, I'm going to show you this round of p and G's again. Look at this. Uh, by the time you use the same exact settings from 256 it again, it just doesn't work. Um, and what did I what did I figure out? 23 23 bands. So if you take 2 56 divided by 11 it's 23 ish bands per grouping. Um, OK, so let's make these initial jumps. Here's the next PNG did an amplification of 30 an index of 0.175 and a step of 0.35 A window of none in this kind of looks. OK, um but of course, you know, I'm like, I'm constantly searching for that perfect set of set of numbers. And for me, uh, this kind of worked a little bit better, which was an amplitude of 40 index of 0.2 and a step of 0.35 Window filter. Gaussian again. Just a nice Ah, nice set of numbers. Now, Um, I love this. I love 11 and we're really gonna blow this out in both 14. But let's just go ahead and run this sketch here. So again, thes files that have all been the same for the past five sketches. Except we've been changing our numbers here between 11. 14 to find that sweet spot. And this is what you're gonna do. You're gonna get an audiophile yourself, and you're gonna go through the same exact exercise I did which is loading in that way file taking the audio range and reducing it down until you you feel like you get a sense of all of the sounds and or instruments that are happening across the spectrum. But you've grouped them in a way where you can say, Ah yes, this first group of 23 bands, right? Because we said that My honor, your range is 11. This first group of 23 bands best represents what happens from base, right, And then we move up a little bit and then we move up a little bit and then you might say, Ah, this next set of bands best This next set of bands grouped together right, might best represent what's happening with the snare drum or this group of bands is this vocalists range. So I mostly hearing the vocals in this movement, right? So that's the goal. The goal is Thio Thio take away file and slowly make these reductions and adjustments so that you're getting something that best represents the file that you're loading it now, uh, excise a bump that up to 50 And, uh, let's go ahead and run this for a little bit and I'm just gonna set it off to the side here . And as I'm watching this, you know, again, I want to make sure that my sounds kind of zero out, and I want to make sure that there's good movement within that 100 range. And yet so be these moments when they shoot past that orange line. And I can sort of look at this and, uh, really get a sense of what's happening with the audio over the durations of its play. And again, I'm really looking for that movement from 0 to 100. You know, sometimes if I throw out the amplification too much, maybe there's not enough zeroing. You want to make sure that your amplification is giving you numbers that, uh, uh, groups of bands settle up to zero you want, right? Because if we're applying this to an animation, let's say we've got some item that we're creating way. Want to use rotation to be controlled by the audio. So the audio is actually controlling this object, right? You're gonna want it to go to zero. You're gonna want it to, like, settle and then spin when we get these spikes. So there's for me there that there's no magic formula. There's more like, Oh, yeah, this is just how you do it. And that's how it's gonna get applied to every single piece of audio. No, any piece of audio that I work with and band that I work with any set list that I work with . I'm slowly making these adjustments. You know, if if a band is gonna give me a set list, you know, I might go through that set list and make adjustments for each song that I know that they're gonna play so that I know that this code is reacting in the best way because one song could be heavier on base and and another song could be Could not be so, so cool. We have reached build 13 and I'm gonna go ahead and close this. I'm gonna go ahead and close this sketch. And now look at the next three bills. We're gonna move in to build 14 15 and 16 and you'll notice that in my particular files here, I have color code them, in a way. Like yellow was caution. We weren just kind of, like still caution and 16 to stop. You know, we've done it, you know, 16 is is Ah, Is is where I want to get to right? We're gonna package up. This little thing is a widget that we could just put in any single processing sketch we do forward. So these next three, we're kind of gonna work in tandem, and we're gonna slowly walk uh, that process so cool. Hopefully you realize that Yes, You're gonna have to do a little bit of massaging a little bit of finessing, but if you're gonna take the time like I do and you're gonna get these numbers into places that best represents the music. That only means that when we start to apply it to animation, we're going to get the best, most dramatic results. Cool. Just shuts down, uh, the spinal tap build build 13. My audio range goes to 11. All right, see you in build 14. 16. Build 14: the doctor. Dr believes questioning my father. Hello. Build 14. Um, I really like build 14. And I think you are too, because what I try to do is kind of clean everything up. We did this. My audio range reduction. We kind of got the amplitude numbers that we were looking for. Hello. And, um, build 14 kinda cleans all this stuff up, right? So, uh, let's go ahead and and and look at what I've done here. So most of this remains the same, right? So all of these variables were just as they were intended in, uh, build 13. The set up still remains the same. Now, I've made a few changes, and one is we can go ahead and bring the background back. So we're clearing the background now inside of this full loop. I just did a little bit of color coding because for this particular song, I thought it would be great if we could focus on to events that I think are working to our advantage, the base in the snare, right. And by watching that build 13 I was able to kind of see where the snare was happening and the sneer was actually happening in the fourth grouping, right? 0123 Uh, that would be the fourth grouping. So what I did was is inside the for loop. I said, Hey, um, the stroke color is gonna be black for everybody, but online 54 I want to make this green color. Justify is equal to zero. So if it's the first grouping of ranges, go ahead and make the fill. I think this is a dark green else. If if I is equal to three, right, So 0123 If it's the fourth grouping, that's kind of where the snare is happening in this particular song. And then I just said here online 56 Hey, for everybody else just do like gray. So we're really just focusing on the base movement, the snare movement and everybody else is just going to get this. I's gonna get this great. Now, uh, in those five bills that I had done before, we weren't using constrained. And so we sort of been using it and turning it off and then using it and turning off. Um, of course, we want to use it. Um, so here I uncommon ent out, um, temp index Khan. And of course, I update the rectangle again to use that height setting to be temp index con. So now the rectangle is always getting constrained. Teoh either zero or 100. So even though, and builds nine through 13. We were watching and visualizing the bands moving past that orange line, of course, and build 14. We want a button that up and just have them constrained between a number between zero and 100. Now, I am just gonna open this up a little bit because I've done one of the things. Which is why don't we visualize the number data that's happening for each range. So at the bottom, beneath the orange line, I'm showing you the number that each of those ranges reaches rights of either zero or 100. So I put in on this line 63 the ability to actually see, and here you can see, I just said, I want to write some text. Um, which is it's gonna be a number, right? So convert temp index con tempt index constraint to an intruder and then pass it to the, uh, pass into text is a strength. So by putting str and then putting it in parentheses, I'm basically packaging up that number from afloat to an into a strength. Right. So we should be able to see a number between zero and 100 beneath that orange line. Then I thought it would come down here and just show you that, um, what's the base? And, uh, and what's the snare? So I just put some more text in there saying, Hey, this is pretty much the base, and this is pretty much pretty much the snare. So let me go ahead and run this sketch. And again I can see down below. Here is the number data, right. I'm watching some of the zero out, and then eventually, when this song picks up, we're gonna watch this go back down 200 because, um, in our previous sketch way, noticed that they do past, uh, push past the orange line. Now, watch this. This, uh, this fourth grouping, right? If you look at all of the boxes, that's the one that really gets triggered on that snare sound. And this is what I'm talking about. I'm talking about loading in a piece of audio, reducing down my audio range and then just watching these bars, right? Which or groupings of bands and spectrum, they're averaged. And then I can really say Oh, yes, it would appear that that snare is happening in this particular range of the bands. Right? So this is what I'm talking about. You could load in a piece of audio and really get a sense of where certain sounds and instruments air happening across this band. Now, again, I've only isolated. These two have only said, Hey, this is kind of the base, and this is kind of a snare. So cool. This this build 14 really is kind of a nice snapshot of where we're trying to get, which is sounds coming in. We're using FFT were amplifying the spectrum. We're amplifying each of the, uh, groupings in the spectrum. In this case, it's 11 and, uh, being able to visualize that number data down below getting nice numbers going from 0 to 100. Now, I'm gonna go ahead and close this build 14. We're gonna move on to build 15 because what? We want to do it. 15. It start to store that number data into an array. So see, in the next bill 17. Build 15: we are in build 15 array and, um, build 15 is kind of this turning point. Because let me ask you a question. Do we actually need to see a visualization of the audio? No, we don't. Um, the these 1st 14 builds will designed in such a way where we can use it as a tool to important away file. Look at what it's doing across the spectrum. Reduce the, um the audio range massage are amplitude numbers. But at the end of the day, that build 14 was kind of ah, sneak peek into all I really wanted. Them are the numbers, right? I don't need to see the bar is moving up and down. I really just need toe mine for that number Data. And, you know, in the case of this, build 15 like, I want 11 numbers. And those 11 numbers are the groups of bands across the entire spectrum. And once I've captured those groups of numbers and those numbers are fluctuating between zero and 100 I can then start to apply those numbers. Two things on screen. I could start to apply them to properties to animations t this, that anything so this build 15 is interesting because we actually reduce a lot of the code . So if I open this up, a lot has been actually stripped out because, um, I don't actually need to visualize the sounds moving. I just want a mind for that number data. So let's just again the codes gotten a little bit shorter here. That's nice. And now let's just look at this sketch as a whole. So let's just recap. Let's just run through everything. So but the top we're importing minimum were importing FFT. We're creating an audio player object recurring an FFT object. We have her my audio range set to 11 again. My audio max is 100 were trying to mind for numbers between zero and 100. And then here are those four, uh, specific variables dealing with amplitude across the entire spectrum and amplitude for each individual groups of bands. Now online 16 we introduce an array, so I'm creating this array called my audio data. And the size of this array is my audio range Settle. I'm storing 11 numbers, right, and, uh, great. Let's go down to set up set him set up pretty much remains the same. Except for look, I've changed size now. I didn't need to do all of that measuring before with the margin and all that other stuff. I totally don't need that anymore. Uh, I'm just mining for these numbers. So now I can make my sketch any size I want. So in this case, have made the size of the sketch 700 by 700. Okay, uh, we load in our audio file, we looper audiophile, and we've got our FFT stuff we're doing Lynn averages, and you can see here. I'm applying a window filter of Gaza right now. Look at the draw. The draw is actually only three things background color, which actually were not even using because we're not actually painting anything to the screen. Um, I go ahead and line 40 do the FFT for one, and then online 41 I fire a new function called my audio data update. Right. So I know that this function is gonna fire 30 times per second. So I come down here and I actually define this my audio data update function, and I say, Yeah, I want a four loop. I want to run 11 times. And yes, I do want to calculate the index average times, the amplitude times, the individual band amplitude. Yes, I absolutely want to constrain that information to a number between zero and 100. And you know what? Save it. Save my audio data sub item. I equals whatever constrained us. So those 11 numbers are now repeatedly getting stored in our array, which is awesome, because now I can talk to that array. I can look at a specific item in the array, like maybe I just want to look at some bottom zero. What is based doing? Look at sub item three, which is the fourth ban group of bands. What a snare doing right. So now I have an array that storing out these 11 numbers, and those 11 numbers are fluctuating between zero and 100. Now, uh, down here at the bottom, I just said, go ahead and print the array of my audio data. So let's go ahead and run this sketch. And what I'm gonna do is is actually, uh, move this to the side and you can see that absolutely nothing is happening to the screen. I don't need to paint anything to the screen. I just want to mine for those numbers. So you might notice that if I come here to sublime text and actually open up the output, I can actually see, uh, the 11 numbers in the array and the numbers that they are fluctuating too. And if I were to kill the sketch again, I get a fingerprint of what was happening. So if I look at the very last output from the array, zero had gotten up to 49 6687331110 So on and so forth. So this is great. Um, this, uh, build 15 kind of really strips things down, which is, uh, cool. It was cool in the 1st 14 bills to actually see what's happening. But really, the endgame here was to actually just be storing these numbers into an array. And any time I can look at that array, I can look at all of the items in the array, or I can look at a very specific sub item, which that's full shadowing. We're obviously gonna do that a little bit later on. We're going to say, Hey, tell me what's happening with snare, which I happen to know it's sub item three or tell me what's happening with base, which is sub item zero. Um so awesome. Eso build 15. Um, is nice where we've stripped out all the stuff that we don't need. We're saving the number data now. I thought yes, maybe you would want, like, a little widget. That shows, actually, the fluctuations of the audio. So we're gonna we want to build 16 where I show you a lovely little widget that you basically can copy and paste now and all of your sketches that will help you see the spectrum when you're wanting to see it. Awesome. This gets shut down and I'll see in the next build. 18. Build 16: we are in build 16 widget. And, um, I mark this one red eye market red because, um, this kind of packages everything up nice and neat where you can copy and paste this code into a bunch of sketches and turn on or turn off the ability to actually see what the audio spectrum is doing. All right, let's go ahead and open up. Build 16 Here and again, most of the code kind of remains the same from build 15 except for appear at the top. I add a little Boolean trigger called Show visualize er equal to tour False. So for the case of this demo, I'm going to say that show visualize er's equal to equal to true so that so that we do see it down here in the draw function. You'll notice that again everything kind of remains the same, except for down here at the very bottom. I put in that conditional, I say, if show visualize er so if show visualize er is equal to true, then go ahead and fire this new function called my audio data widget. Now, I also put a comment online 45 because you want this call to the widget to actually be the very last thing in the draw function. That's because if you were drawing any other elements on screen, you want to make sure that the widget is always going to be on top. It was gonna be the last thing that gets drawn to the display. Now the function here for my audio data update hasn't changed remains completely the same from the previous built. Here we have this new function called my audio data widget. Now since show visualize er's equal to true. I know that this function is going to fire. Now I want this function. Teoh prepare itself for some things that might happen in later sketches. So, for example, you'll see that I've put some calls here that I actually have commented out. But in later sketches we will actually utilise, which is would, if we are working with a three dimensional environment and painting things in three dimensional space. And what if we are using lighting in three dimensional space so you'll see here online? 60. I actually have commented out no lights so that if I'm actually using any lights inside of my processing sketch by the time it gets down to actually drawing the widget. I'm I'm turning lights off. Um, online 61 I fire this little piece of code called hint Disabled depth test. And what that says is is if you are using a three dimensional environment and you're actually moving things around and three dimensional space, um, I want to disable, um, the depth of that three d environment so that when it actually gets to the widget, that ridge, it will actually be on top. Now, if you didn't call that line 61 what would happen is that would draw the widget. And if you had any three dimensional objects moving around, they could potentially move in front of the widget, thus blocking your view too. The stuff that you're trying to get to write so online 61 we put this little hint disable depth test. Now I'm gonna draw a rectangle across the bottom of the screen. You'll notice that I turn stroke off. I, um, using a fill of black with an Alfa transparency of 200 then I'm making a rectangle that is going to expand the width of the sketch and the height of that band is gonna be 100 and two . It's gonna be one pixel of the top one pixel on the bottom and then, ah, 100 for our little e que style rectangles down a to bottom. Now I've got a little four loop. Which, of course, I'm me introducing this for Lupin. Of course it's gonna run 11 times. And yes, of course I'm keeping that color coding in there. So online 65 I'm color coding the base online 66 color coding snare. And for everybody else, it just gets this light gray. Now online 69 I'm actually painting the rectangles, but there's a little bit of a difference In all of the previous sketches. The rectangles got drawn zero and we washed them, animate downward, right, So they were always pushing their height. What I do with the rectangle in this little widget is I actually change its Y position to the number and then change its height of the rectangle. So what that will actually do is make it look like the rectangle is actually pushing up from the bottom. And I did this to kind of emulate some of these, um little equalizer things that we used to see in an old school MP three players. Of course, this is also commented out, but very important. If you're using a three D environment, which is draw the widget with them when you're done, enable depth again because again, if you are using ah three D environment and you're moving things in three dimensional space , you want to make sure that you're turning enable depth test back. On other words, it will disable it for everything in your sketch after it draws the widget for the first time. Now let's go ahead and run this sketch, and I'm gonna go ahead and move it off to the side. Here and again, you can see we're drawing that little bar at the bottom. It's 102 pixels, and we're visualizing our little widget there in the bottom, left hand, side and again, the rectangle was constructed in such a way so that it looks like it's pushing up from the top, and I can still see my color coding and blah blah, blah, blah, blah. Now I'm gonna go ahead. Includes this because you might notice that the rectangle I actually made a with the four and I thought, Well, four is nice because you could still kind of see the bars. What? Not now. I'm going to go up to the top here and just show you a little example. What happens if I change my audio range Back to 256. Now, with the rectangles at four and the range back to 256 you're not going to see everything, you know, I would have to change the size of the rectangles. In fact, if I actually run the sketch right now, you could see that I designed it in a way so that yes, you know, the widget still had some Some weight, Teoh each of the little bands in the spectrum. But obviously, if I go back to 256 they're they're happening out here, they're getting cut off. So of course, you'd have to make in a judgment, um, adjustment to the spacing and, of course, to the size of the rectangle. But again, you know, my fooling is that 256 is way too much. So for me, that sweet spot is 11 you know, for this particular piece of audio, 11 is kind of of where it's that. So with that audio range at 11 again, I'm drawing my rectangles in a way where I can still see them. But they're not, you know, super Super Donna dominant. Um, great. So let's say that I've put this in my sketch. Everything is looking good. Uh, I could then come up to the top and just say show Visualize er is equal to false, and I could go ahead and run this sketch. And of course, now we just don't see the widget. It's still calculating everything. It's still mining from numbers. It's still pushing those numbers to Honore, but we actually don't need to visualize the widget. So this is good, because I could work on this code and then maybe I could shelve it for a little bit, and then I get a new piece of audio from somebody and wow, yeah, let me let me see what happens when I put this audio into this previous sketch that I did, um, I could I could easily put it in and then just come up to the top and just say Hey, let me just visualize that again. So show visualize er equals true. I could run the sketch again and again I would be able to see that that little visualization of the of the audio super. So this shuts down build 16 great little widget again. You can copy and paste this code into any of your sketches and you have that ability to kind of toggle it on or off if if you need to see it or not to see it, um, the next builders build 17 hype where we're gonna look at my hype library a little bit, we're going to put some hype, draw balls on the stage and start to attach some of this audio numbers from our my audio data array and start to attach it to some animation. So I will see you in build 17 19. Build 17: we are in build 17 and here we're going to introduce, um, some hype stuff. Um, now, in this build 17 there's actually a disconnect. I'm gonna show you a little bit of hype code, but it's actually not going to be attached to the audio yet. We're gonna tie it together and build 18 but I thought maybe and build 17 would spend a little bit of time just talking about about the hype library. Now, as I'm recording this video, it's it's kind of in a weird transition right now, because again, processing three has has dropped and a couple of us are actually in the process of converting hype into a proper processing library, which means it will be a jar file. It'll be something that you imported will sit in your documents library folder very similar to how minimus. But at this time we're kind of still re factoring a bunch of stuff. So hype is still just a straight PTE e file. So, uh, build 17. You'll notice that there is hype dot pd and there is build dot pd, and hopefully, if you're taking this class, you will that I've taught to previous classes the 1st 1 was programming graphics, an introduction to generative art. And that's kind of where I show you how to get up and running with hype. And then the second class was programming graphics to, um I think that one was algorithmic animation. So it was about animating stuff. Um, so of course I'm gonna use it here because hype just helps you do a lot of the heavy lifting so that you're not having to write just straight processing code. Of course, if you are familiar with processing and comfortable with process, and you don't have to use hype, but, um, I I enjoy using it on again. It kind of makes things just a tad bit tablet is here, So let's go ahead and look at this. Build 17 again. You'll notice that this sketch does have hype dot PDE in it, and I'm gonna go ahead and open up this build PTE. Now, up with the top. All of these variables are the same. They are are no different than they were before. You'll notice that in. I think pretty much the rest of these sketches you're gonna have this show visible show visualize. Er, there we go, Um, because again, you can toggle it on or off. So up here at the top, you'll see I have show Visualize er is equal to true. And, of course, down in the bottom, we've got a little call to our to our visualize er. So for maybe a few of these sketches, I will actually show you what's happening with the audio and, well, kind of get a A one in one relation to what the fertilizer is doing versus what we're actually doing with some assets on screen. Great line 22. I go ahead and add aged Wrobel pool a straw poll pools going. Allow me to paint a certain number of things on screen. I'm just cutting in a little edit right here because I forgot to mention Line 28 I'll cut this in right after I talk about each draw. Bill Poole on Long 28. I'm actually calling hypes constructor. So online 28 I say h dot in it this and then I also set the background color. So now I'm having hype actually take care of the background color as well as whether it should clear the background or or not. So, uh, here I say, Background color. And then I put in this nice, dark gray of 2020 20. And then, if you actually look down in the draw, you'll notice that I've taken out the call to background there because again, the hype constructor call is going toe. Handle the background for us, so you'll see here. Online's 38 through 56. Let's just run through this, you know, line by line. I do create a pool. The number of things that I want to create in my pool is my audio range. So it looks like I'm gonna paint 11 things. So I brought my audio spectrum down to a range of 11. Well, I want to draw 11 things on screen because again, I want to make this one on one connection to the sound out analyzation and some visual assets on screen. So I'm gonna go ahead and say, Poole added, to stage the Line 40 is going to have the asset that we're going to be painting with, and in this case, it's an H wrecked, which is like processing is rectangle, but, um, a little bit funner a tract It's going to be a size of 100 by 100 we want the corner edges toe. Have a rounding of 10. Now, I've also thrown in an H grid layout. So, um, what I'm gonna do is is, actually, I'm gonna attach all those 11 objects, those 11 h rectangles just straight in a row. So you'll see here. I've said Okay, let's make a new H grid layout. I want the grid to start at an except access of 100. I wanted to start on a Y axis of height divided by two. Right, So it's gonna take our sketch, and it's gonna find the middle on the Y axis. And again, that X axis is at an ex of 100. Now I want this facing to be 50 pixels, and I want the columns to be my audio range. So it's building a grid, but it's only building a a grid of of one row. Right? So we're painting 11 things and columns is 11. So we know that each one of our H rectangles is gonna be one right after the other 11 all in a row. Then we've got our on create method and inside of our on create We've got a call back again . This on create is what fires for each individual rectangle. So it's like a four loop. This on Crete is gonna fire 11 times. Don Online 45 would create an instance of our aged Wrobel. I just call it D, And then I'm setting some parameters for this for this trouble. First I said, I want the stroke to be black. I want the Phil to be white and it should have an Alfa of 225. Remember that in Alpha's, a number between zero and and 2 55 So this is gonna have an Alfa to 25 Now, I've also out here online 49. I've put anchor at H dot center, so that means that the center registration for this rectangle is in the middle. So if our artwork was 100 by 100 the anchor point would be at 50 50. Now, uh, I dio on line 50 say that I wanted to rotate 45 degrees, so I'm actually taking that square rotating at 45 degrees, so it's going to look like a diamond okay, Online 55. I just go ahead and say request also again. This behaves very similar to a four loop. It's gonna draw all 11 things and just show them all at once. Now, if you come down to the draw function, not much is different. The only thing that I've added is this line 63 where I'm asking hype to actually draw stage , draw those rectangles that I have constructed into H grid layout. And again the rest of this stuff is unchanged. So the my audio data update the my audio data widget. None of that stuff has changed in this file. So if I were to go ahead and run this sketch, we should see our widget down here at the bottom. And our widget is in fact showing us our 11 items in the spectrum. We still have that color coding where the base is a dark green and the snare is a is a light green. And then, very simply, I have attached 11 items into the center of the screen. Right, So it put that y axis at height divided by two. It put that first item at an ex of 100 it's done 50 spacing for each of the H rectangles. And again we've got our rounded pores attend. They rotated 45 degrees, so one and so forth. So, a to this point, we've just done a just a really quick refresher for hype. I now want to move on to build a team, because now I want to say, OK, I understand that this audio data is happening. It's getting saved to that array up at the top. And now I want to assign these objects to this information. Okay, so I'm gonna go ahead and close this sketch. This is going to end, build 17 and let's move on to build 18 where we tie the audio to our hype assets in the center of the screen. Okay? 20. Build 18: we are in build 18. And now let's connect our array of numbers to our little hype assets that are parked in the center of the screen. So let's go ahead and open up. Build PDE. Now, let's just refresh that it's his build eight building. It's this line 18 where we've created a new rail floats called my audio data, right. And the number of points and this array is my audio range so that we know that that array is 11 items. Okay, And down here in the set up, we've got our pool here, and our pool is 11 items, right? So I obviously want to tie, um, sub item zero of the array to the first item in our pool and sub item one to the second item and so on and so forth. Now, the other thing that I want to do is if you look on this line 48 here, right, we set a color of white with an Alfa of 225. What I'd like to have happen is is all of those assets that are on screen let's map the audio to the color of the artwork. So black will represent if the number is at zero, and if it pushes up to 100 then it's going to change the artwork to White. And we'll have all of the gray scale in between. All right? Nothing, actually, changes in set up. That's actually all remains the same from the previous build. Now, inside of the draw is where we're actually gonna make this update. What we want to do is is online 63 h dot draw stage. So hype is actually painting the screen. Now what I want to do is I want to actually loop through but pool and all of my 11 items and then change the color of each individual item. Okay, So, um, since my array up at the top of 012345 up to 10 I need to create a variable. That is the movement of that 01234 up to 10. So you'll notice online 67 I say. Okay. When the draw function fires, I want to create this integer. I equals 200 Obviously going to be the first item in our array. Right now, you can cycle through the elements by simply writing this online 69 which is four h draw Bill D in pool, so create an instance of draw bulls for the draw bills that are in pool. So this four loop is actually going to run 11 times because there are 11 items in our pool . Now I put those lovely comment online. 71 maybe up to this point, you've maybe been thinking in the back of your head. Why is Josh trying to get numbers between zero and 100? Maybe I don't want a number between zero and 100. Maybe I want a different number. Now He's talking about changing the color of something from black to white. Well, black and zero and white is 2 55 So doesn't he want to change that thing to be a number between zero and 2 55? Absolutely not. Um, I want my audio numbers to be a number between zero and 100 the reason why I want a number between zero and 100 is because we're going to use this absolutely beautiful thing and processing called Matt and Map allows you to do such glorious things When I learned map for the first time, I just thought, This is my new best friend. What map allows you to do is specify five arguments. The first argument is the input coming in the data coming in. What is the men of that data? What is the max of that data? And what is the men, Max that I want to map it to? So let's go ahead and look at this and you'll see this this naming convention quite consistent through the rest of the files. I'm going to say something like FFT and then what I'm trying to do, so you'll notice your online 73. I'm creating an interview called FFT Fill color because this is going to be getting numbers from FFT, and it's going to be applying it to fill color. So I say, interred your FFT fill color, and then I say, Yes, I do want an integer, and I want you to map my audio data, which is our array up at the top. Sub item zero right, because I is equal to zero online 67. So when this four loop runs for the first time, this is saying map go to the ray up at the top and get the number of that's parked in sub item zero. Now, I just want to let you know that the men and Max that's coming in is a number between zero and 100. My audio max up at the top, right? We set my audio max to 100 so I know that, uh, I've got this array up at the top. Um, it's storing some numbers, and I know that the men is always gonna be zero, and the max is always going to be 100. Now I want to take that zero and 100 map it to an entirely new set of numbers. Right? And so I've said I would like you to map between number between zero black 2 to 55 white. Right? So now what happens is is when that number up in the array starts to fluctuate between zero and 100. It's actually mapping that fluctuation as a number between zero and 2 55 So I can actually use this map structure a trillion times, and my document and I can map it two completely different sets of numbers So that's why my goal here was to get the FFT spectrum to give me number data between zero and 100 because I knew that I was gonna be using map to map it to a completely different set of numbers. All right, so I love this line 73. So now online 74 I can say I want to change this Draw Baldies fill color to whatever the mapping of FFT fill Color is within Alfa of to 25. Now this you'll notice your online 75 I say I plus plus So I zero becomes run. And again this four live is going to keep running. So go zero it goes one it goes to goes three it goes four so on as soon as it gets up to 10 right, 0 to 10 11 Uh, the for loop is finished right then we know that the draw function is going to run again and when it runs again it resets I back to zero and the four loop starts all over again. Now, of course, the rest of the stuff down below remains unchanged. So let's go ahead and run the sketch. And now Let's start toe watch what happens, which is I can see my base vibrating a little bit and I can actually see that. My asset is it'll be transitioning from a state of black through all of the great skills. And if it happens to reach 100 then it would get the brightest white. So now I have a one on one connection between the FFT numbers that I mining. And then I'm drawing these visual assets on screen, and I'm using that beautiful a method of map. What allows me to listen to an input? Know that my min and max zero and 100 then map it to an entirely new set of numbers in this case, 0 to 2 55 so that I'm actually changing fill color here now, as you probably have imagined, we need to kind of progress past this sketch, and we're gonna do that and build 19 because and build 18 were saying, Hey, we got 11 things in our spectrum, but I'm also drawing 11 things on screen. And what if I want to do more than that? So now we'll move on to build 19 and I'll show you that you can draw any number of things on screen. But we'll all have them subscribe to one of our items in our FFT number. So, um, this is going to get bananas. So I'm gonna shut down World 18 and I will see you and build 19. 21. Build 19: we are in build knowing, too. Um, so here's a little departure. Um, build 18. Was this one on one connection between the assets on screen and the things happening in the audio spectrum? Right, So there were 11 things on screen. There were 11 things in the audio spectrum, and the first range was getting mapped to the first asset in the second In the range is gonna map his second asset, but just fine. That helped us illustrate this point of how great map is and blah blah, blah, blah, blah. But no, I want to be able to paint, um, any number of things and just have them look at the 11 things down below. So let's say, for example, I wanted to pay 100 things on screen. I could very easily say I would like to paint 100 things, but please pick a random number between zero and 10 right? So the first object gets painted and it says zero in the 2nd 1 gets painted and it says eight. The next one gets painted and it says to the next one gets painted and says to as well, right, so I want to be able to paint a number of things, but we're just gonna ask them to pick one of our items from our audio array. And we do that and build 19. So let's go ahead and open up, Build P D. E. And let's look at the modifications. So the first is is online 23. That's right. We're gonna paint 25 things, so we've got 25 things, but we only have 11 items in our my audio data array. So what I need to be able to do is, is create this pool but have a way for them to subscribe to one of the items in our A ray of 11. So let's do that, uh, down online line 39. I make an adjustment to the pool to draw poor max. So now I know that pool is gonna pick 25 things. Now, I'm actually still using H grid layout and I'm still going Teoh space these things out. So I actually made a little change to the grid layout. I said, Hey, you know I want you, Teoh, have a start X axis of 50. Um, I want the start. Why still to be hyped, divided by two. But check this out. We change the spacing. The spacing is now 25 columns is now pool max. So that grid is now gonna spit out all 25 items in one row. Okay, Now, uh, what I do is is on this on Create write. That on create is what fires for each of the individual assets. So again, we know that on create is gonna run 25 times for each of our little squares that we've turned into diamonds. Now let's go ahead. Online 46. Pick a random number from our my audio range. So look at what I did online. 46. I said, Hey, I want to create this interviewer called Ran Index Random Index. And it is an integer, and please do random my audio range. So that's equivalent to picking a number between zero and 10. So every time on create fires, it is picking a random number between zero and 10 and saving it to this ran index variable . Now the issue is is that I'm painting 25 things. I want those 25 things to remember. The random number that it picked and we're gonna do that with this beautiful little thing called extras. So if you look down here when we're actually defining our little drop, All right, we've got a stroke of black. We've got Phil of white at an ALF of Of 225. We've got that anchor of center and we've got that rotation of 45. It's this beautiful little line that allows us to save information with our visual asset. And ah, lot of this thinking was kind of back, like in the flash days when you used to create an object in your library and that was a visual item. And if you used code to pull that item out on screen, you could actually write code inside of that visual assets that had some variable declarations. And any time that thing moved around, those variables moved with it. So that same kind of thinking was put into this thing called extras. And in this case, I'm asking it to remember the number. So look at what I do. I say, Hey, drop ball. You've got a little extras thing that I want you to carry along with you. It is a new H bundle H bundle is what allows us to package up or bundle up objects or variables or strings and carry them along with a draw bill. The the bundle that I want you to remember is a number is a number, and I want you to save that thing as a string called I. You know, this could say anything this could say Chainsaw kitten, And then you could say, Hey, durable, you've got a variable called Chainsaw Kitten. What is that number? Right. So that's the reference to the variable. And here I'm just calling it. I write and I is getting saved. This random number called Ran Index Awesome. So that means that this line 54 is well, more importantly, Line 46 54 are now working together. Line 46 is picking a random number. I am creating this visual asset, but I'm bringing along the random number that it picked, which again is a number between zero and 10. So I know that I can look at each draw herbal, and in this case there's 25 of them. I can look at those 25 draw balls and I can say what number to do. Pick and it'll say, You know, actually, I picked eight. I go to the next one. Go Well, what number did you pick it and go? I you know, I picked to go to next Monday, lad. Pick to a swell so that number is getting saved along with the visual asset. Hopefully, that is 1000% clear. So now what I have to have happen is is now that each of those draw balls when they're going created are saving a number with it down inside of the draw. I now need to look up that variable. So let's go ahead and move down to the to the draw function here and you'll notice that again. You know this for loop H draw body while it's in Poole, blah, blah, blah but looking. What I do online 70 and 71 online 70 I say, Hey, I want to create this temp extra as an H bundle. So I want to create this temporary thing in this temporary thing is de dot extras. So all of a sudden I'm saying, uh, hey, I want to cycle through all 25 of you and I can look at one of them and go. Hey, I I understand that you have a h bundle. Uh, I'm gonna call it temp extras. And can you just go ahead and give me your D about extra so that that visual assets is now passing the extras information to this temporary H bundle that I've created? Yeah. So that online 71 I want to talk to the specific thing that I'm trying to get to, Which in this case, is that variable I So I say, well, injured. Your eye is equal to let me look at your, uh, your h bundle. And I'm looking for a dot numb and I'm looking for something specifically called Chainsaw Kitten. Or in this case, right, So now line 71 is retrieving that random number that each one of the draw bulls pit. And I love this. You know, if we didn't have this, we would have to create a separate ray up at the top, and then we'd have to, you know, um, as the assets are good getting created, I would have to, um se This array is 25 items. And here's the random number. I picked in bubble blower, so it's kind of disconnected. What I like about extras in each bundle is that it's it's that the valuable the number that we created is actually attached to the actual physical object. And so that is awesome, because I could just say you you haven't each bundle and you have an extra and you have a numb And that number's called I What is it? And it will on this line 71. Uh, take that number and assign it to to I write. So whatever it picked for, I gets saved. Awesome. Now I can say the same stuff is before, which is I'm not doing that. I like I was in the 1st 1 ride was saying, I is equal to zero. And then in the four loop, I was saying I plus Plus, I don't need to do that anymore, because this I is now the random number that each one of the visual assets actually picked . So this coon actually remains the same because now I'm saying my audio data I but this I is the number that's was picked by each of the rectangles. Groovy. Um, so really, these two lines are pretty much the same. It's just that we've tweet what I is right now. Online 76. I just thought, Well, maybe we should visualize, actually, the numbers that they picked eso What I did was I added this little line 78 were actually set the fill color to orange. I set a text size to 12 ice, set a text the line, and then I actually say, Hey, I want you know, I want get some text and I want the text to be whatever I picked and I want its position to be draw bulls dot exposition. So now there's gonna be a piece of text that's underneath one of our each and every one of our draw bulls that actually shows you which sub item it picked in the ray. Awesome. I will stop talking, and I will just run this sketch and let's see what we get here. Okay, so we're actually seeing all of our assets on screen again. We still have 11 items in the in the spectrum, but now we're drawing. Gosh, I've already forgotten 25. Yes, 25. So we're drawing 25 things. So here's 25 things on screen, and I just want to let you know. Let's look for this last one right here, which is which will be sub item 10 right, which would actually be the 11th grouping in the spectrum because it zero through 10. So look at this one picked 10. This one picked 10. This one picks 10 and so on. So forth. Let's look for 00 is our base, which is that dark green? Uh, there's a zero right here. Here's a zero right here. And here's a zero right here, and you'll actually notice that love, those zeros actually all animate in tandem. So now we're getting somewhere really exciting because we've taken the audio we've loaded it in. We've reduced it down to the spectrum of 11. We're saving that 11 as an array of at the top, but now weaken draw any number of assets that we want, and we're just having them randomly pick one of our 11 items. Cool. I am going to shut this bill down and let's keep expanding upon this idea and I'll see you and built 22. Build 20: build 20 eyes. Just gonna be a slight modification of build 19. That should be a fairly short video. Hey, look, my T shirt. That's May. This'll was made by the ever so lovely Benjamin Fox. Uh, wonderful hype contributor. That is me on a skateboard. Very cool. Still 20 slight modification of build 19. So we drew 25 things on screen, and we were trying to find some of the numbers of, you know, what they randomly picked from 0 to 10 rights for 11 items. So I thought maybe I would do, like, a little modification of build 19 on this. Build 20. And let's Let's look at this. We're still gonna be drawn. 25 things this entire set up is still absolutely the same. Nothing has changed. What I am gonna change is actually down in the draw. So again, here's our little draw stage. And then here is our business for the four loop. And this is actually, um, just been modified ever so slightly. So, uh, here, online, 70 71. We're getting the extra. We're getting the number that each of the individual assets picked Where, uh, looking that number up into the mine audio data array. Right. And then we're mapping 0 100 to 0 to 2 55 No problem. Now, here on the slide 76 before is where I was mapping the, um, the text under each of the diamonds. And I've actually just expanded this out just a tiny little bit. Where, um, let's do some color coding of the text. And let's also move. Um, uh, the assets that get assigned to sub item 40123 which is our snare sound. So if you look here, I simply put in a simple, conditional I said, If I is equal to zero, then I know that that's our base. I change the fill color to that dark green online 80 I say else if it's if it's equal to three, that should be our snare. Let's change the fill to a light green color and hey, draw bowl. I want to change your y axis to be that height divided by two minus 1 50 So if one of our 25 diamond shapes actually picks the snare, I'm just gonna pop it up, okay? And everything else your fill color should be that orange ff 3300 And then again, this line 87 um, is very similar. A zit was in build 19 where I'm setting the text, size the text, align and then assigning the text everything else below. Unchanged. So let's go ahead and run this and let's see if we get something good. We only got one this time, so only one snare was actually picked, so I'm actually gonna close it, and I want to run it again to see if I can get those 25 to pick more than one snare. So let's run this again. Remember, this is all random, and we got to that time, right? So let's just set this off to the side. And interestingly enough, look, only one two of our shapes actually picked zero actually picked through the bass sound, and only two items picked the snare of three. So see, I'm color coding the text down below, and then I just pop these upto actually show that snare. So if we watch these again every time that snare hits, those two should should actually jumped to a white to 55. And again, this is random So let me close this and let me run it again. And can I get mawr toe? Listen to snare. Get to again? Sure. Look, that it didn't pick any zeros, right? So 25 things got drawn to the screen and it never picked zero, because again, this is totally random. And it just so happened that if it's picking a number between zero and 10 yes, there's the absolute possibility that it would not pick zero in this case. It it didn't actually completely, uh, randomly avoided that number. Wow. Surprised? Let's try this again. Come on. More than two stairs mawr than two stairs More than once there. You're killing me. But look, it did it again. It didn't pick zero again. There's no zeros. Wow, I have no idea what that means. Um, at all. Let's try this again. Come on, man. Let's do this. Come on. Ah, look at that. So, uh, fascinating. Check this out. It picked five snares and only picked one base. Right, So we'll let this one, you know, run for a little bit here. So here, you know, five of these 25 actually landed on sub item three, which is, um, again that that snare sound. So when this audio really kicks in way should really watch of these five. Blink to actually this snare sound. And surprisingly, only one base got got, which is which is funny. All right, I'm gonna go ahead and close this. This ends build 20. We painted 25 things and we have attached them to a grid layout. Let's totally switch that up in build 21. Let's get rid of our grid layout and let's just toss a bunch of things on screen and see what kind of visual animation we get. So, uh, ending Bill 20 and I'll see you and build 21. 23. Build 21: we are in build 21. And like I had mentioned before, we're gonna make a slight modification from build 20 which is let's just throw a bunch of things on screen. So in build 21 I'm going to go ahead and open up. Build P D. E. Now, the first thing that you might notice is online. Eight I've said show visualize, er false. I actually don't need to see the visualize her at this point because I know that everything is doing what it needs to do, which is I'm gonna throw a bunch of visual assets on screen. I've got this audio spectrum that's been reduced down to 11 bands and everything is is working like a fine oiled machine. So visualize, er I don't need you for this particular sketch. You're there, But please be quiet. Um, now, let's go ahead and look at Line 23. Why have 25 things when you can have 100 things? Eso will notice that I've changed pool Max to be 100 because we're gonna paint 100 assets on screen. Now there is a slight adjustment to the set up, and that has to do with pool. You'll notice that in pool here I have removed the H grid layout. I don't need to do a good layout, because I'm just gonna randomly scatter these objects across the width and height of my sketch. And so I have removed the H grid layout, and I have added down here online 53. I added a, um, method for location. So I saw dot loc and I say random with random height. So now I know that the location exposition in y position is going to be some random X and Y position based on my width of my sketch and the height of my sketch. Right. So this will scatter a bunch of things around on screen. Extras and bundle Still working absolutely the same. And actually, if you come to the draw, nothing has changed. This this'll four loop is actually the same as it was in build 20 where I'm going to be mapping the audio to gray scale 0 to 2 55 So, black Dwight, with all the grail scale in between and I'm even Well, yeah, I'm sorry. I noticed I did make a slight adjustment to the text. I am actually gonna keep the text. Uh, on top of the visual assets, you'd see the visual asset, and then there'd be a piece of text above it that would tell you again what sub item it actually randomly picked above. So I did make a little change here. Teoh Line 82. But again, if you remember, Line 78 is our base. Start green for the Phil, our lines 79 is our snare, which is a light green. And everybody else gets that lovely shade of orange FF 3300 Yes, I do love it. Um, Line 82. I made a little adjustment where I said the text is whatever it picked for I and its exposition is the draw bill dot Exe and it's why is the draw bill that Why so before? If you remember, I think that was like height divided by two. And so line 82. Now, the text is actually now following the draw bowl asset. So let's go ahead. Run this. And, um, great. We're now painting ah, 100 things on screen. They're getting scattered across the width and the height, and let's just let's just watch this like, you know, again, I can kind of get a sense, um, from my basis, right. I can see these little zeros, and so I can kind of get a sense of how many items actually picked zero. And you all see the moving together. That's the really important part of this sketch. Is is that because we have reduced Theo audio range down to 11 we can kind of see 11 groupings, right? We can we can really get a sense of Okay, I'm throwing 100 things down on screen and I can really get a sense of all their there is the base, right? And again, I could look for my threes and I could see all those things are all moving in tandem with the threes. So Bill, 21 I really like this. I really get a sense of that. We have reduced the audio down to 11 which is good, And by throwing these 100 things on screen, I'm really getting this thing. That is the essence of all the kind of different sounds that are happening. I'm going to close this and we are going to move on to build 22 bad, and I'll explain why I said bad in this one. Built 22 should be a very short video, so let's close this down and discuss badness and build 22. 24. Build 22: bird of the boom. You knew that was coming. Come on. How could you not know that that was coming? Um, build 22 bad. You know, if I could have put a question mark in that final name, I probably would have put one. Um, Hughes, Why? I think 22 is bad and just stick with me here. Let's go ahead and open up this Bill PD. And let's look at the stuff that I've changed. The first is is the whole point of this file is to demonstrate why we did this reduction down to 11. Check it out. Uh, show Visualize. Er true, My audio range. Hey, man, go right back. Up to 256 eso I moved the mine audio range away from 11 all the way back up to 256 and I even adjusted my amplitude, um, to go back to those, uh, amplitude for 256. Now. This bill 22 is very similar to build 21. It's drawing ah 100 assets with with pool Max and the set up in the draw are absolutely the same that they that they are in build Build 21. I think on this one I actually removed. I did. If you look at the draw here, I got rid of the text. So there's the four loop and I'm literally just drawing the h rex, which I've rotated 45 degrees. So they're diamonds. And I'm saying, look at what happens when you throw 100 things on screen and ask them to randomly, um, grab run of our 256 individual bands. So from 0 to 2 55 I'm just gonna run this sketch and let's just talk about this, all right? Okay. The first thing that you might notice, too is is that I made some adjustments to our widget because with the rectangles being a size of four, obviously they would. They would completely run off screen in order to see all 256. So down here online 94 you might notice that I changed the with toe one. The spacing is to you know, So there's a There's a space between each of these thes pixels. Now, here's why. I don't like 22 why I do like 21 is When you take build 21 you reduce everything down to 11 there is similarity, right? If you were to close his build 22 to run build 21 again, you would see that there are several objects on screen that that subscribed to sub item zero, which is our base or self item three, which is our snare. And you would see them moving in tandem and that moving in tandem for me creates this pattern. It creates this aesthetic so that when I'm looking at the sketch, I can go. Okay, I can see that there are some things on screen that are working in coordination. Um, some of you may love this bill 22 that's fine. Um, build 22 is giving you every single little nuance of sound that's happening across this spectrum. But what this is missing for a job Fela Davis is that coordination. That similarity on this is why I say reducing your audio range down to a smaller number means that you're going to see visual assets on screen that are working together. You can say OK, those two things are definitely doing the same thing because they're moving at the same rate. If you open up that my audio arranged to 256 that is less likely to occur. Um, so I'm gonna close this, um, bad with a question mark. Um, I am striving for that pattern I am striving for. Ah, there's some. There's some connections that are happening here because I see objects on screen that are moving in tandem. And so this is why I've been super interested in taking that 256 and reducing it down to a number that still gives us the essence of the audio. But then we start to make these connections. We start to say, I think there's these things on screen. They're all kind of responding to the snare because, in fact, they are, um, on Bill 22 Bad bill 22 bad. That's less likely to occur. I should also note that in this sketch my audio data widget also updated. I took out the color coding so you might notice that the widget did in fact, shrink a little bit the code that was located in that particular function because I just said, you know, at this point, I don't need to see the base, and I don't need to see a snare. In fact, that zero on three year probably pretty unreliable because the snare actually is, is in another position. If you're looking at all 256 I almost forgot. Look, it's me again, but this time I'm writing my chopper. Benjamin Fox Illustration of drastic. Look at this. Look at me. That's me. That's me on my chopper. I love it! Benjamin Fox were Prince so great? Let's go ahead and close down, Build 22 see where we press and build 23. 25. Build 23: we are in build 23. And, um, this one I'm just some messing around with, uh, go ahead and open up, build PTE. And let's just run through a few of the things that I've changed here. Map. We love Mac. Um, and so we're gonna add another one. But on this one, I added this little thing here online 29 where I said auto clear. False. So it's gonna leave a residue of the assets on screen. And, uh, so auto clear, false. Basically says, you know, don't wipe the background with the background color, auto clear. True. That's by default. So if it said auto clear, true or if I just didn't have this at all, then it would always wipe with the background color. Um, okay, I think the pool is pretty much still the same. And down here inside of the draw, I just had another map. This is what I love about Map is again. You're gonna sign one set of numbers two and completely different set of numbers. So I made a new interview variable here online 74. I said fft size right, because we're gonna scale the size of our images. Um, let's not forget that fft feel color Still gonna be doing that black gray scale toe white. But now we're gonna be introducing a, um, scale size. Right. So I said here, fft size Hey, guess what. You know my audio data sub itemize coming in. That's a number between zero and 100 but let's map it to 0 300 So again, it's still taking that min and max 0 100 mapping it to an entirely new, um, new property so online 76 year I just say d dot fill and I go ahead and put in the FFT fill color right, al, for transparency to tweet and then size. I just say fft uh, 50 size. So let's go ahead and run this sketch for a little bit and you can see that we'll just repeat painting the background. And if it subscribes to a specific sub item and that serve item happens to go zero, then it's sizes zero. And if it pushes 200 then it's changing its size to 300. So again, just really trying to hit home here about, um, why map is is great and why We want a number between zero and 100 because we can use math and any number of configurations. It could even be a negative. Number two a positive number. Right? Because you're just mapping one set of numbers to another set of numbers. It doesn't matter what that with ranges on, that so cruel. Uh, let's go ahead and shut this down. This waas build 23. Um, we're going to close this, and then we're gonna move into build 24 where we introduce color coding. See him next bill. 26. Build 24: just living life come abuse that that is the death back the colors blood just so massive that you like with way. Just, uh I'm Listen, I can't help you. You you paid for this class. You're You're in deep. Um, color. Let's do some color coding. Eso I am in build 24 color. We're gonna open up a bill PTE. And what I've done is I've said Well, okay, I know that my audio range has been reduced down to 11. Someone to pick 11 colors. So what I've done is is online 26 year. I've actually created an array of Texas decimal colors. Okay. And more importantly, is I've put some some very specific colors, right? So you'll notice that for the base. Right, cause this this is gonna be sub item zero. So if a piece of artwork pics of item zero i mo that the base is going to get an orange color. And here is survived one here, so vital to hear Sub item three. I know that sub Item three is our snare color. So I went ahead and attack that a very specific blue. So the nice thing is that I kind of picked us range of colors, but, um, I'm assigning orange to the base and blue to the snare. Now, in addition to that, I've made a couple of other changes. You might notice that auto clear is back to eyes. Back to true eso if you wanted to mess around with toggle ing auto. True auto clear to true or false. You know, you just leave it there. Just know that auto clear true is always by default, Which means that, you know, if you knew that you were gonna use it again, you could actually delete it. An auto clear is actually equal to true by default. Okay, uh, but I'll keep it on there, just in case you wanna You wanna mess around, toggle with it? Other than that, the set up, um, almost remains the same, except for the setting of the color. So you'll notice that if you remember from before online 48 that's where we're picking that random number where were randomly picking a number between zero and 10. So online 53 I have said, Well, okay, let me go up to the palate and get my associate id ran Index sub item, right? So it's that line 53 that's going up to line 26 going to zero or 10 to actually get its color. Right? So Line 53 is where that color gets set. Now, I also made another adjustment and, um, this one's fun. Since we're not doing that fft full color anymore because of actually setting the fill color once, the only thing that I would be keeping in here is the size. Right? So if you scroll down to the draw, I've actually done something fun, which is I've said, Well, what if I wanted to attach a specific size to the basin snare but another size took all the rest of the, uh, numbers? Right. So what I do is online 76 I say. Okay, well, let me create an instance of fft size. It's an integer, and then I just put a little little conditional little if else. And I said, If I is equal to zero or I is equal to three than Fft size again is our audio data. It's a number of the men, is zero. The max is 100 and I'm going to change that size from 0 to 300. Right? So the base in the snare are going to get a larger scale than the else which is the rest of the assets. So I say Okay, well, if it's not zero or three, then it must be Thea other guys. And so I say fft size pretty much the same exact thing. So this fft size that it put appear online 76 gets set with a number between believe it or not zero and 100. Right? So map is also nice because I can use it. Uh, great. I'm you know, painting 100 objects on screen have reduced everything down to an 11 down to 11. But I can actually use map to say Ah, well, certain things should be larger than others and others can maybe settle down a little bit more. So just a nice little trick for again using map, but for the same variable fft size. So online 79. I took off the d dot fill because again, we're not using that anymore. And we just have d dot size fft size um, rockin Let's run this. Okay. So again the bases orange and the snare is blue, but again, more specifically on and again. These are tricks that I love doing, which is I am forcing a pattern on you. And that pattern is is that red and blue are more dominant, right? But it's it's in such a subtle way. So you know that bases orange, that snare is blue, and you're getting that as a number between zero and 300 and then all the other rest assets that are on there are just used as texture in the piece. But yes, they do not. Klima's high as the base and stare does. So this is what I'm talking about, kind of, you know, in some earlier sketches of creating these kind of design pattern so you could look at this and go all. There has been a very subtle emphasis placed on these two particular things that happened in the spectrum. And despite there being other assets and other colors, there's no doubt that when you sit and watch this, I'm force feeding you that orange and enforce meeting you that that blue you just feel it. Um, so I'm gonna go in closest. I like this idea of, uh, of finding these these ways of of emphasizing a very subtle design pattern. Okay, Uh, this should pretty much shut down. Build 24. We're gonna move on to the next to which is built 25 26 which is maybe I don't want to use a rectangle or ellipse. Maybe I want to paint with a transparent PNG. Or maybe I want to paint with SPG that I make an illustrator. So let's go ahead and close this. Build 24 and I will see you and build 25. 27. Build 25: we are in build 25 this is PNG, and Oh, my God, I love this file. And, um, because it has a trick. And this is my new trick. And I love this new trick, and I am going to share this trick with you. And after I am done sharing this trick with you, you're going to shut the hell up. You are not gonna tell anybody this trick. We have to have secrets. Way must have secrets. Um, I really love this really, really love this. And what we're gonna do is we're gonna make a PNG. So you will notice inside of the data folder there is blocked dot ai and blocked up PNG. And what I want to do is open up this block dot ai and, uh, look at what I've done. Um, if I go into documents set up and I look at the art board, it's a width of 3 50 It's a height of 3 50 and I just made a vector, you know, uh, uh, square and rounded the edges and turned it. And, you know, however, what I did here is I put an outer glow, right, so I came up to effect, I went to style eyes and I said, Outer glow on my vector assets. So it creates this kind of, like, nice drop, shadowy thing, but all the way around, right? It's not drop show. It's blue. Now you will notice that the outer glow ends. Um, just come in here. This is white. You know, this is all white here, And, uh so I'm making my assets so that they're floating in this in this PNG And if I were actually to save this out again here, you would see a little checkerboard pattern, and that's because I'm going to save this out as a transparent PNG. Now, I'm just gonna close this really quick because there's one mawr important thing that we have to talk about, which is is my trick. My trick is, all of your PNG assets are white. Um, uh, I am going to make sure that all of my PNG assets are white and you're going to see why here in a second. So let's pretend I save this out. I saved this out is my P and G. So now I've got a PNG inside of the data folder. This is all transparent around on the outside. I've got this nice Grady int kind of outer glow that also has transparency and my fil color of my asset is white. All right, so let's go to, um, are build PDE here and again that most of this is the same. However, um, notice I still have the, you know, the palate Color auto clear. Yeah. I actually talked with that guy back to false again. OK, so we're not going to clear the background. Now, let's go ahead and look at line 44 online 44 instead of new H wrecked as previous, we're just saying new H image and then in passing in the string of the name of my PNG, which, of course, inside of the data folders called block dot PNG Now, uh, I'm now going to be painting with a transparent PNG, and, uh, the other thing that's nice is is if you come here, tow line 53. It says fill color palette, Randon. Next rent ran index, and you might be saying yourself, Wait a second. How are you setting a fill color when it's a, uh when it's an image what ends up that in hype. If you load in an H image and that H image is white, okay. And I'll talk about what needs to be white when you set a fill color, you're actually applying a tent. So believe it or not, our base is gonna be orange in our snare is actually going to be blue. So before we maybe continue this conversation a little bit further, let's go ahead and scroll down and realize nothing has changed. The only thing that I've changed is I've introduced this new H image. So let me go ahead and run this sketch and let's see what we get. So, yes, look at that. It ends up that if you use a white PNG and set a fill color in hype, I'm assuming that you want to do a tent. So I actually apply a tent color on top of your white PNG. And since it's white, it actually absorbs all of the color. Right. So this is sort of interesting to speak about, because if I had made my assets black and PNG putting color on black is, he would never would never see the color. So as long as my assets are white. Um, I actually connect set of Phil, and I will actually get that same fill color. But look at this enhancement. I'm getting that drop shadow, that sort of outer blow around my assets. So there's been some times that I've posted on social media and people have said, Well, hey, how are you getting that outer glow on a nature rectangle? Um, how are you doing that in processing? And the trick is, is that I'm using a transparent PNG. So if I want a shadow, I can actually apply the shadow and illustrator on as long as my artwork is right. Then I know that it's going to take all the color in. Yeah, so a really great trick. And again, uh, let's say that you made a PNG and you actually had color rather than white. Then it would fill that color as a tent on top of your color so it can get a little funky. So that's why again, a lot of my assets that have been paging with usually are just bright white, because then I know that when you said that, Phil, it's gonna apply the color just as it is specified up in the your age, right? So you can see I'm getting the orange base color and I'm getting the blue snare. But I get that added feature of having that outer glow. So in a lot of ways, if I would want to keep the serve overall aesthetic of the composition flat, then find eight rectangle is the way to get Oh, just use an eight tracked and everything is fine. But if you wanted to add something that maybe had noise or maybe had a drop shadow, or maybe had an inner glow or an outer glow, you can actually make those assets an illustrator. Apply those filter effects, save them out as a transparent PNG. And now you're painting with a transparent PNG asset, and it's just gonna give you some added features that a direct obviously can't do. And I'm sure you noticed we get the speed to. I've tried to do some some filters on top of draw balls inside of processing, and sometimes that can kill frame rate. So this transparent PNG trick is awesome. It's it's it's one of my favorites. Okay, this is gonna close this down because we're gonna move on to build 26 which is working with SPG assets. Also inside of illustrator eso. Let's paint with SPG rather than a transparent PNG See in the next the next build build 26 . 28. Build 26: great. So we are now in build 26 we're going to be using SPG. So, uh, if you actually, uh, look at the vector files, let's go ahead and go into data and you'll notice that there's a, uh uh hex one dot ai x two dot ai hex three dot ai. And if you were to open these up, you would see that there just some hexagons with different patterns on them. Now, if I were to open this up in Illustrator um great. Um, I might want to use SPG because again Aiken go the same round just like a PNG. Except for I can ADM or details, I can add more lines. I can add more, more pattern to it. So using a direct was great using P and G was great, but yes, you know, at the end of the day, I might want to add some more some more texture to this thing. So obviously SPG is a non option. Now, when you make these vector assets inside of illustrator, you're just simply coming here and doing a save as and you would specify that it is an SPG file. So if we look inside of this data folder. I've already saved them out. Now the nice thing, if you remember from before SPG is just text file, you can actually opened it up inside of sublime text to and you can actually see, um, parameters that have been set to all of these vectors inside of this SPG. So SPG is actually, um, extremely easy to check out inside of a text editor. Now, let's go ahead and open up our build PTE. And as you might have guessed, not much is different. You will see down here online 43 44 through 46 I'm adding are three spg assets. So, um, I go ahead and say that it's a new eight shape. I pass in a string, the name of the SPG that I want to load in. And then I put some initial settings like enable style false, you say enable style. True. Then it's going to stay that black and white as it is an illustrator. By putting this enable style false, it means that I can start to set my own stroke fills size. I can set my own Phil colors so on and so forth um and this is just something I've got into the habit of doing, which is always specifying that the stroke join and stroke cap our center. And that's because if I'm going out, put this back to illustrator. Um uh, I typically want my joint and caps to be to be centered A They tend to clean up the vectors kind of the way that that I usually set them when I'm just working illustrator Straight off . Anyway, So, uh, when this pool is gonna run 100 times, it's now randomly grabbing one of these three S V G files. And again, I can come here and say that the stroke is going to be black and the Phil is going to go up to the palate and randomly pick one of our colors. Um, Now, you will, um, see here that pretty much everything else is is unchanged. So I'm gonna go ahead and run this and we should be now painting with these SPG assets. Okay, So, so cool. We kind of these past three files of really encapsulated, um these different things that weaken do withdraw bubbles. We can use hype, draw boils like a direct h lips So on and so forth Waken use PNG as just kind of like a little bit of a further trick of being able to add some drop shadows or some noise or radiance or whatnot. And then this Final one, Yes, we're painting with SPG assets. So if you did actually take, um, my 1st 2 classes, obviously I provided Cem spg files in that class. And sure enough, you can swap them out, modify the code, and all of this will be, um, adhering to that random number that it picks. A random number that it picks between zero and 10 is related to theme the audio spectrum. And again, I'm doing this thing, this color coding. So you know, the oranges base and the blue is the snare. So cool, uh, great to take a little departure with Bill 25 26 27 just talk about the different types of assets that we can use. Now, let's just dive a little bit deeper and I promise you now, from build 27 the code is going to start to get more complex. We're gonna introduce things like grids and then three d grids and then three d draw bulls and lighting systems, and it's gonna get fun. So let's go ahead and shut this bill down and I'll see you and build 27 where we're going to introduce a grid see in the next. 29. Build 27: Now on this file, build 27 grid, you might notice that I've marked it red. And the reason why I have marked at Red is because I want to introduce a new concept, but detach it from the audio. So on, you'll see. I'm gonna do this a few more times where I want to say, Well, OK, let's let's think about this thing. But let's just attach it from the audio for a moment. So in this one, I'm just gonna add a grid layout. And again, if you've taken my previous classes, this was all the way back in the first class where you could use, um, on HK rid layout, toe layout, things on screen. So this should be a relatively short video. So let's go ahead and go into Build PTE. And most of this, um, should look familiar. Not a lot of changed. You will notice that, um, I reintroduce r h wrecked again. I've brought that back. Right? So we had the discussion about PNG. We had a discussion about SPG, but in the case of this, I'm actually gonna go back to that H rectangle. No, um, if I were to scroll up to the top here. You would see that I've introduced two new variables for columns and rows. So I said, I want seven columns and I want seven rows of assets. Right? So I said Thies to introduce And that means that I am no longer going to paint 100 things on screen. Um, so on this line 43 I'm saying, Okay, I want to draw columns times row, right. And in addition to that online 46 I say, Well, OK, I want to do an H grid layout. And so, um, I say that part of the pool is a layout. It's a new H grid layout. I wanted to start at an X axis of 110 pixels. I also wanted to start at a Y axis of 110 pixels. And I want the spacing of the artwork to be 80 on the X and 80 on the why. And of course, I need to specify the columns. Eso I just go ahead and pass that that that pool calls inter juror that I specified appear online 23 Now, um, in the on create again most of this has remained the same. I have no stroke, I said. Phil, I put the anchor and center. I rotated 45 degrees. And of course we've got our extras bundle that is randomly grabbing. One of our index is now. I am going to set the color so we should be able to see orange for base and blue for snare . But if you come down to draw here, you'll notice that I've commented out the four loop because I'm saying, Let's just attach it from the, um from the audio. So, um, this is some stuff that I'm probably gonna use in the next file, but right now, there's nothing inside of the draw that actually attach is this new grid layout of H rectangles to the audio. So, case in point, if I was to run the sketch great, it does exactly as anticipated, which I've got seven columns by seven rows, and I've got this H grid layout that's doing the spacing of 80 so that they overlap a little bit. But that overlapping is done in a way where I get these nice little, um, close between the between the assets and again in the draw function. We are not doing anything that is tying it to the audio. So I am gonna go ahead and close this sketch, and we're actually gonna shut down. Built 27. I said this one was going to be short. Um and I think I'm going to continue doing this, which is any time we sort of introduced something new. I'm just gonna detach it from the audio and say, OK, let's just focus on the thing that we're trying to do visual on screen, then will tie it back into the audio. So this is gonna end this bill 27. We've built our grid. Let's go ahead and jump in to build 28 where we now attach our audio data to, uh, this grid. So see you in build 28 30. Build 28: So this is built 28 now we've got our Gridley out happening in our sketch. So let's check literally. Just implement a little tiny block of code to tie it to the audio spectrum. This probably will be the shortest video I record. Um, everything pretty much remains the same from the previous build. The things that I will point out is line 55 or setting that fill color that Phil colors relating to that color ray up at the top orange for base blue for snare totally 1000%. Got that? So it's, uh, this is the only change in this file. Um is lined 73 through 79. I'm saying, Great! Now let's scrub through the pool. Let's look at that H bundle. Let's look at that extra. Let's get what random number picked between zero and 10 in this time. I'm doing a new map of FFT Alfa, right, and I'm saying, Let's map the audio data, which is a number between zero and 100 let's now map it to a number between zero and 2 55 so 2 55 being not transparent at all, and zero being completely hidden from screen. So then online 78 I'm saying d dot alfa fft alfa. So let's go ahead and run this sketch and the grid is there. The artwork is there. The comer is there. But were you actually using the audio spectrum to bring up the Alfa transparency so that we get a hint of of this this, uh, slightly hidden grid, right? And then once this audio really kicks in, we'll watch this grid obviously get a lot brighter when all of those thresholds come up past, um, 100. So cool. This is probably like the shortest video that I'm gonna create. Um eso This one is This one's great. We have this nice grid layout that we're making we color coded based on the audio spectrum , and we're using Alfa to show and hide the assets. Uhm, but I have to tell you, you know, I've done two D for a really long time, and I have I have gotten the golden fishhook of three D, and I just love doing things now in three dimensional space. So, um, you might notice that build 29 is now going to introduce a three D environment. So let's go ahead and shut down. Build 28. We're gonna move in to build 29 we're going to see if we can attach that audio data to a three dimensional environment. So I will see you and build 29. 31. Build 29: alrighty. We are in build 29 3 D. And there's, uh, no question for me that this is where it starts to get fun. I really like doing these sort of animations in three dimensional space. In fact, we're gonna cover um three D pretty much from 29 all the way to 42. All right, but we'll start off, um, pretty mellow here. So let's look at build 29. Let's go ahead and open up build PTE. E and let me just open this up. So there's a couple of initial things that we need to modify, and that's that you need to tell processing that it's going to be a three dimensional environment. And then, in addition to that, you need to tell hype that it's going to be a three D environment. So that way it switches from two dimensional coordinates to three dimensional coordinates. So you'll notice here on line 32 we say p three d, so the campus is gonna be 700 by 700. But we're turning this processing sketch into a processing three D sketch. And then, just as I mentioned before online 33 you need to tell hype that it's to start using three D coordinates, right? So things like P vector, um, will actually use that that z coordinate. Right? So we want a toggle from a two dimensional space to a three dimensional space. And now that we have access to X, Y and Z Um, OK, so we've added p three d to size, and we've added use three d true, um, toe hype. Now I'm gonna be taken pretty much the same exact sketch that we didn't build 28 which is, you know, building the grid. But now I'm gonna do is I'm gonna kick that grid back into three D space. Right? So, um, the Z axis was always zero when you're in two dimensional space. So now that we're in three dimensional space, I'm actually gonna take that entire grid and just move it back into space. And I'm going to do that by, uh, adding a Z coordinate. So you'll see here online 58 I've added a Z coordinate for every single one of the cells in the grid to kick back to, uh, negative 600. Okay, so now when all of those draw balls get drawn in that grid. It's just gonna move back to negative 600 on the Z axis. Now, the other addition is inside of the draw function. So in build 28 inside of the draw function, we were changing Alfa. So we were way were changing the transparency of the items in the grid. I'm gonna turn that off on this one. And what I want to do is actually change the Z depth of the objects in the grid. So you'll notice on line 78 again. I got rid of the FFT Alfa stuff and here I'm creating an interviewer called FTZ again. It's still the my audio data array. It's still a number between zero and 100. But now I'm saying, um, if the number is at zero, it should rest back at that negative 600 in Z space, which is right here. And if it reaches ah 100 that it should push negative 500 negative. 400 negative. 300 negative. 200 0 up to positive 100. So now it's actually coming closer to us than it did in build 28 right? So, uh, Line 78 is just moving that asset from negative 600 toe positive. 100 on the Z axis. So let's go ahead and run this puppy. We'll just set the softer side and again, you can see that everything is resting back in that negative 600. And then as as the audio starts to vibrate, it's actually pushing the Z access of that asset forward. Um okay, Uh, cool. So we're moving this two dimensional flat object. It's very similar to having, you know, like, a flat sheet of paper flat card. And we're actually moving that flat card forward and backward in three dimensional space. But if for some reason we were to turn the card, we would actually see that this is actually flat, as is the case with system. These things that we're moving or actually two dimensional draw balls right and a direct is a two dimensional drawing ball. And so we're moving this flat two dimensional shape in three dimensional space. So let me close this, and you might realize that, um, let's move on to the next couple of sketches where we start to actually use three d drop. So, um, we're gonna dio h box which allows you to define three dimensional box. Now when that thing moves in three dimensional space, you would actually see the sides and the top in the bottom of the box because you're actually moving a three dimensional object in a three dimensional space. So there's build 29 is kind of a nice ramping up of moving into three D, But now let's swap out our two dimensional drawing Bols for three dimensional troubles. All right, this closes built 29. I will see you in build 30. 32. Build 30: All right, we are in build 30 and as mentioned and build 29 We're gonna convert the to t draw bulls to three D draw balls. And then also, we're gonna address, um, a couple of other things. So let's get this party stored. Food bill 30. We're gonna open up build, um PTE. And let's look at a couple of things here. Let me just open this up show visualize, er I have turned back onto True and I've turned show visualize er back on the truth so that we can see why I wrote certain things down in the visualize her function down at the bottom . Um, great. So we are going to convert to three D draw balls. So that way, we're seeing three D objects in a three dimensional space. So great show. Visualize her true will address that here in a second. And let's come to the pool and in the pool. I've changed from h repped to H box. Now H boxes a three dimensional box, right? And, uh, when I add h box, I have to define, um, dimensions of the box itself. Right. So down here on line 52 I say, OK, H box D. I want to now set the dimensions of the box so the depth is gonna be 500 pixels. Right? So that box is gonna have a depth of 500. It's hype is gonna be 50 and it's with is going to be 50. So, essentially, we're creating like a French fry, right? So 50 by 50 by 500 right? Um, everything else pretty much is staying the same on here. If you look at the draw, I am still keeping this f ftz. Except for, um, I am setting this back to negative 900 on the sea. And actually, I'm gonna come up here and just say dot z negative 900 so that that's their That's their starting spot for each of these boxes, so dot c negative 900. And then our FF TZ down here again is taking that audio data is gonna go from negative 900 toe positive. 100. Okay. And then, of course, we're saying dot c go ahead and and map this ftz Now, um, I want to show you this my audio widget, which right? Um, it says no lines. It says hint disabled depth tasks and hand to enable depth test. And those air commented out. So since this is a three D environment, I actually did one of the thing that I just sort of skimmed over, which is lined 73. I said, Lights. So lights is just sort of ah, very soft, ambient white light that covers the entire scene. So if you didn't call Lights online 73 what happened is that you have these three d draw bulls, but there'd be no highlights or shadows for you to actually see that there were three dimensional. It's kind of like flat cell shading, right? So by calling lights online 73 I'm gonna be able to see some highlights. Some shadows on all of the different surfaces of H box Now coming down here. The reason why I imagine eyes because you know no lights is is, um, commented out Eso If no lights has commented out, you'll see here that the fill color for our little visualize er is almost white cc, cc, cc. So let's go ahead and run this sketch and cool. The first thing that I want to look at is is that there's no way that that right there in the visual izer is cc, cc, cc. And that's because what's happening is that lights is actually casting three dimensional, like even on this draw ball, right, even though it's a two dimensional drawing. The other thing, too, is, is that because I don't se him disabled Depth test and hint enable depth test. This is what I was talking about. Look how draw balls actually will cut past and threw it over. They will exceed the plane of our little visual Isar, which I sort of wanted to eliminate. So if you look here, I'm going to say that the my audio data, which it should have no lights so it's gonna turn the lighting system off. It's going to disable the depth test. Then it's gonna draw, and then we'll really enable depth test again. Great. So let's go ahead and rerun this sketch and you'll notice that that modification has been accounted for. One. We get back to Sisi Sisi Sisi because no lighting is being applied onto these two dimensional Rex. And also the hint Disabled depth test always means that our little widget bar is always gonna be the highest depth. And those three dimensional draw balls actually won't break the plane. So it's sort of like permanently putting something always above, even if you're doing three D transitions. So, um cool. This looks great. And again, in fact, you know, just just for the heck of it, let me just show you here what happens when I'm talking about when you turn off the lighting system? So if online 73 I turn lights off and then I go ahead and run this sketch again, you're going tohave. Ah, hard time. Really noticing all of the shading and details on this three d, you know? So if you look at the front, you look at the bottom and you look at the side, you'll notice that, yes, there is Alfa transparency. So it is giving you some semblance of that. It's a three d draw ble. Um, if in fact, if I was to even go in here and changes right, so here online 58. See, here's the fill. Here's what I'm saying that the alphas to 25 watches. Let's go ahead and put this to 2 55 Let's go ahead and run the sketch again. And now it's almost even harder to tell that we're actually using three D dark probables. You said Okay, well, you know, Yes, it looks three d, but you don't have any of that shading. You don't have any of that, any of that definition. So to fix that, you know, I'm gonna put it back to where it was, which is, you know, by putting in that Alfa transparency of to 25 means that it's a little bit opaque, and we can kind of see through to some of the other draw balls by putting this lights on, right, we really get an opportunity to see the light being cast across these three dimensional surfaces. Right? So the front indeed looks very different than the bottom in the side. And you really get a sense that, um, that these are actually three draw balls moving in a three dimensional space. So I'm pretty happy about this. This looks this is starting to look really, really fun right now. This is gonna end, build 30 and build 31. We're gonna make a modification of this. So, um, 31 will take out the color coding, and we'll put in some. It's a different colored lights rather than just on ambient light. So this is gonna go ahead and end, Build 30 and I will see you all in build 31. 33. Build 31: already we are in build 31 build 31 is a slight modification of build 30 eso. If you open up the PDE, you might notice up at the top. I've removed any reference to the color array, so we're building this grid. We're putting three d draw those in that grid, but I actually don't want a color code the objects based on the random number that they pick. So I've actually I've actually removed the color coding. If you come down to the pool, you'll see that I've changed. Line 55 I've said. I just want the fill color to be white, so the fill colors to 55 with an Alfa transparency of to 25. Everything else pretty much remains the same until you get down into the draw function. Now you'll see here online 70 through 72. I have point light. Skip over that for just one second and come down tow Line 80 and show that we have reintroduced FFT fill color, so I want the audio to effect the draw bills fill color, so if it is at zero, it's going to be black. And if it's pushes up to 100. It's gonna run through all the grey scales, and if it gets to 100 it's going to be white. Now, the reason why I've reintroduced fill color is because of point lights. Appoint light is gonna let us set up these points in space that radiate light. And if I had to objects and one was black and one was white and I set up these these points of color, um, obviously black will always remain black. It'll absorb any color that you set in the environment. If that draw bulls white, then that white will reflect the color that's becoming that color. Yeah, So I want this This transition of black to gray scale, toe white toe actually come back because that will affect the color that we're seeing in the sketch. So as it as the audio reacts and it comes towards the front, the color saturation will actually get brighter. And as it moves back down to zero, the color saturation will subdue. F FTZ is still the same. Negative 900 to positive 100. Line 83 has been updated so that it's that d dot fill and we're setting that fill color with an Alfa transparency of to 25 and the easy access to FTZ. Great. Now let's go up to lines 70 through 72 I just want to talk about teal and orange first. So point light is a point in space that radiates color in all directions. Yeah, so, um, any time you get point light is kind of my go to lighting system, it's It's what I I prefer over ambient directional point Light is great. It's just what I prefer. Point Light accepts six arguments. Rgb x Y z So you can see for this first point light, I set a red of zero, a green of 1 49 and a blue of 1 68 for the orange. I set a red of 2 55 agreeing to 51 a blue of zero. So I'm setting up my RGB values, which, of course, you can get from photo shop or illustrator wherever. Um, now let's talk about the next three argument, which is X, Y and Z. So, um, if this were my canvas here and let's just pretend for a moment that this marker is is my my point light. What I'm saying is okay, I want to go with divided by two. And I want the why to be zero. And I want the Z to be negative 100 so just drops back in space. So what we're doing is we're creating this point in space in which light is going to radiate. And again that light is radiating based on, uh, this teal color. Yeah. Now, the second light that we set up is this orange and I say again with divided by two. But I say that the Y axis is height, so it's gonna be on the bottom and again doing this negative 100 is just gonna set it back in space. So now we've got a point down here that is radiated point of light orange, And then we got a point up here that's radiating teal. So that's nice, because we've got these, You know, we've got these three d draw, bols and so sense that that point of light for Thiel's up on the top and it's radiating light from from from everywhere. Only the top of the draw balls are actually going to get hit with that Teal light. And then, uh, the other is is Obviously the orange is actually gonna reflecting off of the bottom of the assets. So, um, point lights great. Um, set up these two points to you bouncing off the top orange bouncing off the bottom. Now, this third point light is just kind of a 2nd 1 that I always throw in there, which is if this is our sketch, I've just said the RGB is 50 50 50 So it's it's it's a dark gray. And, um, I've said that its position is with divided by two height divided by two. So it's in the center of our sketch and then I put it Z axis at 200. So it's a point of light that's sitting basically in front of everything and radiating light out from that position. OK, so let's go ahead and run this sketch to see all of this in action. Okay, The first thing that you have to remember is is that we're doing that fft fill color, so that's gonna control the saturation of light that actually hits our, um, our three d draw bulls. And because we've got that t light up at the top. You can actually see right here that the teal is hitting the tops of those surfaces. And because we have an orange light point like down here, it's reflecting the bottom of the surfaces. But we also have that gray. That's kind of sitting in the front so that when those draw balls actually moved back in space, the front of them are sort of getting a nice soft settled gray on top of them. But again, as they sort of move forward and they're shifting away from these point lights, you'll see that you know the front faces actually go to Black at one point when that front face would actually move in front of that great life. Because that great light is is that 200. But our draw bull is 500. So as it moves forward, that very front face of the Wrobel would in fact, move past all three of our point line. So that's why there's this moment when the troubles come, uh, totally to that front, that front surface actually becomes completely black, becomes black because it's no longer in the range of getting any of the any of the light as they back in space. You can see they'll start to get that great back. Great. So I'm gonna go. Includes this. This is a beautiful sketch are really, like the texture that this is giving off. However, we've been working with a two dimensional grid. Right. So we've made this three D space. We've got three d draw bulls, but we're actually aligning them onto a two dimensional grid. So we're gonna close down, build 31. We're gonna move on to build 32 where we actually build a three dimensional grid that holds three dimensional draw balls. And I imagine that that's going to look amazing. Okay, See you in the next bill. 34. Build 32: So here we are and build 32 on its three d grid. And you might also notice that there's a little red warning light here. Um, that this particular sketch is going toe have the audio component removed from the sketch that we could just focus on on what we need to get done. Um OK, so, uh, up it to this point on the previous bills, we were actually using a two dimensional grid in a three dimensional space with three dimensional drop balls. Um, hype does support the ability to do a three d grid. So, um, the first thing is, is that you might notice we re introduced the the color array, so the color array is actually back in the sketch. I do want to do the color coding again. Base is gonna be orange, and snare is going to be blue. Okay. Lines 23 24 25. I've now added three new integer. So there's poor columns. Five pool rose five and pool depth five. So now were specifying how many columns we want, How many rows we want and we how maney rose we want in depth. So we're doing a five by five by five. Uh uh. Grid. All right. Moving down to the pool, you might notice that I did update the pool so that you're doing columns, times, row times depth and I also switched out the draw ball. We're not using H box anymore. We're going to be using H sphere now online 47. Um, I'm showing you these adjustments to, um to the H grid layout. Notice that now we need to specify all of these coordinates and these jumps. So I've said that I want to start X the start. Why? And the start z all to be at negative 300. And I want the spacing to be 1 51 51 50 on the X, the wine, the sea. Now why? Why have we set up the H grid layout this way? Well, I want to make this cube in the center of the screen, and I want to be able to actually spend that cube. I want to rotate it. So what I'm trying to do is basically say you know, what is the with divided by two, the height divided by to this point in the very center of the sketch. And more important than that is when I actually make this grid, I want the very centre point in the grid to be at 000 Right. So, uh, what I'm doing is I'm saying, Okay, um started negative 300 but then do a 150 jumps. So it goes negative. 300 negative. 1 50 0 Positive. 1 50 Positive. 300. And it's doing that on the x axis, the Y axis and Z axis. So, essentially, we're building this grid from the back. Coming forward, OK? And now let's just go ahead and look at our sphere. So online 53 we say x fear Now for sphere, it's not like h box, where you have to specify depth, width and height. You just simply said a size. So online 55 said that size of this H sphere is 10. I've also just a little note, like any time I'm working with, um spheres in particular. And if for some reason like later on down the road, you end up getting into using meshes. I've encountered these problems where, like if I'm scaling or oscillating a mesh and the stroke would is not sent to zero. You'll get these errors in the output window saying that it basically can't draw the stroke because it's it's to to doing big. So I kind of got this habit that any time I'm working with three D draw balls and I'm not wanting to visualize the stroke, I have gotten into this habit of setting stroke weight equal to zero. Now, online 58 we put back our Phil. Hey, go to the palate. Remember that random number that we picked Look that up into our color array And a sign the Phil that Phil Color with an Alfa Transparency of 225? Um, great. Let's come on down here to the draw. Notice that that four loop or I'm cycling through the pool. I've commented that out. I don't want to do any audio related stuff to our grid just yet. I really just want to concentrate on the grid. Will add audio in the next sketch, so you'll notice I've got lights. I got rid of our point light. We're just doing a nice, soft, ambient light. Um, and I've got a Line 74 here which says sphere detail 20 sphere detail allows you to specify basically the detail of the triangles that make up that spherical mesh. So the lower that number, the larger the triangles and the more polygon all it's gonna look, the larger that number, the smaller the triangles are gonna be. And obviously, the more it's gonna look like a perfect circle. Okay, so, um, I will sometimes tweak sphere detail based on the amount of assets I'm drawing from drawing a lot of sphere assets. I might want to put the sphere detail down to, like, 10 so that it can actually paint faster so that it's painting less polygons online 76 through 80. I do a little push matrix pot matrix, and inside of that push make drinks. Pot matrix is where I put the h dot draw stage. So whatever happens between that push and pop is gonna get applied to our our hype troubles . Now, what I'm doing is online 77 them doing just like I said here, which is with divided by two height divided by two. I'm setting the the point of creation to the center of the screen. Okay, so translate with divided by two height divided by two. And then I'm pushing back into that negative 900 on the Z axis Line 78. I want to see just a rotation of that box. So I'm saying rotate. Why radiance? And then I just pass it frame count. Okay, so let's go ahead and run this sketch. You should be able to hear the audio. But of course, it's been disabled from changing anything in this sketch. Um, we're getting this nice rotation of this three dimensional grid and you can see that it's a five by five by five grid and we're back to doing that color coding. So all the little blue dots are actually the snare, and all the little orange dots are base. So I'm gonna go ahead and close this sketch down because now we've implemented a three dimensional grid in a three dimensional environment using a three dimensional drawing so that this is looking good. So let's go ahead and close this down. Um, I'll see you in build 33 where we now are going to attach, um, some oscillation to our little each spheres. See in the next match 35. Build 33: build 33. Um, not much of Ah, huge jump from build 32. But we're just gonna add or audio, um, spectrum back in. So let's go ahead and look at build PT. You're going to find that it's it's extremely the same as build 32. Um, this set up is the same. The everything at the top is the same. Eso we immediately want to go to the to the draw. Now, if you remember, Online 55 is where we set the size of our sphere. We said that it was gonna be a size of 10. So if you come down to the draw here doing the lion's doing this fear detail doing that push matrix pop matrix rotating the cube painting are good, but here it's these line 82 through 88 that we bring back. So we commented this out in line 32. We're bringing this back and then and then adding fft size so you can see on this line 86 I'm saying, Well, OK, let's do something new called fft size where I'm actually gonna expand the size of our H sphere. The my audio data is coming in. It's a number between zero and 100. Well, the menace 10. Because that's our initial size that we set up here online 55. So the men for FFT size is going to be 10 and the maximum is going to be 100. So we're scaling now. Zero was mapped. A 10 100 is mapped to 100 right? So now we've got this new FFT size and in Line 87 were just readjusting that this initial size that we set up online 55. So we're doing d dot sides and then passing in that FFT size number. Okay, If I run this file now, you'll notice that our grid is still spinning. We still have the color coding in place, and, uh, now we can start to see the size of our sphere being affected by, um, our audio spectrum. Right. So don't forget that we oranges base and blue is snare. And, uh, as this file starts to ramp up, really watch this whole kind of three D grid come to life. Thank great. I'm gonna go ahead and close this because, man, I have, like, two more raise to adjust this So both 34 We're gonna introduce a new idea both 35. We're gonna introduce another idea. So these should be relatively short videos. But again, just stepping through that process of how we can use audio to effect animation. Right now, we've got a three D grid. We've got that three d grid rotating, and the audio analysis is changing the skills of these h spheres. So this ends this video. I will now see you in build 34. 36. Build 34: build 34. Um, so build 33 34 35. Pretty interesting, because I'm just adding a little bit more functionality as we progress. So I'm build 33. We've got our h Gridley out. We've got it spinning on. We're attaching the audio spectrum to the size of the of the items in that grid. So I'm build 34. I'm gonna make a few adjustments, okay? So build 34 1 of the first things that I do is I update the color palette. So I've updated it in a way where I'm now assigning base to black and I'm assigning snared a white and everything else I just set that, um, a medium gray 666666 Okay, so I'm still doing color coding from this array, but, um, you know, later if I want to maybe introduced some other other kind of medium graze I can. But the point is, is that I want to put the focus on on the base and on the, uh, and on the snare. Okay. Um, so again, everything in the set up has remained the same. And down in the draw is where We're gonna put some other adjustments. One is Okay. So you've got this. We've got this three dimensional grid, and what I wanted to do was, was just put a very soft box. Kind of around all the points in the grid. So you might notice. Here. Online 81 this is addition. Where I set the stroke. 2333333 I set the filled A 24 24 24 but an Alfa of 50. And then, you know, at this point, I'm not using any processing code. So I'm not using any heights. Misfit code. I'm going to use, um, processing specific code. So I just said box 600 then after I'm done, I just said, no stroke, no feel. So you'll see women run this. It's actually gonna put a very transparent box around our, um, our grid layout. Now, I want to get back into doing that. Kind of like, uh, specifics where I'm saying like, Oh, this is the basis is the snare. Let's attach different behaviors to those things, and I'm doing that with an adjustment to FFT sides. So look right here. When I say online 89. If I Z equals zero, then I know that it's the base. And, um and not only do I know that it's gonna be the base, but I also know that it's going to be a black ex fear. And specifically, what I've done is I've said I want to make the men negative 3 50 and the positive 3 50 And this is kind of fascinating, because any time you take a draw bowl and go negative to positive it, what would happen? It's not zero, right? So if it said 0 to 3 50 you would know that that went from nothing to 350 right? But I said, Negative 3 53 50 So that means that if we're looking at the audio spectrum, uh, base would have to get to 50% threshold in order for it to be zero. If it zero, it actually folds in on itself and is negative 350. So even when the base is quiet, it's still basically a sphere that's 350 pixels. But but inverted right, so the audio would have to get to a range of 50 in order for it to be zero if it went to 100 than its positive. 350. So I'm putting a lot of focus on on base by going that negative. 50 Positive 3 50 range. Okay, um, else if if I is equal to three, then I know that sub item three. I know that it's my snare, and I know that it's going to be white. And so I said that its range was between 50 and 350 right? So, um, so some heavy emphasis on these two particular items that get picked hay for everybody else . You get to be that medium grey, and it's going Teoh scale between two and 1 50 right? So fascinating. Because all the other stuff just ends up kind of being just sort of ah, hint Teoh to the audio. But we're really focusing on on that black base and that white snare. So let's go ahead and run this sketch toe watch. Ah, this in motion. All right, so you see that that the base has to get to that range of 50 and order for it to go away. So it's kind of always it's kind of always present. So the base is kind of in this scenario is gonna be kind of like a very, uh, present object. And again, we're also putting a lot of focus on that. That sub item three, which is that white snare. So it just kind of hangs out at a much larger size. And then when it pops, it pops just as big as a zoo. Black base does on. But again, all the other numbers air just kind of there to create the texture. Yes, we can see that they're reacting to the audio, but we're really bringing their emphasis down. So in this particular case, I'm really just focusing on that black base in that white and that white snare. And then again, look at this, like, nice boasted box, so I can see. Um uh, this box with that Alfa transparency is just kind of holding my little my each grid. Now, this is awesome. But again, let's take it like, you know, 11 step further, and that step further will be is in the next sketch. We've got this this rotation that's just kind of automatically happening. What if who tied to that rotation actually to the audio as well. So in the next build, what we're gonna do is we're gonna keep the same exact animation. Except for the audio will now control how that thing that box actually rotates. All right, so let's go ahead and close this down. That was build 34. And now let's add another layer of interaction by having the audio control the rotation of that entire object in build 35. 37. Build 35: we are in build 35. And this is going to kind of close out this this series of sketches because the next one is build 36 to build 42. That's a new idea, you know, completely flushed out. So this build 35 is gonna is gonna end this this thing that we've been playing with 32 33 34 Now, now 35. So build 35. We wanna have the audio actually control the rotation and rotation. We obviously have three. Access is right. We've got rotate X, rotate wide, rotate Z s. So we've got three Axis sees that we can play with. So if you go ahead and open up this bill PD, most of this business is the same up here on lines 33. 32. I entered do some new integers, so I said, rotate non X rotating, Um Y and rotating, um Z are all equal to zero. So now I've got thes thes three new inter jurors that I'm going to be throwing some numbers at. Okay, uh, nothing changes in the set up. Um, that all pretty much remains the same. So now here in the draw. We're just looking at where we're applying some rotation and you can see that I'm doing that online 83 through 93. Now, in the previous sketch, we just said, you know, rotate radiance, and then the and then the frame rate. So we were getting that that rotate. Why? And we're just spinning it just at all times. Okay, so let's look at this in steps. So rotate x, Y and Z, we're gonna skip over that because that's kind of where we want to get to the end. The first is is that I want to map these new numbers, right? So really, I need to talk about 87 through 89 1st So 87 through 89 is where I'm setting tying it to the FFT number data. So I'm saying, for example, on, um, line 87 ff t rotate ex, uh, and then look at what I'm doing. I'm going to, uh, look at very specific sub items in the array. So down here, like when we were doing fft size, we were saying my eye audio data, and then we were looking at I and I was looking at each assets, you know, random number that they picked. But what? This? We're not doing that with this. We want to look at a very specific point in the spectrum, and so I don't need to use I like I did down here. I'm just saying my audio data and then go to sub item zero sub item zero. We know for a fact is the base and and we know that sub item three is kind of our snare. And then just I picked an arbitrary number. I just figured, Hey, we're doing odds. We might as well go with five. Um, but this obviously could be any arbitrary number. Okay, so, uh, I'm seeing here. Okay, go look at you know my audio zero. That's that's the base. It's a number between zero and 100. And let's just go ahead and make that. Let's map that to negative 1 20 and then fft rotate. Why? I said my audio data. Let's look at sub item three, which we perceived. Bees snare. It's a number between zero and 100. Let's also map that to negative 1 20 and then rotate Z is are just are arbitrary number or it could even be, You know, if you wanted to, it could even be base again. It it really doesn't matter. You can pick any number that you want as long as it's zero through 10. Since our ranges a number between zero and 10 Eso here. I just picked five for the love of odd, um, greats. A number between zero and 100. And I said, Well, let's map this to positive one. Negative 20. So, you know, you can tweak what you want your map ings to be, but, you know, I just thought I would do something fun now online, Uh, 91 through 93. I'm just plus equaling. Um, this mapping. So, uh, if you remember about the top row, take Nomex numb y and, um, Z was zero. So I'm equaling itself, Plus this mapping. So as these guys air popping off there plus equaling the variables that I set up the top now, after I've done that, let's jump now up to line 83 through 85 I'm actually doing an entirely new mapping. But instead of you know the audio data as the input that's coming in, I said. It's this rotate Nomex that's coming in and that's rotating. Um, why that's coming in And this rotate numb Z that's coming in. And then I also did a min and Max just the same as I've been doing, which is 0 to 100. And then what I'm doing is I'm mapping that to, um, negative to pie divided by 20 and two pi divided by 20. Um, And again, you know, if I wanted to speed this up or slow this down, I could change how this is getting getting divided. So now we've got this way. Teoh map the base to, um rotate X. We've got snare rotating, uh, rotate. Why? And then we just have this arbitrary number that's changing. Rotate, See? So let's go ahead and run this sketch and, um, see what kind of feeling we get. Right. So now you know, notice that we're pivoting on all access is now and again as the audio starts to trigger. That, in turn, is actually going to change how this thing rotates accordingly. So let's wait for this Teoh to do its buildup. Right? So now not only do we have audio data changing the scales of our objects in our grid were now also rotating this environment based on the audio. So we're using that same kind of audio input in two different two different methods. One change the scale, size and one to change its rotation. And again, you know you can. You can tweak the number map ing's, or you can tweet how it gets divided. So, for example, if I were to come here and go from 20 down toe five, watch how the system gets changed. So notice we definitely are speeding this up, right? So it's it's much faster and again, I would start to tweak that division based on really the kind of audio that we're listening to. In this case. I think divided by five doesn't. It's just too erratic. It's the The rotation doesn't best a company the, um, audio. So again, if I were to change this Teoh, you know, 1 50 Um, hopefully you understand that this is gonna be way too slow. All right, Look how slow that thing is moving. So, in a lot of ways, I think that you again, it's a number massaging like, you know, any time I make a project. I'm constantly trying to find those sweet numbers. The sweet spot numbers and, you know, again, uh, five too fast. Didn't quite a company. The music and 150 is is way too slow. So here, you know, we're still getting rotation. But But based on the tempo of the music, the interaction just just isn't right. So again, if I were Teoh, go back to 20. You know, it took me playing with the numbers to figure out that 20 may be best suited. Um, uh, this kind of rotation for this particular audio, right? It feels like it's an accompaniment compartment. You know, watch. I want to wait for this build up again because I really think that that snare kicks a nice rotation right here. See, it feels like the movement suits what's happening in terms of the in terms of the audio. Okay, so, um, great. You know, I think at this point, I've been kind of doing these The sketches now that have been kind of like OK, here's an idea. Here's adjustment. Here's an idea. Here's an adjustment, and with the case of this one, it like it started with 32. We did a grid than a little adjustment. 33 than a little, Just my 34. And then this kind of final adjustment 35. Um, now, I kind of really want to blow that out. Which is 36 through 42. Which is, um, is where we're gonna go eyes where we're gonna go next. And this is seven files. So this is this is taking a kind of more blown out idea and keep doing that step process Keep kind of adding something new, adding a slight adjustment. But here, I'm really flushing it out a lot further. So you might notice that the 1st 2 in this sketch build 36 build 37. We've got our red lights, so we're gonna we're gonna introduce an idea, introduce a new idea, and then build 38 tie it back into the audio. So this shuts down build 35. Um, and I will see you in a, uh, ever amazing and going to be super complicated build 36 38. Build 36: All right. So, as mentioned before, we're gonna take these next seven sketches to really flush out, um, or complex idea. Um, and these 1st 2 were not gonna attach any audio spectrum stuff. We're just going Teoh work on a new idea, Okay? And the idea is is that I want to draw this sphere in the center of the screen, and I then want to, um, have six objects a bit around that sphere in the center. Okay, so let's go ahead and open up. Build 36 we're gonna open up build PD. And again, let's let's just take the time to step through this slowly. So up at the top. Um, line 23 Pool maxes six. Right. We're gonna have again six things orbiting around the screen. And not only am I gonna have six things orbiting around the screen, but I'm gonna give each of those six a specific color. Okay, So, online. 25 you can see I'm gonna do red, green, blue, cyan, yellow, magenta. All right, so I'm gonna draw six things and attach a color to each of those six things. So one movie red one will be green one will be blue so on and so forth. Okay, I'm not randomly picking color. I'm gonna draw six things and specified those six colors. All right, Now come to pool and you'll see that pool is poor Max, which means that the pool is is gonna run six times. And the thing that I am going to have it draw with is an h ellipse. Okay? It's gonna be an H lips. And the size of that lips is going to be 10. All right, um, moving right along here is where we define the H Ellipse on lines 49 through 54. It's not gonna have a stroke and red appear at the top Notice I say integer I equals pool dot current index. And so that's ah, hype function that just keeps counting. So when this runs for the first time, current index will be zero and then one and then two and then three, you know, up to up to six. Okay, so this fill color is saying Okay, go up to the palate. I so goes up to palette. Get zero, which was read then when this pull on create runs again, I is now One goes up to the palate gets one that's green so enforced so forth, so skinny red, green, blue, Cyan, yellow, magenta. So, um great. Now there is an animation behavior written by the ever so lovely Benjamin Fox called h Orbiter three D. And what it allows you to do is specify a size, and then assets will actually herb it around that around that sphere. Okay, so the first thing that I want to do is specify where is the point that the orbiter is going to start? So we've got our sketch, and so I say, Okay, well, the point of interest is with divided by two hike divided by two. Okay, so we've got this. We've got the sketch, and we're saying that the very center of that sketch is where h Orbiter three D should start its position and notice that the Z axis is is just zero. Now, we're gonna go ahead and say that the target is D. So that means that there's going to be, um, six different orbits. Okay. And then the next two lines, I'm simply just setting a Z speed and a y speed. Okay, so I say random, negative to positive to online 58. But that random could pick a zero. So I just always, you know, add, um 0.1 onto the end. So if random or to pick zero, then obviously I would be in trouble because of speed actually wouldn't be moving. I do the same thing for the why I say negative one positive one. And just in case that random pick zero, I just go ahead and add 0.1 onto it. Now, the next thing that you specify is the radius. So we've got our sketch. We said that with divided by two height divided by two was the point. Now we set the size of the actual A little bit radius. Okay, so here I've specified that the orbit radius is 250. So it starts from that with divided by two height divided by two, and basically creates a radius around that point of 250. Now, these next two calls are basically the start position, right? So I want them to start at different positions around the three d orbit. So I basically say here online 61 that the Z angle is random. 3 60 And here on 62 I say the Y angle is random. 3 60 So when each of these little orbits start, they're gonna have a red, green, blue, cyan, yellow, magenta. They're all gonna have a radius of 250. But each of them are gonna have a random Z speed, a random lie speed and that Z angle. Why angle is going to start them somewhere on this three d sphere. Um, business. Right? So they're not all going to start in the same same place now, online 65. Um, I go ahead and save, um d dot extras. And then I just save I. So I'm saving what I picked up on line 47. Great. So come down here to the draw. And I commented out the floor loop right before this. This for loop is the set up for getting the audio reaction tight end. Um, I'm going to just create some lights for this sphere, and so I create a teal, light and orange light and a great light. Right. So, um, the teal light is shining on the top, right? I'm positioning it with divided by 20 So it's Ah, it's a a point of radiating light up at the top so that I know that it's gonna be like coming from above through the orange is with divided like to height. So there's a point at the bottom of the sketch that's gonna radiate light. And then, of course, there's that gray, which I just kind of set right with Divided by two. And in this case, I didn't go height divided by two. I just kind of wanted it to have, like a kind of a bit more of a top down kind of look. So I said that the Y axis on that was was 1 50 rather than height divided by two. And of course, it Z position is a little bit forward at 200. Now, this next business here, the push major pop matrixes where I actually create the sphere. So I want to put a sphere in the center of the screen, and that way, when the lights hit, you know you'll see the lights on this year, and then, in addition to that, the orbits will actually look like they're, you know, orbiting around a a planet. So I said, Push matrix pot matrix, I said, With divided by two height divided by two. The Z axis is zero on here online 94. Now I bump up the sphere detail. I knocked this up to 75 so that you know, it's a lot of little triangles so that I get a lot of detail on that point light. Then I say no stroke. I set the fill to white and then I create a sphere at 125. Groovy. So hopefully you understand all of that. Now let's just go ahead and run this and see what we get. Okay, so we should be hearing the sound. But again, nothing is tied to it. We use the pool to set up our six er bits. They did pick red green broom, cyan yellow and Magenta Bay picked completely different speeds. They started at completely different positions, and they're simply orbiting a radius of 250 between the with divided by two height divided by two. So right in that in that center screen and again we got this point light of teal just shining on the very, very top here. Notice that I put it at a negative of 100 so that teal would actually kind of kicked back a bit. Whereas the orange I just put it zero. So see how there's kind of less teal and mawr orange. So by changing the Z positions, I can kind of control where the light hits on that sphere on. Then again, Look at this gray. If I had gone height divided by two year, that means that gray would have been perfectly right here in the center by putting that, uh, that point. White 51 51 51 at Rift divided by two on, then 150. It just means that the light is like right here, but it's pushed forward. 200 on the Z axis. So again, way kind of get the highlight. Great on this side of the sphere. Now, what's nice is with this Is that little bit that three d orbit, So those are actually moving in Z space. So you do see them actually go behind the planet? Sort of in front of the planet. So on and so forth. Okay, so this is a great start um, to this particular sketch, and I kind of want to introduce, um, one more idea next before we actually tie the audio to it. And thats in build 37. So let's take this build 36 we'll end this video here and will move on to the next modification and build 37. 39. Build 37: good to go Stick with me here. You're gonna like this one. Build 37. Now, um, we've got these orbits. Yeah, I got these orbits moving around, and, um, we're gonna make some some changes, but we're gonna go through them slowly. So the first thing that I want to look at its pool and there's a little adjustment here. So what pool does is is pool handles our draw bulls and our draw Those are the h ellipse. Yeah. And, um, we make the six h ellipses and we color them red, green, blue sigh in yellow magenta. Um, And then we create this eight orbiter okay, and way create six of them. So we create six h orbiters, and we tell the h orbiters who the draw bulls are. Okay, so now what I need to do is actually create a link between these two things. So look at line 65 Line 65. Says, uh, I am going to add an extra onto the draw bull on before it's always been like a number. This is what I love about extras and specifically extras. H bundle is You can also pass it an object. So I'm saying, d dot extras new h bundle. I'm going to give you a numb called I, which is equal toe I. So each draw bill knows whether it's, um zero through five, Right? 012345 or six troubles. And then I pass a reference to the h urban or three D. Okay, so I say 0.0 b j and I'm just gonna call this Ofer for orbiter, and I'm gonna pass it. R h orbiter three d herb here online 56. So now we've got this pool of six h ellipses that are storing what number? They are zero through five and their associate ID. Um, each orbiter three d behavior. All right, awesome. So now we're gonna go down into the draw, and now there is this huge chunk. So the first thing I do is I say, Well, OK, let's cycle through our six h ellipses inside of the pool H bundle temp, extra d dot extras. And then I say interred. Your I is equal to put each draw balls. Um, number I. Okay, so now I know my zero. My one. My to my three of my former five And then I create a new temporary instance of H Orbiter three D. And I say, oh, is equal to go to the Ellipse and get the associate id each orbiter with that ellipse. Okay, so the reason why I want to do that is is because now I've got these orbits moving around. Well, now I want to make these point lights, right. So point lights have always been a point in space in which light radiates, uh, great. But now I want that point light toe actually be moving in the same exact position that the orbiters are. Right. So I put in a little switch case here so that I can cycle through my six assets. So I just say switch, I and I look for case zero case, one case to a case five. So case zero. I know his red case one and it was green, you know, to lose. So I say Okay, Well, if this four loop is running for the first time, then eyes gonna be equal to zero. This case passes Troon. It says point like I want you to be, uh, red to 5500 in your position right. Your X y Z position is read each ellipses X Y and Z position. So now, in build 36 we had these six things moving around. Those six things moving around are now also the positions of the point lights, right? And then I say, Okay, when the four loop runs, you know, again and it's cases one it goes, and it creates the green light and says, Okay, cool. You know, find the green h ellipse, and wherever its position is, that's now the position of the green light. So I'm using the switch case to, um, attach lights to these moving three D orbits. Okay, Other than that, everything else is the same in this sketch. So let's go ahead and run this. And it is a thing of beauty. We have the audio loading in. We have these six h ellipses orbiting, but those orbit positions are now the positions of point lights and the, uh, it's map. So red is a red light. Blue is a blue light. Green is a green light, so on and so forth. So now we've got this sphere in the center of the screen and now we have these orbits moving around of the planet and those things moving around also happened to be the positions of radiating light. Okay, so super rad. We're off to a really, really amazing start. So I'm gonna close this because I want to say that obviously, we're gonna add sound now. So build 38. We're actually gonna have the audio, uh, change this, this lighting environment. So, um, you know, uh, sound controllable lights. You know, it doesn't just necessarily mean that it has to animate things forward or or animate a box around. Like we could actually use this audio synthesis Teoh to move around a lighting system in our in our three D environment. Okay, so this is gonna end build 37. We're gonna move on to build 38 we're just gonna we've in our our audio into this, uh, letting environment. All right. Still 38. See? There 40. Build 38: we are in build 38. So let's go ahead and open up this build PD and we are now going to attach the audio to this lighting environment, but we're going to do it in a pretty interesting way. So that sphere in the center of our screen waas um, a sphere at 125 pixels. But our radius was larger, was at 250 radius for the actual orbits. So if our sphere in the center is 1 25 but our orbits is 2 50 What if we could use the audio to actually control? Um, the radius of the orbits right. So I would have done here is about the top. I have two new integers radius men and radius Max. So urban Radius men is set at 1 25 which would be the same size of the actual sphere that's parked in the center of the screen and oh, radius max or orbit Radius Max is set to 2 50 which is that that outer radius that was running and, you know, build 36 build 37 Now, um, before we were using a new H ellipse right. And so I thought, uh, it would be fun to actually switch out this draw able to an eight sphere. So now we've got sort of this planet in the center, and then these air kind of like little moons. Um, and I've made them a sphere, so I've made spheres orbiting a larger planets. Fear. Right. So here online, 46 I changed from H Ellipse, too. T h sphere down here in line 52. I added this, you know, size 10. So each each sphere has a size of 10 again. No stroke again. The Phil is getting a red green blue sign, yellow magenta from above. And pretty much everything else in this set up remains the same. So the only change to set up was swapping this out from H Ellipse two X fear and adding a size of 10. Now, let's come down into the draw and realize that, Okay, I'm setting up these point lights and these these point lights are, um it's it's points in which light emanates from on. That's great. But I run into a problem because now I'm drawing these little H spheres. I want to make sure that these H spheres aren't actually getting lit by the point lights. So look at what I dio I say no lights. So I shut all the lights off and then I just turn on that very soft ambient white light. Then I say that the sphere detail eyes 10 and then I draw stage. So now I'm drawing my six h spheres, which used to be a chill lips is my six h spheres with a soft ambient light. And then, after h draw stages done, I actually turned the lights off again. So the the lights turn on for the little orbits, just toe light them with a white ambient light, And then I shut the lights back off again. Now, inside of this inside of the draw When we're looping through our pool, I've made a little change here, which is before we were just setting inside of the switch case. Just the point light position. But I'm also gonna change the radius, so check this out. I say, um, fft radius. Now, um, go look at my audio data. I So go look at position zero. Right. So actually read is going to be the base because it picks zero. And then I say, OK, it's a number between zero and 100 but map it to 1 25 for our men and 2 50 for our max, and then online 101 I actually changed the radius of the orbit. So the audio is actually expanding and contracting the radius of the urban in which they in which they behave. And again this code just gets, you know, copied. And I'm doing it for every single one of, um, the switch cases here. Now I think I also made Yes, I made a little adjustment to the my audio data widget toe actually show the colors inside of the because right, because zero is red run is going to be green to is gonna be boo. So actually, the six orbits, even though our expect room is 11 I'm only using the 1st 6 toe actually get mapped to these to these orbits. But I think if I remember my audio data widget up here at the top has a setting of false. So let's go ahead and talk all this to true so that we can see this working and I'm gonna go ahead and save and run my sketch here. Okay, so look at this now. So we down here is our audio spectrum, and I'm color coding Red, green, blue, cyan, yellow, magenta. Right. So even though our spectrum is 11 were only listening to the 1st 6 So red is our red dot You know, green is our green dot We can see the H sphere, right? With the, uh, with the lights. Just a nice, soft white ambient light. And what I'm doing is I'm expanding and contracting the radius of the orbit. So if the spectrum goes quiet, it really is just going to skim just on the top of the surface of that 1 25 sphere that we have parked in the middle of the screen. And again, as these start to amplitude up, they then jumped their radius. I'm expanding that radius to 2 50 So they appear that these little moons air actually dancing off of the planet. OK, so this bill 38 is, um, is looking like a fantastic start. So now let's start to think about you know, what other rays can we use audio to control this environment so I'm gonna go ahead and close a sketch, because now what we're gonna start to do in the next couple of builds is would if audio could change the speed of the of the orbit. So not only will Bay the moon sort of dance off the planet, but their speed of of the moon's would actually slow down or speed up, also based on the audio spectrum. So I'm gonna close this sketch build 38 I'll see you in build 39. 41. Build 39: we have four to go. Build 39. Um, great. Let's go ahead and open up this build PTE here. And, um, in the case of this build, 39 as I said in the previous sketch, would be great if the audio could also control the speed of the orbit. Now, when you're talking about speed, if you look down here in this h three de orbit, we've got a Z speed and we've got a Y speed. Right? So appear I'm actually gonna set up some new floats. So the 1st 2 floats that I'm going to set up is Ziemann and Z Max, And then I'm gonna set up Wyman. And why, Max, Right? So I set up Zemun at 0.5 z max at 5.0. Why? Men at negative 0.5 And why Max at negative five. Um, so cool. So now I've got a min and Max for speed. That, of course, I'm gonna map to the audio spectrum and so that if it reaches zero, it's going toe our men. And if it goes to 100 it goes to our max. The important thing to remember is that these are floats, right? Cause I'm doing things like 0.5 So I'm not working with, uh, with interviews. At this point, I'm working with with float numbers. Now again, Not much changes in set up. Everything is pretty much the same. You'll see here in Z speed and my speed I put in that Ziemann and that Weinman Um I was good and again it's just sort of a larger addition Teoh to the draw into the switch case. So now I'm changing the radius of of the orbit. Now I want to change the speed of the orbit based on the audio. So online 100. I create some new temporary floats for FFT Easy speed and FFT wide speed and what I'm doing here is saying Okay, Well, Z Speed is mapping the audio spectrum again. Looking at, you know, for red it's looking at sub item zero a number between zero and 100 then looking at Zemun and on Z Max for the speed. And again, of course, is doing that also for the why and then this line one away just changes because you're saying I want to change the orbit, Z speed. I want to change the orbit. Why? Speed and I want to change the orbit radius. So now we're expanding and contracting the radius, and we're also speeding up and slowing down the orbit speed of these of these assets. All right, let's go ahead and run the sketch. And of course, you know, I do. Um, the same four for green, for the blue, for the sign for the yellow magenta. I forgot to mention this in the 1st 1 but so red would be our base and actually sigh in would be our would be our snare, because that's that case three, um, so that that that science light should be acting on the snare. So look here. Now we're expanding and contracting the radius, but we're also getting a nice, nice movement in terms of speed, of getting these little moves to sort of rip around the planet based on based on the audio . Okay, so, um, cool. How can we take this even further on in build 40? I'm gonna show you how we're gonna rotate this thing even more based on so closing down. Build 39. Getting so close to the end and I'll see you and build 40 42. Build 40: So before we, um, start build 30 were actually gonna look at the past a little bit. We're gonna look at build 35. Do you remember this sketch where we used the audio to actually control the rotation of the entire universe? Theo Entire universe. Remember that when we were screwing everybody up by rotating the entire universe? Yeah. Uh, that came out weird. This universe, right? So we were using those odd numbers were using 03 and five. So base was controlling one rotation three for snare was rotating another rotation. And we just threw in an arbitrary number five t change this rotation. Now imagine we apply this to our build 30. So I'm gonna go ahead and close this, and I'm gonna go over to build 40 now and let's go out. Go ahead and open up, Bill PD. And let's imagine that not only are you fluctuating, the radius is not only are you changing the speed, but based on snare and that arbitrary number are actually gonna be rotating the entire universe, right? So I had to re add these three integers for rotate Nomex numb winem Z set them Teoh 20 sorry, I had to cut in a little edit right here because I initially had said that there was no change to set up. And actually, that wasn't true. I did need to change here online. 71. Um, in our previous example, the H Orbiter three D, Its start position was with divided by two height, divided by two and a Z of zero. So I actually have to update H orbiter three D Toby an X Y and Z of 000 So if you were looking at the sketch, it would actually start in the upper left hand corner of 00 now for the orbit because we're going to use down here online 95 a push matrix pot matrix translate to go toe with divided by two height divided by two. So I had to edit this in because initially had said that there was no change to set up, but you do need to make this adjustment of where the ah position point of H three d orbiter is, um but look at what we did here. Now we had toe kind of, uh, do some some nesting and all the way up at the top here I say no lights and then it put in a push matrix. I put in a translate with divided by two height, divided by two. And then remember, I put in this rotate X rotate Why rotate Z But it was all based on getting f f T rotate, X, Y and Z you know, looking at 03 and five. And it was putting in that, you know, these numbers and dividing them by 20. Right? So this was now this push matrixes, translate this rotate All of the stuff was now getting applied to the entire sketch as a whole. Then we go ahead and do our lights. Then we do our sphere detail. Then we draw our stage, which is our little H sphere moons that are orbiting around. And then we can go ahead and start to do our HDE Wrobel pool. And after our h draw bowl D in pool is done and it's it's ran through this switch case where it, um ah changes the speed for the Z, the why and the radius. Right? Then we come down here and we push pop matrix for the sphere in the center. And then here is that last pot matrix right there Online, 1 83 So, uh, here is the start, that push matrix on 94 and then all the way down on here is the closing pop matrix. So now we've got the audio controlling the, um the well, let's go, Let's run it. And then we can weaken recap. Okay, here we go. Right. So not only is the audio spectrum changing the radius of the orbit the speeds of the orbit , but we have based snare and then just that arbitrary number, actually rotating the moons and the planet. It's hard to see. In fact, actually, you know how we could see it. Watch this if we came here. Uh, here's where we're actually drawing this fear in the center. Let's just, you know, again, this is just for for funds. Ease. Let's say that the stroke is 2 55 So look, I say no stroke. And then it's like I'm getting stroke. Um, sphere detail 75. That maybe. OK, let's Let's test and see what happens. So by visualizing the stroke of the sphere, I am illustrating the point that not only are we fluctuating the radius and fluctuating the speeds. But we're actually rotating the entire universe as a whole. If you actually put the stroke of 2 55 in the previous build 39 you wouldn't see the sphere in the center actually rotate. It would remain fixed, because now we're actually using that base. That snare and that arbitrary number toe, actually really take the entire universe in which the planets and moons live. Groovy. All right, let's go ahead and take out that stroke of 2 55 on. Then how? Let's just go up to the top and say Show visualize. Er equals Fallston. Let me run this one last time. Okay? So, uh, everything looks good. And this is the perfect set up for our seconds to the last build build 41. So if I'm using, um, if I'm using, uh, this rotation of the entire universe, right, it would be great if we could see it. You know, the only way we could see it is when we actually change that center sphere to have a stroke of 55. So online photo one is like the perfect opportunity to actually start to do some texture mapping. So we talked about the center thing being a planet, why not make it a mesh and texture map that mesh with PNG and actually make it a planet? So, uh, I'm gonna go ahead and closes down 40. We're just We're getting a lot of weird, really great movement from the audio spectrum again, we've reduced it down to 11. But in this case, where the animation is only controlled by the 1st 6 that red that green that blew that sign that yellow them agenda. Um Okay, this ends build 40. Um, second to the last file. Man, build 41. It's coming to an end. Um, but we've made some great sketches. So, um, see you and build 41. 43. Build 41: best shirt ever. Ah, we are in build 41 second to the last sketch together here. And, um, what I've done is is inside of the data folder. So if you look at build 41 you look inside of the data folder, there's a new PNG called texture dot PNG. And if you open this up, it's just a two dimensional texture that we're gonna wrap around. That sphere that sits in the center is our is our planet. Um, and and actually, um, for this particular exercise, there's, you know, there's a lot of ways and a lot of libraries to create meshes and texture mapping. But I thought I would just use the easiest one, which is right on the processing dot org's website. Um, it doesn't work in the browser, but there is this example up here for textured sphere on the processing website where you know, this this code actually behaves quite well. So we're gonna we're gonna take the coat from that from that example. Now, let's go ahead and open up build PTE. And you'll notice that up here when I'm setting my variables, I had to create a new block of variables, that is, from that processing dot or GTA Example. Sketch setting a planet, texture or planet with plant height. You know, numb points. Um, coordinates X, y and Z multi x y x z So on so forth. So this is copied over from that processing dot org's texture example. Um, you'll notice here and set up I needed to add some new stuff here online 60. I say planet texture equals load image. Hey, go get that texture dot p and G that I actually just showed you, and I'm gonna say planet with Sequels Planet High equals 50. And then I'm gonna go ahead and initialize a sphere, okay and initialized spheres the function that's that's down below, that we're going to get to in just a moment. Other than that, nothing has changed and set up. Everything pretty much is still the same. And for the better part, most of the same stuff is happening inside of drawl, right? We've got our push matrix and pop matrix were rotating the entire universe. All that same business is still going on. We still have this switch case that's moving the moons. Um, across, um, the, uh the the little planet the middle on its changing The radius is in the speeds and again unchanged. Nothing. Nothing different there. Great. But if you come down here before, it was this area right here where we were actually just drawing a sphere in the center of the screen. And you know that that was fine and all the other sketches. But now I want I want to run this texture sphere. So what I've done is is ah, put in a push matrix of pop matrix. I've got a translated that a rotator axle rotate. Why rotate Z And then I make this call to textured sphere 1 25 1 25 1 25 and then I pass it the texture. Now, if I look here on line to a want to 09 all of this code specifically, these functions like the initialized sphere function and the textured sphere function these right here are really just copied over from the processing dot org's example on their on their website. I didn't really change these functions. Also, these were just these were just carried over. One of the thing that I just wanted to point out is, um, when I actually go to draw the textured sphere, it's gonna, you know, it's always gonna draw that spirits office map, the texture to it. But what I noticed was is that as the moons were, you know, moving around the planet Ah, it wasn't enough white to really kind of really bring focus to the to the texture. So again, notice I just drop in one little simple point light. So I've got this sphere and I just dropped in a point light with its Ah, it's a medium grey. Right? So the red green blue is 101 102 100. I said that the width was divided by two. And then I actually said that it's why position was negative. 500. So, you know, if this is was our sketch, I came with divided by two. And then I actually put the point, you know, negative 500 up here. So it was kind of like a nice elevated, um uh, light above. And I also moved that light to a positive 500 on the Z axis. So that would kind of cast this very large, very subtle medium grey light on top of our textured sphere. So let's go ahead and run this and see how this is all buttoning up. Okay, so the first thing that you have probably noticed is that grain light that I was talking about It's nice, Nice grow light that really just sits on the top of this planet, and it starts to Peter out, kind of like right around here. So, you know, eventually you see it's it does kind of remain dark down here when the when the lights aren't, lights aren't on it. So I am giving this this overall kind of soft glow on top of that Texas fear. But you're also getting the colored lights like that as well. Now you may have noticed why was the texture sphere grayscale again? The reason why it's great scale is it goes back to a conversation that we had many sketches ago, which is as long as you're using black or white or using grayscale when you start to tint color or, in this case, use lighting to light something, anything that's great scale will or white upto white well actually soak in that color. So if this textured sphere was actually a colored texture. Then we'd be doing color on top of color, and it kind of gets muddy super quick. So any time I'm gonna be using this sort of lighting environment, I try to make sure that most of my textures are gray scale so that they really soak up the color rather than two colors trying to trying to fight each other. Now, um, this this class has been has been great. I really hope that I have. I've showcased all these different possibilities of what we can do with sound, and somewhere you should be saying like, Oh, my God, it's been 41 sketches. And, you know, I think you the reason why I took this class is I want to know how he does these live show things, you know, um and obviously that's going to be our build 42 is how do we tie this, actually to to live audio. So up into this point, build up to build 41. We've just running been running off of way files. Or you can run off of MP three files or whichever, um, that we will be like wave best. Um, great. Let's go ahead and shut this down and and build 42. Let's have this live audio discussion. See him build 42. 44. Build 42: shit. We are the champions, my friends, Because we made it to build 40 to him and, you know, anyway, that Hey, listen, uh, it had to get weird. Um, we're live life. Live audio. Um, so look at the data folder. Took the way file out because we're actually going to run off of life data here. So inside of data, the wave files been removed, and I've added a little PNG here for rings on. Actually, this is transparent. If you open it up in Photoshop, they're beautiful. Beautiful rings because this planet wants to be Saturn. Um, and of course, we've got our texture map, but we've got this little addition of having some rings now, believe it or not, the code mod for doing live audio isn't that much big of a difference If you come into the build PDE, what you will notice is, uh, at the top here. Before we had audio player, an audio player was an object that we could specifically load a audiophile into. So the adjustment that we make is we just say that instead of my audio being an audio player, we say that my audio is audio input right, Um, so this is, um, whatever the audio input is and so I can go into preferences and I can say that the audio input is a microphone. Or I could say the audio input is the is the line in from from the computer. So if I go on play a live show and I just want to listen to the musician, what I would do is inside of preferences. I would set the audio input to actually be lying in, and then I would ask if I could get a line from the soundboard into my computer that's running this code. So that way, my code is just hearing the music that's actually coming from the musician soundboard, you could obviously do this, um, with audio input being a microphone, then you're actually making art that is yes, listening to the music that's being played in the space, but also the audience. So that's massive because that means that now the people seeing the show can participate, and they can learn very quickly, like the more they yell the mawr that changes the visuals. And so again, changing my audio from an audio player to an audio input. Okay, Now, everything else pretty much has remained the same until you get down into, um, just above set up, I want to load in that little rings. So I just said P image rings on then here online 65 I said rings is equal to load image, and then go get that rings dot PNG So now we are loading in that transparent PNG for rings . Now, uh, when it came to defining the audio and set up, we had this, which was, you know, my audio is equal to minimum dot load file and then load in this this way file. And then, you know, we would specify online 75 either to my audio dot loop or my audio dot play. So the only adjustment is, as I say, that my audio equals minimum dot get line in minimum dot mono. So now that line 72 is pulling in. Ah, the music live as it's as it's happening. Okay, Now, I would love to say that there's a bunch of other stuff that's different, but unfortunately there isn't set up pretty much remains the same. You might notice here inside of the pool. I just cleaned up everything. Teoh use that method, chaining all in one line rather than doing the hard returns with tabs. And that's just a make the code a little bit shorter. Um, come to the draw Push matrix pot matrix. All that business pretty much is still the same. The only difference is is that right here on line 1 92 through 1 91? Yes. I'm creating the textured sphere in the center and then right after it Here is that new addition of actually adding the rings. So I actually put the rings also in the center, and they're going Teoh, rotate around as well. Now, I'm gonna go ahead and, uh, run this sketch and you can see that we're now using the line in microphone Migra. Phone data. It's me, Mr Crabs. SpongeBob. You got to go make me crabby Patties. Right? So now I can take all of this and have it run off of lie badia, right? And it works. Um, absolutely perfect. So the softer makes him sounds, uh, who? Uh right. So it works. Perfect. So it's this final step where I can actually have this stuff run off of um, that line in and that line and again could be soundboard or can actually be three audience . And so I say to you that we are the champions. We are the champions of this class way I thought, All right, Uh, this ends build 42. It's been fantastic, really. It's like, you know, the I wanted to showcase all of these different ideas, but it's it's built on these simple building blocks, which is just taking this audio spectrum, reducing it down to something that that fits, um, the the genre of the music. Identifying what parts in that spectrum are things that we should tack on to, like bass snare vocals, guitar, you know, and then using that using map to say, Listen, this audio spectrum stuff is happening. I could see it happening here. These bars that are moving up and down as I talk and I wanna know that that's a number between zero and 100 then I'm gonna map this to whole wide a range of different things. It could be moving things in space that could be controlling the speed of a behavior. It could even be controlling the lighting in a three d environment. So, um, I can't thank you enough for, um, for going along with this adventure with me. This was painting with sound, and I think I will end this with one last video that I'll close this one and I'll meet you in that video. And we'll just talk about, um, the next class because there's actually 1/4 class that I'm working on. So this was done, and I'll see you in the your awesome I'm awesome were both awesome. The love we share kind of awesome closing video. 45. THANKS: So I just wanted to create a closing video just to say thank you. Um, I really hope that you have enjoyed this class. We really rocked it through 42 sketches. And it's funny because, like, I think I had mentioned before, I had written an entirely different class. Um, that I was gonna dio before this and that was going to be programming graphics, three texture mapping and many controllers where I show you how I use many controllers to trigger all these different states that I do for bands when when they go into her. But like 3/4 through that class, I realized, you know, sound is such a huge component to this. And a lot of people were asking in some of my other classes like, Hey, are you ever gonna cover sound? And I realized that I should probably take that class, set it off to the side and do this one first. So I hope you really enjoy this. The next class again is going to be texture, mapping and midi controllers. If you have any suggestions of other stuff that you would like me to try to entertain you with through the learning process. By all means. Let me know. Is it Arduino is a leap motion. Is it connect? Is it? Is it What? Um a heartfelt thank you. I love doing this. I wish this was my job. I want this to be my job. I want us to make magic together. Not now. Not in a weird way. Just like in a a couple of good wizards way. Thank you again. Um, keep in touch. Speaks it.