Introduction to Creative Optimization for Digital (Online) Marketing | Stephanie Seow | Skillshare

Introduction to Creative Optimization for Digital (Online) Marketing

Stephanie Seow, Digital Marketer in NYC

Introduction to Creative Optimization for Digital (Online) Marketing

Stephanie Seow, Digital Marketer in NYC

Play Speed
  • 0.5x
  • 1x (Normal)
  • 1.25x
  • 1.5x
  • 2x
5 Lessons (46m)
    • 1. Creative Optimization Introduction

    • 2. Understanding terminologies

    • 3. Anatomy of an Ad

    • 4. Creative Optimization Process

    • 5. Creating a Test Plan + Class Project

  • --
  • Beginner level
  • Intermediate level
  • Advanced level
  • All levels
  • Beg/Int level
  • Int/Adv level

Community Generated

The level is determined by a majority opinion of students who have reviewed this class. The teacher's recommendation is shown until at least 5 student responses are collected.





About This Class

In this Beginner to Advanced Beginner level class, I will be explaining some technical marketing terms and what they mean - we will be unpacking the concept of creative optimization, AB testing and multivariate testing. I will be discussing various common ad formats, what are the elements making up an ad, and therefore what we could be testing on to improve performance. We will also be going through a simulated process of creative optimization process for marketing campaigns and begin to conceptualize testing plan to optimize our marketing efforts. 

Please follow and stay tune to any future classes, where we deep dive into more tactical optimization work on platforms to complement creative optimization process. Thank you for learning with me and I hope you have enjoyed the process of bite-size learning and new knowledge-seeking.

Meet Your Teacher

Teacher Profile Image

Stephanie Seow

Digital Marketer in NYC


Hello, I'm Stephanie.

I'm a digital marketing professional currently based in NYC. I have worked and consulted with a wide range of start ups in North America and APAC regions. Visit my portfolio if you'll like to see more of my work and experiences:

I am also passionate about design thinking and human-centered design. Please reach out if you'll like to collaborate with me on any interesting initiatives. 

Professional work aside, I am a budding nail artist... perhaps some nail art tutorial coming your way too? 

See full profile

Class Ratings

Expectations Met?
  • Exceeded!
  • Yes
  • Somewhat
  • Not really
Reviews Archive

In October 2018, we updated our review system to improve the way we collect feedback. Below are the reviews written before that update.

Your creative journey starts here.

  • Unlimited access to every class
  • Supportive online creative community
  • Learn offline with Skillshare’s app

Why Join Skillshare?

Take award-winning Skillshare Original Classes

Each class has short lessons, hands-on projects

Your membership supports Skillshare teachers

Learn From Anywhere

Take classes on the go with the Skillshare app. Stream or download to watch on the plane, the subway, or wherever you learn best.



1. Creative Optimization Introduction: Hi, everyone. My name is Stephanie and continue to my classes. I'm a teacher marking professional here in New York. And I have over 60 s experience now marketing digitally on various at channels like Google Baseball Instagram s such a for direct consumer prints. I have worked previously in the Asia Pacific markets, and I'm now located in the North American markets. So for those of you who have taken some my classes that are released earlier this summer, I like to say my big thank you for your support and for learning with me, understand that I am primarily a marketer and not a teacher. So teaching online has been a very new and wonderful experience to me. If you have any questions to any of my content, please feel free to reach out and ask any of your questions. You can find my personal website and contact on my profile peach or if you feel like you can feel free to connect with me over, lengthen my classes at tailored to the absolute beginner to advanced beginner audience. So please see these as you're stepping stone to jump start, you're learning process on to other more advance concept, which are hosted by maybe other teachers on this platform or other more technical based platforms, which provides certifications. If you are intending to pistou, that's your marketing as your full time Korea, beyond just interest learning so in their friendly and let's get started. So once again, this classes for anyone who wish to know more about this, your marketing concepts and you do not need any background to take this class. But of course, if you're totally foreign to the whole dish, your marketing discipline, then perhaps taking my introduction to ditch your marketing class might be a health, a way for you to get started and to be introduced into the landscape off the practice. So in today's agenda will be covering for some of the Juergens, which you may have heard marketers toss around in their work discussions and conversations . So we'll be diving into the two concepts that are relevant to today's topic and finding out what on earth they are. Next. We will dissect the various asked that we encounter each day when we served the website, or when we sift in and it or when we browse on social media, we will check out what an ad is made up off and understand them as variables, which we can test bonus as well. We will be talking about landing page, and basically this is where, ah, landing pages where users will land after clicking on your at and obviously ah, good user experiences crafted when you send users to a relevant landing page, where they can find out more information that they seek to learn more about when they see your at. So this is a crucial page that is worth our attention and study for various ways to improve and to drive users to the desired conversion action that we wish for them to take on on out page. Next, we'll look at some examples off creative optimization in action, and lastly, we will start thinking about how to craft a test plant. 2. Understanding terminologies: So first up, let's unpack the Juergens. 1st 1 we have is creative optimization. Let's see this as a magic process off, changing up and improving on your creative or, in other words, you're ATS. To drive increasing conversions, conversions here can refer to any actions that is the end goal for your business or your advertising goals, so examples could be purchase orders, revenue, app, download or lead Sign up, etcetera. Next we have is a B testing, and this is a method that is used in creative testing, also known as the optimization process that refers to creating maybe two versions off a creative with just one variable that is different. In order to find out which of the two works better for testing, more than one variable is referred to ask the multi variant, multi very testing and for now, and that's it. And that's it for now. Hopefully, that was simple and easy to understand. Otherwise, please sit tie as we will dive deeper into the examples off. This two terminologies played out in action in section three of today's class, and for now, let's get the anatomy off and ask 3. Anatomy of an Ad: for a dose of your have taken, sell my other classes on at platforms introductions. You might be more familiar with some of these examples off the types shown here. If you're not familiar with this, do not worry. You can still check out some of my classes after this, which is a few three part series that I created called the introduction to At Platforms where I walk you through the at set up processes on the various and most common at channels out there today. For now, let's not worry about the platforms, and let's just simply study this adds and see how different they look as a result, off the differences and set out and in nature or contacts of this adds, and where and how they appear on the networks. The first up that we have is a dynamic display at this is what we call programmatic ads that we buy on a life bidding auction that's entirely automated by platforms like Google and other vendors to allow advertisers to set their own bits to buy at spaces on the in in it. Think of it like art auction, where people, you know, basically come up with different bits and basically the one with the highest bit when ah, place in the auction. For most of this at platforms, however, the winning bid will be the second highest bit. And, um, that's a little bit too technical here, so we wouldn't worry too much about it. But that's basically how an at auction function is very much like a real life auction there . We know off with this, with the entire with the whole process entirely automated body machine powering Theis platforms. So for a dynamic ads and average heuser will simply upload a bunch of images various copies for headlines, some headlines and call to Action's onto the platforms, and then this get automatically rendered into different combinations that best fit and given at spaces available on the website. Uh, all around in and the machine will also study and decide which combinations of this will give you the best performance. Teoh have to use us. Take the desired actions that you want. So for some of this ass, which you may have come across, they may look very much blended into the native environment of the website that your browsing it's almost like the content is a part of it and not inserted as the by 1/3 party advertisers. So for this asked, they're called D native acts which are also programmatically place and also put together by machine learning off the at serving platforms. So basically, that's the contacts behind this ass and how they come to show up in the way that they look . And next we have social ass, which I feel we are the most familiar with. So the study asked that we see on Facebook and instagram, and typically they also try to mimic that negative experience that is present on. They have made to look a lot like a regular organic post at anyone that your friend or some a peach that you follow What post on on the feet. Lastly, we have what we call the standard in the banners, this creatives or banners, as we call it, all static images they're created by. Perhaps you're a graphic design team, and it's already created in the various specifications and sizes that exists as a requirement on the various at serving platforms, and this is simply uploaded as they are, Um and you will Puris. They are just the way that you have designed and intended it for it to look like. So now, less deep, dive into each of these ads and your variable components. First up, we have our standard display ads. As we see here, there are a variety of sizes I want dive too much into the specifications off all these at sizes. They're available, as it is slightly off topic for today. But since there are a variety of sizes that in itself can be available to test for and to focus, your limited resource is into creating at units that actually performed better versus creating for all of the sizes they're available to run in the universe. So size aside, we see here something common to all these brands. Bettors are the company logo, which is important for creating brand awareness. The hotline, which is often your value proposition or, in other words, you're selling point to the audience and lastly d call to action. The culture actions are abbreviated. Often asked the C T A is simply put the action, which you once, which you want users to take upon seeing your app. These may be to shop now download seamore learn more or even trying now. So recently, businesses have, you know, started with home. Try on programs for free, where customers don't get charged for trying on something until the A certain that they will want to buy it, and so they will keep it beyond the trial pig grit and then at their after their credit card will get church for it. So in such cases, you often see businesses. I'm using the C t a. Off trying now for free, for example. So all this sounds very common sense to you, but a CT a button is the most crucial one of the most crucial elements of an ad and cannot be missed. It is universally known as the industry best practice, especially in an environment where use is increasing clean desensitize to banners and wire to almost ignore ATS in all of the environment. Wet environment, their browsing. So the C T a button then invites uses in erections and provide a reason for you, the user to actually click on the add and to welcome the brief distraction from he's on her viewing experience. In order to engage with your offer on another Web page. So next we have the dynamic ads, which we just learned earlier on dynamically rendered by the at serving platform. So rather than uploading ready made images instead, marketers will input a library of images and copy into System 40 platform to come up with various variations off combinations. And this is in some situation. However, advertises can also choose to upload their own HTML built betters to the system in order to have more control over how the ads are to be displayed. This is especially for brand conscious brands who wants just more control over their bringing. But in general, this is actually a time saving and resource saving tool for advertisers to use. And again, this at made off also very similar elements as the standard display banners that we just saw. In this case, we often CD. We often see the missing brand logo eso unless it's incorporated into the image to advertise this upload otherwise often the only place where the brand my show, will be the preview off the u. R L on the at, um, and for text based at even smaller at units, they might not even be in Mitch component to it, but only ability text, as we see here, in example. So here, unlike the standard displayed banners, there is not just a headline, but there's sometimes room for a sub hit life in some of the at units. A subject line here is a chance for you to elaborate more on the offer and maybe explain the mechanic off the promotion that you are highlighting in the headline on Finally, we have to see ta again the C t a. Here. Usually it's just a button, or it can contain tax like a standard display at wood. As we see in one of the example here. Again, the system will decide on the final rendering of the elements into an at unit that would best fit thesis eyes and specifications off the website that issue appeared on. So it's Yeah, it's a good tool to have everything automated for you and to not missed out on any opportunities to show up for any at units they're available and Web pages that you may not have otherwise created for, or have the internal resources and bend with Kuwait and upload a standard is better for so next thing. Finally, we have thes social ATS, and this is on Facebook and Instagram. Of course, they're also adds on Twitter on pinches and Snapchat, but I've chosen to focus on this two most common platform. Stay on now. I was driving the most amount of revenue success for advertises at the moment at the recording of this class and here. Since we see that the brand logo will appear as a profile posting the at, there is no need to have the brand logo appear again on the image itself. The image theories often cap that's it is like a just an image. If there any tax to go along to highlight an offer. The rule by Facebook is to keep the tax in its ratio as under 20% off the total image area . Otherwise, they would say that it will be limiting the amount off delivery off at, and you might not be able to reach as many audiences possible. This is because Facebook and Instagram typically want thing at experience to mimic very closely with the rest off the native post environments appearing on a uses feet and so having a very largely text based image would take away from that type of experience, So, um, yeah. Instead, Facebook wants advertises to keep the offer to D at caption space. So like writing any organic post Aires a space for you to input your caption. So this is where you shoot right all off the details to your promotion in. And that's also headline and some hit line space in the for Facebook wise. So that's also on the opportunity for it to elaborate on your offer and notice here that the headlight ends up hit. Lyon is only display on Facebook and not on Instagram at unit, because Instagram do not have this features for your app. So even though Bull Facebook and Instagrams at ah served through Facebook at manager Platform of switches just a single platform serving to true of it, Theoren did differently in their respective environment. So lastly, we also have the C t a button, which is slightly different on Facebook and on I G. And that's all for understanding the anatomy, often at and before we move on to talk about optimization, I would like to jump briefly into landing page optimization. This is because a laying pages essentially the extension or even the culminating point off a uses journey through your at into your website. So once a user landed from an at, we need to ensure a continuation off that same dialogue and an elaboration off the information that the user sort when they click on the at. So here we see an example from Pharma Stock. This is a landing page from the at, which call out of 50% promotion for first time customers. Eso What are the key components to a leading page? Of course, a landing page is much longer and more informative than a banner ad would be simply because this just more space to elaborate all that. Thus we need Teoh conceive of the content off a landing page as, ah, birth the full and below the full. So above the fall sits all the content that a user can see. We don't have to scroll down while below. Duffel, obviously then refers to the rest of content that sits below, um, that the user would have to scroll down for And of course, common sense will tell us the most crucial informations and action you want a user to take should always sit above the full here. If we study farmers stop landing page, it seems like the team that a team their wishes to Kate wishes to cater to a true possible user. In 10 off course, this is purely my own deduction. I am not stating that this is the actual thought process behind them building their landing page design, but it is possible that they're catering to two possible use it in 10. The 1st 1 might be too big catering for users who would like to no more about the product offering and what they do. So this might cater to users who are very new to the brain, and it is your first encounter with the bread. The landing page, therefore, provides the chance for them to further explain the product offering and convince users to purchase. So and for it is, we see the information on the main Benecol out and the C T, a button that says Learn more, which doesnt lead users to engage or to content below duffel, which then elaborates further in great details about the offering and also offer up social proof off. You know, happy dot clients on instagram feet and so that will be to cater to user intent off wanting to learn more. Maybe a user is not yet ready to make a purchase. The second intent, I would say what before. Users who are already who already know enough about the brand and are about to make a purchase decision. And they just simply wish to claim the offer that they saw from the APP. So for Discover uses, we see there prominent white, but then on the top right hand page that call out D offer again and prompting people to claim that offer. So as we see here, it has a lot of content going on, and the pages crafted Teoh in ways to help users navigate those information. As we see here already, the information has to be ordered in priority and present it in an almost hierarchical order. Now that we understand how a landing page is structured, the types of thought process that goes through organizing is content. So again, let's look at the anatomy off landing page. So first up, of course, we have the main banner, as we really briefly spoken about. Um, of course it's it is meant to draw your focus to. It has to be eye catching and helps to visually relay the product proposition. So here we see that is, simply conveys a very delicious nutritious dot food with ingredients that you can see and that is the main value. Prop off farmers, Doc and Marketer is our on site managers or product team toe. Periodically change up this mean banners on landing page to find the most effective way of communication with audience and see what works best to lift conversions. And this Bennis space is not constrained to just in the just well. Companies have also explored, like short, looping videos to introduce some elements of movement in a static space. And typically, if you're using like a looping video, I would suggest the movements to be a little more slow motion and less, I guess, jarring on glaring to the eye. Next up, in the same area off the banner is where the headline and sub hit Lying Tip of You typically sits to call out to the key selling point or demean offer. So here we see a smarter, healthier pet food, and then we have the subject lines that further elaborates on it. And other times they may even be more for the elaboration on the key message that can be conveyed on the landing page as well. And we'll see, like maybe a short body of texts here. But most of the time it might not be above the full. It might be beautiful for people who sick for their explanations and further details. And finally we have the click through rate brother s or the C T A buttons. In this case, we have two of them. So this is all the essential pieces off a landing page, and each variable is off course open to testing and learning. And all sections of information off a page have to be well thought out as we really learn and organized according to priority and to, you know, we have to decide whether it should rest all buffed awful or below the full. And now, finally, we come to the most exciting part of today's class 4. Creative Optimization Process: ization. So after going through the anatomy's often at and a landing page, we have now some ideas off. What are some variables that we have to conduct a B testing with? And here are some key components that are most often tested for. I've done my best to be as comprehensive as possible with this, but the possibilities could be endless. Um, it And so I just listed out some of the different possible elements that, uh, most commonly thought off and tested with the each categories of variables. So let's go through them one by one image and our video. So, of course, in every banner there it will be the key emit to the center image nd or sometimes even the video. If the format allows such us on Facebook and Instagram and of course forties, we can play with the key message of the image or video. We can decide which products that we want a feature whether products are going to be display as a ghost product on its own or arrange into a flatly or to be model with. So those are their various ways off displaying and presenting the product in the image of video, and the other thing would be the background. Color is also an important element, and sometimes they're. People might choose to play around with patch to its surfaces as well, just to see if something that is more roar and captured my rings think better with people might be visually more captivating. So besides that, we have also off course props, a sonata element to the image and last see. We can also consider the filter or tones to the image. That's a trend. You know that there are trans that shift from time to time and seasonally so, for example, and fall perhaps like warmer hues and tones, it signify the evokes. A sense of coziness and warrant might resonate better with your audience. Next one, we have hit lines and subject lines. This is another big components that we have if you have a compelling offer, all selling point to why people should buy your product versus their competitors, this is where you would be elaborating on it. You can consider testing out different phrasing off the same offer once you have found an offer mechanic that works best for you or sometimes picture speaks 1000 words, and you might wish to test having no texutil on your betters. That's quit. Also be worth a test. And last but not least, definitely play around the fun off course within the limits of your brand guidelines to see what works best and how you can creatively make your attacks stand out better and be more kept evading. Um, next we have C t A and the button itself. So the city and message, as we learned before that can be a wide variety of C t A. We can use and we can see which one works best for. You are audience always tested out and the button style itself to it doesn't have to always be a rectangular. So you know you can test out. Maybe you should have a bought border Or, you know, if he has a border should be a thick border or a thin one. Should it look more three d you look gave like popping out. Should have rounded corners should be even know box at all And just be a simple underlying tax Or maybe an arrow off the signifies that you can click on it and yeah, you can play around all the different styles and off course, different colors. Some brands find that, you know, maybe having brighter colors might get people to click on. It better stands out more. But some brands, by feel like being on brand, would help them to create that that brand to to create a whole vibe or if the brand that appeals to their customers. Rufus. So all of that. And finally we have the landing piece we just covered. But from an at experience perspective before you jump into the optimization off existing landing page, we should that we should even start thinking through where we should be landing our users after they click on an ad. So always think about which pages makes the most sense and, you know, perhaps even like splitting traffic to land on Page eight vs Ph. Be from the same campaign that you have. So this is another A B test It people typically do that that's not involved, even optimizing the page itself. So maybe for us find out which is the right page swollen user on and then look into holding in onto the page and seeing how to optimize it. to perform even better, so less work through some examples up optimization process together to fully fleshed out this ideas that we just discuss. So now let me give you a disclaimer. This is not the actual optimization process that transpired with this at sets from the brand, they said. Just some add examples they have curated from the Internet that I feel could be used to illustrate some potential process optimization work. So again, this probably not how they were put together, but this just for illustrative purposes for me to explain the concept of optimization to you guys. So let's take a look at the first set of examples off Pacino's from ever lean. If so, looking at this to ads, what do you think that the variables being tested here you can post the video and take a look at it right? If we noticed the headline the C T. A. All being held constant, both display the same product Pacino, pans, and what is different here is that one is showing it on Model versus the other one is a close up, off the pants being worn. So now let's say the pants that is close to the close up. Shot off the pants emerges the winner in the A B test with the higher conversion rate. Then what can we next test on? Maybe you can possibly a again, and think about it. So maybe the next round that we could hear it on this banner to test for more insights and results might be to test maybe having to pants on its own as a product flat lay versus this close up, off the pants, off the pants being worn. So, yeah, that's typically how a B testing goes. Now let's take a look at another set up as to illustrate this process off at generations and how we build out a test pipeline. So, for example, your company just released and new a pair of rain boots collection. So in round one, you want to start off with a close the model pair shoes. Maybe you learn that a close up shot of a model part of work into your Linda's on the Chino pants from another category. You wonder if it works and translate into your shoe category as well. So you might start off with that. You know that black is usually the best seller, but for display at us a thought that maybe a brighter color option my offer a better click bait and stands out better on a wet environment when you when you're competing for two, uses attention. So you decided that you know you wanna feature a pair of millennial pink shoes along with the black one. So you conduct the first round of A B tasks, all right. And it turns out that all millennial pink wins today. And so for round Two, what shall we do in round two? You may introduce a new format to against the pit against the winner in round One, which is a mill annual pink shoe. So this time, maybe you want to find out if goals products go skews would work better. So people so meaning that the products appear on its own and not being worn a model with So instead off just having the pink and millennial pink shoes. You also want to introduce out of color variations to tell the users at this pair. This boots collection exist in all other colors as well, and so you decided to come out of a slightly different format on That's what it is in Be here, Um, and so that's it for around two. And now, let's say in round Three, it shows up that version these air in round two shows of the version B is the winner. So what would be our next step? So here, in Round Three, you might wanna like, um, carry out our again the You might want to carry out a single product test versus the multiple product tests in like in various color vagrants, which we just tested. And again there was the winner in the previous round. And so in this round, one might wonder test, maybe just having one pair of shoes and seeing how that would work against the winner in the previous round. So again we might wish to bring back the pig vary in for a test, since we know that millennial pink seems to resonate, and then so depending on what we learned again, we might reached a conclusion on the winning for meant for the image. So say that a remains the winner then we know that for around four onwards will focus on testing out other variables to boost this winning format so it could be changing up. The headline are the background color to see what else we can work better or even testing out the culture action message. And but in style and so like, you know, As you can see, the iterations and rallies can go on and on and on and each round getting smarter and smarter as you accumulate the learning from the previous round. And as you test out different variables, you should ideally see your conversion rate is going up and up. So that's it for, um, a B tests. And right now, after we sin how a process a B testing can look like, let's take a look at multi Vera testing. So multi. Very testing is often used in landing page testing, and this applies to at testing as well. So as we see in D a B testing pipeline example above. Testing and learning can take time because you're testing one variable each time. Um, yet often businesses, you know, have to reiterate quickly and learn quickly and move quickly. So therefore, a multi vera tasks can be a time saving way free to test and learn more quickly. Photo more Sometimes you also need to know the impact off a combination of variables together. You know the synergy that can be built when you have a new awesome headline with a pair of awesome CT A that sings together. You know you might give you that conversion lived and boost. And so we have multi vera test, but a So you can see in this example, it can very easily go into six versions. Read it, and just to. And so it could be a much more complex experiment to set up and to test for, um, so let's talk about some pros and cons. When you compare to multi vera testing vs A. B. Testing the pros obviously is that is time efficient, and you know you're able to measure the impact off combination effective on the better. The cons, however, is that it require a larger traffic and, in other words, at budget because in a B testing, your traffic is split 50 50 whereas in your M V T D combination at variations can add up very quickly. SS shown. So in the example above, if we're testing to headline versions and three content segments on an at an example, we will end up with six variations off ads, and therefore traffic will have to be split into six ways, and thus your overall website or at traffic will have to be larger in order to make sure that, upon splitting six ways that each free aviation will still have a significant amount of traffic for it to be since a statistically significant. So that is definitely something to consider. And the other thing would be that in the event that one variable has action, no measurable effect conversion goals, for example, changing your content from A B or C has no impact. Then you might as well have been better off working with an A B test off just hit Line one versus headline tooth. And you know, you might have being able to save some at budget as well. Um, yeah, so that's the two cons off multi vera tastic. And I know I've mentioned briefly about statistical significance. So let's jump literate into stats. And what are the things involved and what we should know off when we turned out a B task or a Multivariate test? So what is justice? Statistical significance. It is basically, in other words, how confident you are off the results. A standard acceptable statistical significance were ranged between 90 to 95%. And in order for you to be 95% confident off an experiment result, we will need to find out what is the minimum sample size of audience that your experiment should reach before we can't make that conclusion. So to do so, we typically use something called a sample size calculator, and I will provide the link in the Knicks slight. And before that, let's talk about what is a baseline conversion rate. So you know, when you're imagining doing an A B test, what are you essentially comparing your experiment with? You are comparing it with your baseline conversion rate so your basic conversion rate is the current performance level off your at or your marketing campaigns if nothing else changes. So you know your version A from previous rounds of iteration that we have seen might become your new baseline conversion rate. And so you wanna have something that performs, hopefully better than your version eight, which is your baseline conversion rate. So that's how every testing works and sometimes you also like to take an average off your last 30 to 90 days gold conversion rate and then compare your variance alongside with and see if there's any up lift in performance. The expression that next question it might be running in your hit might be. How long should I run a test? So it depends again on the sample size needed to reach statistical with chickens and how large your website traffic is or how big your at budget ISS again. The smaller your traffic and or two smaller your at budget is thin, the longer time your experiment will have to run in order to reach a target Alyan size for each Fabian's. So again, this being suspect to the concert off multi vera testing And that is that even, like take a longer time if you do not have a sizable traffic or a sizable at budget to run it so as I promise this is a sample size calculator, this is an example that I got from up to my sleeve. If you go to this website here, they actually provide quite a good, useful tool for you to input so you could just input. You're a bit your current baseline conversions. And what is the percentage of lift that you want to see? The difference that you want to see from your baseline conversions. And of course, you want to achieve your statistical significance of 95%. You want to be 95 close and confident that your results not a fluke, that it iss significant. And so with that optimized, Lee would then calculate the sample size that you need for each fair in in order to make that conclusion. Um, so here we see, we need 13,000 eyes in order to make a conclusion for it. Alright. And here I've included some additional resource is for some of you who you know, are eager to learn more and would like to gain even more advanced understanding. There is so much going on for a B testing for a multi very testing, especially that you know a lot deeper and I do not want to lose people. And the weeks on this class is for, you know, more beginner level. And so I've included some of this additional reading for you to go through at your own pace in on time. And if you're interested to know more 5. Creating a Test Plan + Class Project: Finally, we come to last component off today's class, and that is creating a test plan. So how am where should I start? You might wonder. So I would say that before we begin, let's think through a checklist of questions and based on the and says to those questions, we might then decide what is a good starting point for ourselves. So you know, if you want to do it, maybe task, maybe ask yourself first. Is this a new product or a new category or product releasing as we've gone through? In the example in Section three, it could be that you have just released a new product category of Rain Boots. And you know, if it's a new product category, then you might not have existing knowledge off what is the best practice and what might work best on the Benner. You might want to borrow some ideas from out of categories that have, um, I have work, but it may or may not translate over. If you are launching a new product within the existing category, then I think you might already have an idea off a winning at format, and then you can create new variations to it to test against it. And so again, it was a new category that, um, you have never done before that maybe it's better to go broad and task that testing from, like the format itself and layout and see what is the best layout for the image and format of the at that would work best for this new category of products. If it's a new product within an existing categories, like, for example, your Tino hands, you have sold many other pins. Then you know that maybe you can start off with a closer off the pans and you can walk around tweaking the headline, tweaking the SETI buttons and all the other little media greedy often at the next question , I would ask myself is also Is it time for me to revisit my offer? So, for example, I've been running ads for a while now. I don't exactly have a new product, a new category down releasing. I'm just simply updating on running my business as usual ass. Then, you know, ask herself, Have you been using the same value proposition or presenting the same selling point to your audience for a while now? and maybe you have started to see diminishing return. Is it time, then, to think of a new selling point to your audience or even find a catchy away to rephrase that selling point in order to, you know, provide a fresh eye to your audience again and recaptured attention? Finally, I will also ask myself, Is it time to give my F a fresh look for a new format? If I've been riding on the success of a certainly out, perform it for a while now, chances are the audience are you have already seen it for a while, and you know, this is what we call creative for teak, where your audience that seeing it over and over again and start ignoring it. So is it time for me to come up with something new? A new format? Maybe, um, something that's more creative? That would capture the audience attention again. And so if it's creating a new format, then again, you know we might have to go through another series off another CDs and process off a B testing and finding out a new winning formats. So let me be an idea might be like, you know, splitting your your image into half. Having one Teoh looping video all the other one is static image. Or, you know, if it's a purely aesthetic banners like maybe having one that is closed up. One there is, like flitter away having a model presenting the model's entire outfit. So your best fears ways and, um, to come up with a new layer and format. And I would also recommend looking into your competitive ads and seeing what new formats they have come up with. And, yeah, basically, that is how we typically it start off and not a testing plan. And, you know, once we have that first few ideas of what we exactly want to find out the hypothesis that we have, then it would then lead yourself into the next few rounds of innovations on his own. Like how in Section three in the example, we have shown how the boots would then Venus into new formats and new testing, and how each round basically iterated off the previous one. And you know, you gleaned the insight from previous rounds and introduced new elements that you have in new hypothesis that you have off the results so That's typically how we create a test plan . Well done. So we have come to the end of class for today. Well done. And thank you for joining me to learning about creative optimization for your class project . I would say Try to create a pipeline for your company or your clients New product lunch. So you know, you may assume that the winning formats from each you may assume the winning formats each operations like how we did in the example in Section three off today's class like we just assumed that the Ping Miletto show was the winner from round one. And if that is, that's the winner than what shall we do in Round Two? What is the new element that we're gonna be introducing and testing again? And then again in round two, who is the winner? And then what it would mean for around three. So go through that process and make those assumptions and, you know, see how you work with the insight from previous generation to plan for the next round. It orations try to do it up till five rounds, and you know, if you would like to, you may always put it down the common below and share your thoughts and ideas of your test pep line, and you know, if you will otherwise, if you would like to get in touch, feel free to get in touch with me on my website. I have a contact form there or even lived in message Me and you might even share your ideas were a test pipeline, and I would also be glad to share some of my thoughts with you based on that. And once again, thank you for looking with me. If you like to learn more and to find out more about other topics, please check out my other classes in this whole cities. In particular, you might want to take a look at the introductions to at platforms Part A, B and C, which Mike, if you a little bit more holistic understanding on at serving on those platforms and how what we learned today on creative optimization can go hand in hand with knowledge and played today in my view, everything. So with that, I would like to think everybody's time attention and I hope you have a good certainty. But by