Market Research - How to Test Your Idea in 2 Weeks (with real-world case studies)

1. 1. Introduction: Hi, everyone. My name is Brendan Wu and will come to the idea validation. Of course. Since 2010 I have Bean running a development studio where we work on ideas from a client supreme, a lot of their own sort of digital products to market. And what would found Waas. Oftentimes, these products don't actually getting attractions on the market. We've done lots of different disappointments, including social media, platforms, games, different platforms, mobile PC's and as well as contact products, books, magazines on some of these more technical factors such as beyond a our projects from And I often wonder what if we could tell the clans before we start before we start spending months and months of development on an idea to see if we can validate the idea somehow before we get started. On the other years, we've done a fee off projects, would some experiments with some validations, and in this course I heard to share some experience with the and hopefully we could reduce the amount of waste and cost off development. When it comes to developing new products. I think a lot of times people get started without doing enough validation. We just spent too long on the development. We found having a clear idea where the products carrying so hopefully discuss the score is what will help with that. 2. 1. Introduction - Intro: so this course has broken into four parts and each part, except for the first part, is going to have some assignments for you to dio nothing. Teoh complicate it. This courses end for beginners, people without an M B A degree or when he sophisticated technical skills. And also I probably tried to record the scores unscripted. I think it makes for a more natural style of communication, and it just feels a bit less stiff and hopefully more fun for you as well. 3. 1. Introduction - Overview: What is this course about? It's all about answering this one question. How do we know if an idea makes good business? Some people have lots of ideas, but I have a file name ideas, and I just keep adding to, and most of them are probably not very good ideas. They're kind of fun or exciting when I first came up with them, but some of them obviously bad that I feel to them out. But after you've done that, you probably still going to have a huge list of ideas that you don't know if they're good or not, and the best thing to do is to test them right? That's how you never even idea has any legs. So So this course is about testing these ideas some of the ways we could use to test them, and it will probably take you one or two weeks of where to do. Some of that is going to be a bit more hands off where you set something up. You wait for the results to come in, and the way this course is structured as I'll be talking to you through the slides, sort of learning style more traditional teaching style kind of video. And then I'll talk about some of the case studies and the things that were done in the past . And then we will have sections where I would like you to get and create some of these things where hopefully you have an idea that you could use to actually test throughout this course and the goal off the courses to successfully test one idea and then make a recommendation. 4. 1. Introduction - Who is Brandon Wu: So before we began, let me sort of introduce myself for those of you who don't within their My name is Brendan . We and I started my professional life in 2003. I joined E A Games company to work on This game called The Sims. Started with the Seems To and it was a fantastic experience. I met a lot of great people. A lot of creative people at the Games. Industry is always bean really exciting for May. I was there for a few years and then eventually decided that perhaps it's time for me. Teoh get an advanced degree. So I left. And when Teoh Business School at the University of Southern California. And after that, I joined Sony in the strategy division, where we were mostly the Connect Consumer Electron ICS and your Internet. How are these things can fit together, and that was quite exciting as well. Strategy work is something that I've always been interested in, always been curious about. So we did some acquisitions, wait, built a few services. We sold one, and that was great. But in think around 2000 and nine, three iPhone, three Gs came out. I think it was 2000 and nine, and I just I just love to think I just thought, This is fantastic, A new platform where you can have knee a mean experiences, gonna reach a lot more people. And I was also kind of I kind of wanted to go back to the development world. I kind of wanted to go back and make things. So I left to make games and abs. I said, In 2010 I'd left started my company called Studio Pepper, and since then has done various different projects for brands. A lot of family friendly stuff. One of the last projects were done was for thanks, help You created Captain Underpants. And then he had a new book being released. So we made a game to help promote The Big Were also did this project for Alicia Keys. She was releasing these kids interactive books way about with the Siris of books. I also ran a developer community in Seattle, the Unity User Group, and that was fantastic experience. I learned a lot from all the speakers and met a lot of good friends through that, and in 2015 I started this new company that was mostly working on our our own projects. So we've done a social media project and which which I will talk more about in a bed. And we also have bean experimenting with ER and there is a our ideas, so that's me in a nutshell. 5. 1. Introduction - Why: No. I have built a lot of things on Dwork down a lot of especially digital projects, products for clients, from my own starters ideas and and it's always sad for me to see projects fail now. Oftentimes they don't have any problem building them. We don't have any problem bringing them onto the market. And then we have a polished product that has good reviews, good feedback. But it doesn't always work out. And I always wish that we have done more validation and demand testing before we finished building over before we even start building them. The first reason why revalidated ideas is for us to make it go or no go decision, I said we know that, yes, other people also want this thing. Or now you know, when I was. But I was once does right, and secondly, we go. We approach this through the quantitative lens. I want to be able to say this is a good idea because this percentage of people who have seen the product ones or this is the percentage of people that will pay this amount of money for it. Right. So we're gonna quantify our test and then once we have these numbers, we can use it to build a business model, right to sort of star validating. Is this going to make any business sense? Or is this an idea that will like? But no one will pay for my so So by quantifying the's ideas in these tests, way can actually build models and and on and use those Teoh make recommendations. 6. 1. Introduction - How: So before we begin, I will tell you the process that will go through and it's broken down into three parts. We're going to first start to plan the tests, and during this face, we're going to define our assumptions of products our audience, and decide what kind of method we're going to use to test our ideas. And then we're gonna run our tests, and I'll go through how you set up your landing pages, your ads and then analytics and then finally reviewed the results. And we'll dive into the numbers that you've gathered throughout his test and then sort of figure out what all these numbers mean and how How can you use them Teoh to build a case. 7. 1. Introduction - Principles: before we start planning the test, I think there were a few principles that I want you Teoh, understand, and just remember these things while we're running these tests. First of all, speed test should be done quickly. You don't want to spend too much time worrying about various things like what we want is to run a short test for us to understand something, maybe to reality an assumption, or to learn something about this idea and then move on to the next test with next thing. So speed is important. Don't overthink it. Similarly, simplicity keep a simple test should be easy to implement. They should be easy to understand and easy to realize. Don't over engineer things because it's very easy, especially if you're a programmer or engineer who likes making things. And it's very easy to start tinkering with sayings right to start actually building something out, building more complex analytics tools said's or making a prototype of mark up more functional. But don't do that because we were. All we want to do here is testing the demand, right Focus. Keep your eyes on the ball. Remember that tests are going to be schooling away What I mean by that is you are going to create a markups you might be creating. Sum's websites Don't worry about keeping them. Don't worry about that. You know, dresses that you might collect. Don't worry about people's Commons or various things that you might get out of those outsiders. The test results right, because if you get to keep in websites or the, you know, addresses that collect great. But don't worry about to focus on the results from the focus on the results that can help you make a decision. That's what we're doing here and then working backwards, like working from the end. Result backwards to figure how we get there, right. The test should mimic how u think people going to find it and engage with your products and services as if they're really today, I said. I always think about how the product and service would look acids there, riel crying. Then that's how you contest this demand on. Finally, you want to keep it tough. Optimization is not go. Understanding off the market is what I mean by that is it's easy to sort of get into the mentality of okay when it optimized for my conversion rate or I'm gonna optimize my ads. But the thing is, don't don't do that just yet, right? Because we were No, you don't have to put a get you're trying to figure out what you're trying to refine the idea, figuring out the idea. And I rather if he keep the test tough so that it's harder to convert people. And this is when something really valuable stand now, if people have Teoh go through foods to get to the product, that probably means the product is worth so much more to them. So much more value Pope to that, then, if you make a really easy for them to cast a test, Okay, so that's the introduction, and next we will start planning. 8. 2. Plan: No. I have a better understanding of what the courses about and also why we want to validate our ideas. Next thing to do is to start planning for tests. So in this section I will go through a few things we want to define for your tests. A few case studies of things that I've done in the past that probably could have been planned better. And hopefully we'll get some learnings from those. And once we have test properly, planned out where you will go ahead and start running those tests. But in this section, we're only going to focus on the plan. 9. 2. Plan - Considerations: ideas calm, you know, shapes and sizes. So there are different ways we can test them, and different things would probably need to consider before we test them as well. Now, some of the ways we could test ideas, I thought I go through a list of them on There are other ways or contest ideas, but just for you to sort of think about your ideas and what it's some of the best ways to go about testing them, right? So the first thing you can do is look at a problem that you have in your life or your professional life. And you might come up with an idea to fix some of the problems that you count it right, scratching their own itch. There's not a lot of problem with those, except there are still things that you would want to test, even though you know there's a demand where there is a problem that needs to be solved. What will look at a couple case studies in this category? Right? So there's that. And then the second thing you could do is you can buy some ads and then just look at how many people click on the ads. You have different descriptions off an idea, and then you can look at how people behave, how people respond to these ads and there might be graphical there might be just text, but you can use that. That would be a really simple way to start testing some some ideas. Just looking at the ads and then next is that this is quite a a popular way to test ideas, which is landing page testing. And we will go into more detail on this particular type of testing because it's something that I've done quite a few times, and I find it to be very effective for you to understand potential future customers. And then there are things that for people who can actually build things fairly quickly and prototype things and have a way to reach out t potential audience, uh, you can start, just build something, build a prototype and then give it out to people. Have it, have them, player, have them try out this product. Giving you feedback on this is I think this is a white test ideas particularly useful if you are making on entertainment product, for example, making a video game, then the best way to test it is create a prototype, then give it to people and just see how they player, uh, there. It's really hard Teoh tests and idea off a game without having that sort of textile game control and actually feel the game and see how it place. So there are ideas that you kind of have to actually build something and and and then just give it to people. Now this leads to leads me to the next point, which is You can also do interviews with people. Interviews with customers can go find offline groups and then just talk to people. And this is again. This is a good way of testing some ideas. It's a bit more qualitative than quantitative. So you want this really have a lot of numbers that you could use Teoh to sort of culture business case with and also you sort of running to you. This problem of people might say something, but they might not actually do it in my say they like an idea, but they might not be willing to pay for it. And this we encounter this in a project that social media project that I would touch home later on where we did outside of the landing page testing we followed on with, Ah, a lot of interviews with social media users, which media power users influences. And, uh, we can't read a T back. But once we launch realized that there is a lot of other factors for these people to actually commit to an action I supposed to just telling you that they love the product or they love the idea. So So, yeah, so if you are, if that's the way you're going to test your ideas, definitely pay some attention to how the actual behavior might change depending on you know who you're talking to, you and what kind of part of it is. 10. 2. Plan - [Case Study] Markd: Mark Dougherty. This is a project where I wanted a tour for me to be able to bookmark people, organize people that I find online that I might potentially want Teoh collaborate with in the future a lot of programmers, artist, or sometimes your investors that I want to keep in touch with or I want to reach out to later on. Sir. Before I had built this tool, I had a massive Google. She could close Freshii and just different categories different, contacting, fair and their websites. And it was it was very difficult to manage. So I ask one of our engineers and see if we could build something just for internal use, started talking to people about it and eventually released a public version on When I say release that we just basically ball a domain name and and sort of hook up the bitten pieces Teoh to make it available for people who, if they've found their taking, use it. And it was it was sort of discovered by someone that we didn't know who this was, and this person submitted Teoh Credit Hunt, which is a platform for people to vote on the project of new ideas. This was all the Christmas holidays, and the one of our team members will come in and saw three checks part of hunt every day and saw that this thing was on the top five. The list of the day we're and then we're all really excited because we didn't win plan for this to happen. Andi. In fact, the thing was still quite buggy and lots of things that needed to be to be improved before before it's properly public phrasing. But it was it was discovered. We got a lot of traffic that day, and then the following week. In the weeks afterwards, lots of people signed up. Have you ever used it? Which is great, And then to me, that's a great validation off the idea, on top of our offer to solving solving my problem. But what was not tested was the scalability of marketing, right? So we got organic Trafigura people getting discovered, and that's all great. But we never tested how we're going to tackle marketing. How are we going to keep, uh, growing the use of Berries growing, growing the service? It's not so much of a problem because it's it's a tour that would go internally. This is some values every day to get it. If I were to run this as a proper sort of start up, then you know, we we would have had to continue to test market intense profitability, test conversion if before we want kept upgrade to paid version. It doesn't currently have a paid version, but if that's the case, you know what we need. Teoh test that So So this is sort of one problem with. If you're just solving your own problem and so confusing yourself as a validation method for the for the idea, then you're kind of overlooking some of the business problems or business issues that you probably need to test as well. So if you're solving your own problems, think about what other things that are related to the idea, you know, business voice that you should look into testing. 11. 2. Plan - [Case Study] Authorlicious: next case study that we want to look at as this thing called off of licious North. Alicia's was a WordPress theme that we built for authors, particularly self publishing office. I have a small publishing business, many just publishing books, my my own books and then and then my wife speaks. But throughout the years we met other authors. Other people were self publishing and started helping them with marketing and having a website, having what presence was always a key component of that. So we ended up building a lot of websites for people and realize that there's a lot of There are a lot of things that we're doing repeatedly and decided to build a repressed theme that has everyone integration, good read integration and various other things built teams for people to be able to quickly set up a website themselves. And instead of doing that landing page test, we should have had this thing where we basically reach out her network and spreading the ways about this upcoming product, and that people come into the website coming to the landing page and there was an email side of simple, too to be, if you wanna be updated about product and leave your email address here. And that was great because we I had a new initial list of people that we can reach out. He when? When we finally finished the product. Onda again. This is, you know, we were solving our own problems, so I thought, we've got a new idea that there would definitely work and we got a lot of people signing up . Lots of people willing t purchased the product when when we launched. So there was a great you mess it all money into building this product. Hire a team in Seattle to go, Did we launched on? We have good sales. Ah, lot of a lot of themselves from the U notices we've collected, but we had failed to test to test marketing and profitability off the product or, as you can see from from the leading period. So the image at the top here, that's the lending page that we had, and there's no mentioning off the cost of the themed because we were getting tested. So we don't know what's the best price point for for this product. And also we did not test different marketing channels we had used all network and relied on word of mouth. But that's great for the initial launch period. But after a year or two eventually starts to slow down. And it was difficult to make the decision between continuing development and updating the theme versus how much more? How many more potential sales coming, expected the next month and the foreign months on going forward. So again, this is a case where I had relied on my own intuition. But always remember to Fella date and test other aspects off the business related to the idea, because if you do want long term success, these things need to be tested. 12. 2. Plan - Product Definition: crazy Now coming back to you your ideas. The first thing we want to do is we want to write down the description off the idea so that you can use that description in many pairs or in your ads or you various different places where you minding Teoh. Tell people about your idea and think about how you can use it to tender into test and also how you can make it concise and easy to understand. Right? So you want to use plan, language, something that short and simple and something that people can understand. Try to avoid using dragons unless, as you are targeting a very specific market that understands that turns that you're using. But remember to keep it really concise, really simple. Often time from do these tests people don't have alone attention span and very quickly within 23 seconds to decide to leave or stay. Focus on the benefits that you are bringing Teoh customers, not what the product can do about what God can do for for users. I said on this is this isn't that you called me here 1,000,000 times from any sort of marketing courses or marketing advice, but really focusing on the benefits began. You are always thinking about themselves and how your product and benefit, then very suitable for looking at a product. I'm thinking, No, how does This is a really cool technology, without knowing how it all of it and use the technology for themselves, and then you want to keep it realistic. You don't want to promise something that you cannot build within a reasonable timeframe. Right again, we're testing something that you might potentially want to build. So you don't want to be overly confident and promise something that you can actually deliver because that sort of defeats the purpose of the test. You might have a really good result, but it's not really the product that you're going to be able to deliver, so to make sure that it's realistic, you understand your product. You kind of now how long something would take to build. And if you need to provide a timeline for delivery, definitely keep that in mind. The definition and prescriptions you can't various different versions of it, but definitely have sort of a master version and that different versions that changes a few sentences and wordings depending on where the poppy needs to go on to. So this is a good exercise, not just for the test, but also for yourself to understand what the idea really is about and what the benefits are , two people who might use it. 13. 2. Plan - FAQs: It's often nice to have some effort is ready to go is, Rale said. These are questions that people might have when they read about your idea, and it's good to have these answers ready to go. So you put these on the website, the landing page, or when you talk to people about your ideas. You are prepared to answer these questions, I think, especially if your idea is a little bit rescue for the customer to to use your product or service. For example, if you are doing something that's connected to President finance or if you're providing a platform for for lawyers or if your product or service has ineptitude, was things that people might have more complex thinking behind the purchase. Particularly true, If it's a B two B idea, then it's good to have these questions answered. You eliminate their doubts as quickly as possible. 14. 2. Plan - Target Audience and Customers: you want to spend some time thinking about for yourself to with a lot of ideas, you probably have a pretty good idea of who is going to use your product, but it's important to make a distinction between the people that uses your product and then the people will pay you. I said, for example, with social media thing. And there is a group of people that our audience these are, uses people who used the app. But they are not the ones. Pain is directly right, because our business model was to sell the traffic to advertisers. So then they brands or restaurants or shops they pairs to display. Some put promotional materials in the APP, right? So there's a difference between people who uses the AB. And there are people who pay the company that builds. Yeah, so that that's difference between audience and buyers and buyers up your actual customers will pay you three provinces Onda. That's very different from something like a an app that people pay for war and ab that I've been free, but our purchases that the actual users pay for you to get that money right, so make sure that you understand difference. And also you see you are building media type of business that you might have to run multiple tests. Wanted to understand the demand from the audience side can get users and then a few more to understand the buyers. Customers who are the advertisers interested enough in the kind of audience that you are going to bring in? Secondly, you want to think about scalability and profitability and how different they can be, right? So, for example, if your test it's about audience building, can I build an audience for my app from media business? Then you want to think about scalability, how quickly or how cheaply cannot potentially get a lot of uses. You probably not toured about profitability when you do your tests on your audience, but when it comes to your customers, you definitely want to think about what if I get your 1,000,000 monthly users, How much money can I get for that? How popular it is for business? And that's it's a different kind of test, different kind of validation that you need to understand when you think about the different types of users and customers of these at the kind of questions you might want to think about, particularly because they the results from these tests are ones that you were going to use of. The journal Building of business models and without thinking of theory, might get very different number that plugs into the wrong part of the equation. Also, think about the different types of customers you might be selling to. Your idea might be Abie to be idea, which will require different types of communication that might need to find out who is the final. So the decision make every gun actually may make the purchasing order. Figuring out different organization structures and that sort of things that might, you might need to take those into consideration. If it's, say, B two c idea. It might be a bit more straightforward but still need to define your demographics again. Entertainment partners, utility products, educational, part of a lot of different types of products all required a different kind of thinking, and this will help you communicate with the target audience. This defines a tone, the sort of voice and you won't have, and what's the best way of finding them 15. 2. Plan - Marketing Channels: after we have our target audience defined, we want to think about where we can find them. For this test in particular, there's one key thing to keep in mind, which is you want to think about where you will find them once your product or service is ready. All right, so you have to imagine that your product is ready and it's on the market, and these are the places where you think your customers are going to come from. We want to use thes channels for the test, so its representative off what might happen in the future. Okay, so the first thing right off the bat. We don't want to use friends and family for your test asking friends and family more often than not, they might give you feedback that would have leads you down the wrong path. But you can talk to your family and friends about about your ideas, but don't use that as part of your validation. I use a lot of ads for my tests and find them to be probably the easiest way to get some traffic. Tiu my tests. The causes is fairly low. Probably about 102 $100 or less. It's a BTC product. Facebook, instagram, Google AdWords, thes air, all popular. And I recommend just going with these because it's not really about optimizing the performance of the as, but to get something going quick, right? So we're not optimizing. We're trying to get results from these tests. Pinterest and Twitter can also begin, depending on again your audience. What kind of community you're you're trying to reach If it's be to be linked in ads, Cora adds. These are Filic adoptions on. Guy would say Facebook and Google AdWords are still pretty adoptions. If you're going to be to be type of my tears and then they are from communities and forums that you can, you can try as well. I haven't done these these myself, actually, product hunts. You had that kind of submission, but we haven't done anything sort of manually. They have some new photos that helps you test your ideas. We haven't really try. There's either, but feel free to. If there is a try read. It is also required a popular destination for a lot of different interests. If your idea is targeting people off with specific, just copy that Maybe that could be a good close to two place mats. You can also leverage your existing platforms if you have a huge YouTube following order. If he needs Twitter following to try to get people todo test and see how you have it. But again, the problem is they sort of already know you. So there is some buyers in these kind of tests. You can definitely use them, but you know that these might be a little bit buyers. Finally, there are offline groups. Me does even bright a few places where you can find an event. I go and talk to people and get some insights into other people's problems on. That might be helpful for for you to to really find your ideas. But like I mentioned before, I know that what people say, my nap, your body actually do show. So whenever you do qualitative research, you just have to keep that in mind. 16. 2. Plan - Result Planning: I always recommend deciding how you're going to read results before you run the test. Once you have the results coming in, once you have the numbers, it's very tempting to start reading it in a way that gives you to go ahead for the idea. It is very easy to have that buyers. It's always better if you decide that. Okay, the install rate needs to be 25%. Otherwise we need to run more tests or you want to test the profitability. So you might say, for every $50 I spent on ads and I want to generate 200% of that amount, right, So having some sort of pre defined rules or guidelines to interpret your results can help you down the line. Once you have them, it will make your life easier goes so that you're not trying to bend the results. Two year old buyers 17. 2. Plan - Assignments: Okay. So I think now is a good time for us to get started with your test. By doing this very first assignment, I want you to write down a description for your idea, be it a product idea or service idea. Have a good description that you can use a a website or something that you can use to summarize in in an ad. So have this description ready? House wants you to define who your talking. Who are the people that you want, Teoh? Reach anti. Who are the people who will find your offering valuable and then decide which platforms you're going to find them on? Are you going to use Facebook? Are you going to use Google ads? Think about what's the best place to find these people. And then how are you going to find them on those platforms and finally define how you're going to use the results of your test to make a decision on the idea? I define it as something that you would use ones. The results come in once you have all these things finished, I would like to invite you to show them on forum. Obviously, if your idea is quite sensitive. Then you share what you can. And now that my sound really basic that some of these things and you probably have them ready to care while you were, you were coming up with the ideas, but it helps to write them down and have them ready for our next step. 18. 3. Run: Okay, now the fun part. We're actually going to run these tests. As we've discussed, there are many ways to run a test, and here we are going to focus on running a lending test, which I think it's a very first a title tour on Dwyane that could be used for a lot of different times of light years. It's also one that requires a bit more to set up and run compared to you. Some of the other ones just play testing or prototype testing or solving your own problems , so I think it's worth going through it here. 19. 3. Run - Creating a Mockup: Before we start creating a landing page, it's good to have a markup of your product. It's basically a visual representation off your idea, and oftentimes it's maybe a screenshot of an app or a website. Or it could be a three D rendering off a physical product that you are going to produce. Some of this is gonna be tricky. It's great, but it was a little bit of help from perhaps a graphic designer or maybe using something like Canada or photo shop. You can create something that looks like a final product and images you can use on your under landing page. One thing to keep in mind as this is a mock up and not a prototype a prototype. It's functioning. So the prototype has features a prototype works as his intended T. Um, aka Does not A markup simply conveys the idea off the product, but not it doesn't actually work, so people can actually use it. Another thing, Teoh. Keep in mind of a support. I can look really simple and plan and kind of ugly, a markup you wanted to be polished. You wanted to look like a good part of you wanted to look like the final thing that you are going to deliver. So that's something to keep in mind when you agreed in your mock up. 20. 3. Run - [Case Study] Visual Stories: So here's a quick example of a mock up of an idea that I had on. The idea was that people can create visual stories, and it's sort of a story chained kind of game where you would follow up with the next bit of the story with a picture and some tax. And then you continue writing the story together with with the community. It looked like a final product. It's got this sort of IOS design aesthetics, and I just made this in. I think I did this in Adobe X'd, which is a nice design tool, but you can create this camp opinion, power points or chinos or anything that allows you to put images and tax together and lay the marry in whether you want. 21. 3. Run - [Demo] Create a Quick Mockup: So this is mark upper dot net. It's a free tool. First thing I want to do is I want to pick a background for my markup, and I will use a an iPad as my marca. So imagine if this is an idea, a nap for for tablets in particular, and then you can add up late an image or enter and you're out. If I enter my website here, it's going to go to the site and then take a screenshot of the website and use that Teoh as the markup image for for the iPad. Next, you can crop scale your image. Once you're done, then that's pretty much you have a markup of something that looks like it's it's on the device, their various different settings that can do. You can download different sizes and, um, it's great, it's free and it works really well. 22. 3. Run - Creating a Landing Page: next thing I want to do is to create a landing page for your test. Have used to froze lending page a few times now and haven't really defined it. A landing page is a website that is very simple. It's made one to your three prejudice. Large ties. It's created solely for the purpose of marketing or advertising. Or in our case, we're only going to use the Langham edge for the idea. Testing. Oftentimes you have various information about your product or service on this landing page , so that includes descriptions. FAA cues, as we've mentioned earlier, definitely your mock up images. And sometimes it's helpful to have some information about year your team or wherever. It's behind these ideas. And if you have a little story about how you came up with the idea and anything else that you think will help sell the idea, you can include it on the landing page. One of the most important things about the landing page is the call to action. Oftentimes is button. Sometimes it's collecting email addresses 23. 3. Run - Call to Action: you include a call to action on the landing page so that you can mention how engaged people I have interested. They are most straightforward Way to do this is by including a button and this. This could be a button that says, Download or install the APP. If this is the mobile APP idea or it could be a purchase button, and if it's a purchase button, you could say by now I would recommend you use another page after the by now. But to increase the difficulty of purchasing. This is done so that you have a better measure of how how many people are actually going to go through the entire purchasing process, because buying something online takes a few more steps than that. Gammell. The piece of software that been said. You can also use email sign ups, which requires a little bit more work on the end users, part where they have to type in any more tries and and click on sign up right so it's a little more engaged and then a single click. So you can also use that when you use different types of call interactions that's going to have an impact on your conversion. So it's influences how you would reach a result. Really think about how you want to structure 1/4 action. And oftentimes, if I can't decide, I just have different versions of lending Page one with the button, the other one with with an email signer on. We'll see an example in a bet, often times when I when I have friends with with these kind of things, they would want to continue building amid email list on based on this landing page. So which I think in a way, is it's kind of a nice thing to have if you don't as in afterwards, it's a result was kid, and you want to go ahead with the idea. You already have some people that you can reach out to once the product is ready. But I also feel like you don't need unnecessary Want this to be a platform where you build up a 1,000,000. This because that sort of moves to focus away from the test into building um illness, which is a very different type of marketing activity. You'll start to optimizing the page for that, and it might have a negative factor on how you're during test several. Now, I'm not entirely sure you want to focus on that. I think it's a nice tohave if he happen to collect a few knows, but don't let it become a priority. 24. 3. Run - What Makes a Good Landing Page: what makes a good landing page. I think this is pretty straightforward. One. You want easily readable and concise product descriptions so that peak when people visit on your side within a few seconds, they understand where you're trying to sell them, what the idea is about. I don't want it to be over complicated. Second, we want polished microbes. Visual communication. We're expected, then something that just has tax on it. Eso spent time on your mark herbs or, if you have to hire a graphic designer to do it and finally have clear culture actions. So if it's a downloadable app, you make sure that that's clear. Teoh visitors that can click on download easily big buttons. Make sure that it's visible and testing different platforms, right? So if you if you are making a lengthy bridge that's responsive, make sure you test on the various different devices more with devices that you want to run tests on. And if you have any adult in questions, feel free to create different versions of it. You can run test are different tests on different versions, and often times you could tell you different things about your idea what people respond to better. What part of your particle more interesting to people. So it's always nice to have more information that you can you can act on, and having different versions of the landing page gives you some insight into how people actually respond to your idea. 25. 3. Run - [Demo] Setting Up a Landing Page Part 1: There are a lot of different platforms and tools you can use to build your landing page. A lot of them are free, and I've listed a few of them here. Include these in the resource was slightly at the very end. This well, but so these are some of the ones have used. The nice thing about the 1st 2 male Munch and mail chimp is that they are directly connected to Melt Trip, which is a female service, but you can use it to manage your illness. It's were integrated into that so we can sort of start collecting email addresses, and they don't have any other steps. You can dio if what you're trying to do is collecting emails, because if you're a bit more technical from familiar with HTML, you can do that and you can host your landing page. Get huh? That's free. You can use a tight form to embed a form that people can enter some person informations as part of the test. So this creepy. If you're asking questions or if you are again doing human collection, you can use type form as well. From its this very well designed. It can be nicely integrated into a blank page. Some other options, if you use wordpress dot com or CFO said, were put site or a tumbler block. The key thing is not to think of it as a website that you are going to grow its basically just a page or two that you want to use to do the test. So don't go crazy with with the site. Andi. There's another tour that I recently found, which is called London Lending Deco. And it's free and it's really easy to serve. You don't need any technical skills. You don't even need any design skills. There's no tracking tropical moving around things and sort of just gives you some options, and you can create any nights. Look inside very quickly. So here I've created a quick one as an example. Next I want to do is just to show you quickly how we can create a landing page. Using London again created lending, bridging a lot of other different tools. And if you're interested in learning more about any specific one, we could go over those Maybe in a separate video 26. 3. Run - [Demo] Setting Up a Landing Page Part 2: So after you go to land in that K, you sign up, you locking, and this is what you see this sort of your dashboard so I'll just quickly create a new site on the idea is going to be. It's a physical product Next, and then you can choose different sections during included on the side. So have pricing have a sort of the hero image of the top. Maybe some effort cues in a contact for him. And then you choose the style. I'll say this playful product. Maybe it's a toy. Maybe it's a game, your board, game, something. And then next we have sort of design page where it's very simple. You just keep pressing on refresh until you see something you're happy with. Can I go with this sort of red in color? And then you just click on star anything. Now you have a language. Right now I think it's the everything is sort of placeholder text. So you go to this bit so that the first button here is changing a designer. The design and color and fades. I think I'm pretty happy design, Goto added content, and I just kind of this say great and include on image that this is the image that would show up one. When you purse this pain when you pursue this landing page website, which was quickly it see if I have anything can use so well and then there. Let's go. Can I use a number? But here I'll just use text and have separate links at the top. I don't really need this, Merriman. No. So people, when people click on that, then they'll send you after me. That's fine. And then you can start customizing this. So each section that's it gives you some options, you to customize it. So that's a pretty straightforward, pretty simple. We'll come back to this after we talk about the next bit, which is setting up analytics. 27. 3. Run - Setting Up Tracking: before we finish our landing page. Want to make sure that we set up tracking for our culture action buttons? And also we wanted to know how many people have visited our sucked. So they're a few tours that even used most of these air free used grew analytics to track your site again. I won't go into how to understand that. But if you use some of these lonely place creation tours there, oftentimes there's an option for you to integrate Google analytics quickly. Men in those use nail trim for type form. Again, something simple is fine. You don't you don't need anything to come complex. And I think Melcher allows up to 2000 email addresses in a free plan, so that's quite a good option. My favorite way to track for for these kind of test is to set up a bit. Lee thinks so, basically, is a service that allows you to make to turn a really long you're out into a really short one, and you've probably seen this when you use treaty or are various different services for God will use you team they oftentimes the share where you go to share and then copy your L. They will shorten the link, but it's something similar to that. But on top of shortening links, it provides a nice way for to track how the links been used. I'll show you how to do that in a bit. One quick turn and then you could use a Z. You can use the question mark key word to create new links for the same euro. So if you typing to send your out to bid Lee, they're only created one short link for you, and this works fine. But if you have different versions off the test that you want to run, you want a have different tracking set up. You wanna have different sure links available for you to to copy and using them for you to see the performance of each one. And a quick way to create multiple short lines for the same euro is simply by adding a question more at the end of your your out. And then you can just happen some sort of key word for you to remember what it's your hour is used for. And finally, I want to just quickly talk about custom domains. I'm off the opinion that you don't need custom domains at this stage because you're merely testing the idea and costing demands. You'll have to pay for the domain. And oftentimes, if you want to use custom domains on, that's a wordpress dot com, and you have to pay for wordpress dot com before we continue accustomed Dough Man is name for your website. So, for example, did you might want to test an idea of direct cars? Tory's car toys? So you want Red Car? Tory's dot com? That's your domain. I am suggesting that don't don't go and buy the domain. Don't going instead of tinker with that just yet because you don't know if you want to launch this product, and it's costly, and also it's sort of distracts you from from the test in a while. You may be a little bit too commencing now that you own this domain for a year. Don't worry about the domain until after the test and also oftentimes will visit the size. They're not paying attention to the euro anyway, so it doesn't really make a difference. Um, for the most part, unless if your product is, it requires a high level of trust, and you do not have your custom domain makes your product will look less professional. If you think your audience or the people you're a testing on really needs to see that, then I think that's the only time I was such as having it. But other than that, I think most of the time you can skip it. 28. 3. Run - [Demo] Setting Up Bit.ly Tracking: Okay, here is my Bentley dashboard. So after you log into bitterly, you have the stash board and you can see that mine for different lengths that I'm testing have different And the text that I can look at. For example, this one. I was testing some core ads for my for my G Matt Burke. And I see the performance. So I did it along with testing, and I could see the performers off the ads and then I could use that Teoh in combination with some members an affiliate Data can use that to work out if the ads are working right so very quickly That's create a bit. Do you think so? You click on create and then you just typing your you're out. I'm just gonna use, uh, my present website, right. And then now I'm gonna and to question more and announce a test. And this isn't I just two more course click on create. Now, I've created a bit of ink on this link up here, and I can customize it. I won't bother with that at the moment. Can add tax so you can organize your links better. So I'll say this is our days. So that's the tag and it's safe incidents. And now I have this link that I can use to track how many times a button has been clicked. So I'm gonna copy that. Go back to my landing page that I was creating earlier. Now is there are two buttons on this page. I think I'll cover it of or one of them. I just need one and you get started. That's fine. And a link here. I'm going to pace that bit swimming. So now whenever someone clicks on that, it takes them to my website, which is Brendan were OK, and then it also tracks that click. And then I was sure it. They're all said. Sure, yeah, tell you. Total clicks. So So that's that. And pretty straightforward. Pretty, pretty simple. Two years now, I'll quickly publish this page so we can see how the button the Culture Action Button works . Had the analytics work. Let's go to Page set up and here you can change to sub domain off this parish, and I know it doesn't really matter. Progess. Yes. Developed nation cores test published carriage. Now this purchase life here idea validation. Course Testa and then care. Okay? And if I hover over this button, you can see that the it's using a bit the link. And if I click on it goes to my website with, uh, the sort of key word that I use afterwards. And I have a feeling that it won't. This won't be real time. Let's see. I really loved it. Actually, it is real time. Shall I got one click today on Tuesday at 11. 57. Great. So this is a quick way for each to track the usage of your call to Coach Action button. Sometimes it's click on. Obviously, you want to match the timeline with your ads. So you know this sort of when you test your button, you wanna make sure that that click is not counted towards your test results? Great. That's quite a fella. Nice looking. Lending her just so happy with that 29. 3. Run - [Quick] Setting Up Google Analytics on Landen.co: and just very quickly on the Settings Page here on the page set of page on London, she's grow down there. Various other and analytics tours that you can integrate said, If you want to use Google Analytics is just copying your tracking. I d there and then you're done. You can start using it. So that's a choir, a nice nice tour to everything sort of integrated easily. 30. 3. Run - Before We Run Ads: Okay, now we have our landing page ready to go. It's time to start driving traffic to landing page. And the most straightforward way to do that and this is something that I usually dio is simply to buy some ads and then start sending traffic through these and platforms. And nice thing about that is, you can define the demographics to find interest and definer geographic locations and language, setting them platforms and a lot about other different things that you can set so that you know that when you get these traffics they're not. They're not random traffic. These are people that you are going to target for your final product Now, before we start a few things to to give your mind one. You want to be responsive. Pay attention to your ads while it's running, because it's very easy to think that you're gonna set it and that have run for a week. But you might have made a mistake in the settings, and it might end up costing you a lot more than than you thought. So definitely pay attention to your budget. Monitor your ass consistently, at least on the databases, particularly because ad platforms nowadays often charge on a per action basis so often times per clip you pay for each time someone clicks on the ads, or each time someone does something on your side, and this probably takes a bit more to set up. I don't know how technical you are, but either way it can be costly. So definitely keep it on your your campaigns. Monitor your budget and especially the first day or two. Keep checking it. I've made mistakes before and and it's costing much more than than I wanted to admit. Definite. Keep an eye on it, especially if you notice that you're getting traffic from a country or region that you don't necessarily want to test on. Because what the ad platforms often does Aziz. They will send you traffic where it's cheapest to serve or where it's most responsive, right? So, for example, I have this test preparation book, and I sell it primarily in the States. But when I did my testing, I was getting a lot of traffic from India from Southeast Asia, where the book isn't actually available. So that was something that I had to quickly change so that I'm not buying ads from those areas because that changes the result. That sort of scared the result one way or the other. And then finally, keep a note to make sure that everything happens. You write down the time they if you are seeing spikes, definitely make a note of that and try to analyze those later on. If you can find out why you're seeing a spike, it's good to have some of these notes ready. 31. 3. Run - Setting Up Ads: Now, when it comes to sitting up ads, you want to keep it simple. Use the description that you've created. Find something images that you can use. It's always better to show people in the ad always performs better as humans. We like to see other humans. You can show phase that. That's great. Keep it simple again. Don't worry about optimizing the ads and that you can spend ages optimizing your ads. And then that doesn't really tell you much about about your your demand for your product. So don't worry about optimizing them. Make sure that you pay attention to demographics, the device platforms, language, regional settings and these sort of basic things that you want to get right, and particularly with device platforms. Perhaps you want to set up separate landing pages for this because people behave differently, are mobile than on laughed hub for certain types of ideas and products that you probably only want to test on the PC right. For example, if you're creating a creative tool that people will most likely be using the best up, then you want to turn off mobile traffic review. That is because sometimes they're their own by default search revealed eyes. Otherwise, you're getting you're paying for traffic that that's not relevant. And also you're going to get much lower conversion rate because our mobile use is not going to care about their creative tour. That only works on desktop. Here is a screenshot of Facebook ads that ran before, slightly complicated when you first look at it. But all you need to know is, Are we getting impressions? If you're not getting impressions, adjust your bid, how much money you're spending and patent into the time you running a one week tens. Where are you running a two week test? I would always recommend running at least a week because the behaviour during the week and on weekends are oftentimes very different. So you want to make sure that you capture capture those. I definitely feel free. Teoh. Ask questions in forums or reach out to me, and I'll try to get you through setting up the ads. My rule of thumb when it comes to bidding for clicks is about 50 cents per click, depending on the idea. Depending on what your trying to sell, this number will go up and down. Personally, I find that to be a good starting point. And once you start by Nazi, your public want to adjust according to how many impressions you're getting every day. Obviously, if you want more, you kind of need to increase the bed because you're competing with other ad buyers for these traffic. So but just accordingly. But I always sort of start from 50 cents. The ad platforms are going to give us adjusted bit. Take those into considerations as well, but I find that they're not always accurate after after you started running in the numbers don't always match. 32. 3. Run - [Case Study] PrettyPost: Here is a quick example of an ad campaign that I ran for offer. This idea, called Pretty Post Pretty post was this idea of a blah greeting AB where the Apted's the content of blog's and then light out the content nicely designed away with some editorial designed so that you can consume it kind of like magazine. It's a nice, nicer experience, then endlessly schooling Syrian article so that that was the idea. So these are the two examples of as that we ran. One had no mock ups, just a woman reading on the firm sort of generic stock further and the the other one, the one on the right we used for clear messaging. Where you now this is an app. You kind of know what the AB looks like, what the main benefits are and same copies, Berries ads. And the interesting thing here is So we ran these ads on Instagram. I think he's a grand was the main one that we used simply because we thought Instagram has a more design focused audience, and that's the audience that we thought would like the idea. But what we found was, it's not a lot of uptake on the idea and also way tested two versions, one with email sign and the other one with a simple installed button. And we found roast. The email signer version actually converted better than they install the one with the quick install button, which slightly slightly surprising. But at the end of the day, both are really low numbers, so we did not go forward with the idea. 33. 3. Run - Assignments: Okay, now it's a Simon time. Yes, And now I want you to start actually run your ads and created landing page. Have Markham's off your product and start tracking these performances. Eso creator Mark of career Lenny Perry said, attacking set of ads And then, more importantly, make sure you discuss, ask questions, discuss with anyone else that's taking the cores. One thing to be a little bit more careful about if you are going to show your landing page . Probably do that after the test. Or have you some of those tricks that we talked about before to set up separate lending separate euros so that you can accurately measure your conversion rate. You don't want people visiting your site from the course or you want friends and family to be counted as part of your attest something todo half. But yeah, so go ahead, create any page and then discussing for him 34. 4. Review: we'll come back to the course the next step. Next section old discourse and this is the final section of the course is reviewing the test results. Now I'm assuming that you have follow the steps in the previous videos, and then you have said of your ads, and you have let it run full a good amount of time. You've gotten some results, gotten 100 visits or 200 visits, and now you're ready to take into the numbers and see what this all means for your idea. It's a great time for us to dig into our delicious data. 35. 4. Review - Business Modelling: Now that you have your conversion data, you can start looking at the industry average and see what you're observing from your test . So if you are being, say, retail market, what is Thea Varejao purchasing rate for from someone seen a product to some of purchasing one? And how is your conversion rate performing when compared to other retail products on the market, and you might not be able to find out. You can check on Cora or can check on various different websites to see if you can find any industry numbers. And I think the key is, Are you at least performing ADM average? Maybe this is OK, but it will be re lot more compelling if you're doing twice that if you're doing five times their or even 10 times that. So benchmark you were conversion rate. Another thing that you want to keep in mind if it's what's your cost per acquisition compared that to your lifetime value of customers? And this is kind of hard to know without alone study because you're only doing this Richard amount times we don't know what the lifetime value of a customer is and that lifetime value of customers means how much revenue you can expect from each customer. But you can potentially use what you've gathered to make some forecasting and say, Well, if this person is making this purchase ones today and this is a season of product, maybe we can reasonably assume that you know that he or she will will continue to purchase every every few months. And then you can sort of extract like your data that way. So you have to make some assumptions when you read your result in order to model your business using these results. Now here is a very simple mathematical scenario. For just just as an example to to get your head around how you can use the data to try to construct a business model. I show that, say, if you spend $100 on ads, your 50 visitors and three purchases have 35 dollars each. Rights of Mary, its rightful wig. You can conquer your profit, and that's $5. He should further compare the conversion rate. 50 visits, three purchases. Is that a good number? Is that a good ratio of purchases compared to what other people were doing? I'm on your industry and also is a $5 profit on $100 spent. Is that a good return on your investment? So some of the things to think about before we started taking into the numbers and looking at some examples. 36. 4. Review - Ad vs Landing Page Conversions: after you run the test, you mind noticed that you have two very different conversion rates. One is the click through rate of your ads. Let's say you have 10,000 impressions and you have a 5% of clicks. 5% of people who have seen the app click on the APP eso that's your conversion rates from the at and then secondly, you have your landing page conversion rate, which is that's 100 people actually visited side. And then two people have taken actions, right? So then then you have the 2% landing page conversion rate. Now, the way I think about these two different numbers, these two different conversion rates is that your ads are testing to see if there are enough people with the same sort of problems. So this kind of depends on the copier using for your ads. But typically in the address, your sort of describe the problem and the number of people click on the ads are representative off the people, the number of people with the ratios, people who are experiencing the same problem, right? So, out of 10,000 people, 500 people feel like they have this problem that they want t resolve and they're looking for a solution. So that's sort of a general assumption that I made, which is ads after versions are telling me that. Okay, Is there enough people with this problem that are seeking solutions and the lightning perch convergence that tells you that if your specific idea is actually solving this problem for people are there believing that this thing that you're going to create, it's going to solve these problems for them? And this is more specific to your idea and less about the problems. And obviously these are sort of general, and it really depends on the landing page that you created and the ass you've created. But if you're having some trouble, I understand your difference between these two numbers and you aren't quite sure how to. You have to make sense of them. This is sort of a nice way Teoh to try to understand, or the different numbers mean 37. 4. Review - [Case Study] 30 Day GMAT Success: So here we're gonna look at a case study of this is actually the first landing page test that I've ever done. And it's about this book that I was thinking about writing the long story short guy. I was going Teoh Masters degree at Carnegie Mellon and very last minute decided that I wanted to go to business school, and everyone's only had a month before the last day off applications for scores that I wanted to apply to. So I didn't have a lot of time. I came up with a plan to study for the G Matt, which is to test that you need to take to apply for a business schools. Fortunately, I got a good result from the test, and I have been thinking about writing a book throughout business school. And I was tutoring at the time and, you know, sort of teaching people how to do better on the test and after business school. While while I was working at Sony after work, I sort of wanted to do something for myself and still had this idea that maybe I could still write this book before I commit Teoh, spending a lot of time and energy right in this burg. I wanted to know if there is any real demand for it. And what kind of profitability? A book. My brain Obviously writing a book is a lot of words. Publishing a book is a lot of work, So I I wanted to get a better I get for a start. Ondas You could see this test was done in 2000 and eight. So this is 10 years ago and I decided to run the test in a few different ways. Package the material different needs Some landing pages will have a bundled kind of steal. Something would were not tested. Different price points. So the whole thing took a few months. As you can see, I started the very first house in January and the last one was in a pro. So I was doing this just on the side after work. And as you can see, that different price point was producing different profitability. But for the most part it was generating a good profit and I could see that. OK, there are people that are willing to spend upwards $200 for for for this kind of studying material I decided. Well, okay, so it looks like there is something here. And after those, I feel a lot more comfortable, more confident in my ability to actually sell this book. And also, I knew that I can at least rely on Google AdWords from a marketing efforts because from this test I can see that it is working. So I spent the next year and 1/2 of writing the book maybe a bit longer than that, and eventually published. And the book actually did fairly well. When I left Sony to start my own companies, the berg was what was funding my living expenses when we first started the companies that would have an income, and unfortunately, the Berg was selling where enough to cover it, the majority off our living expenses. And eventually I sold the mentoring right. Teoh Business Weekly Publications and the Chinese version of the book can be found in bookstores in Asia. Solve these Asia, and I think the highlight for me was one time I was in Seattle. I went to a Japanese supermarket and they had a bookstore attached to the supermarket, and they have a tiny mentoring section and actually found like Berg in Seattle in the state and being able to see my Berg in a physical form in a bookstore and that was that was quite incredible. So, yeah, anyways, eso I still published a big and went on to become something that publisher ended up picking up. Um, I could talk more about self publishing, but that's probably a story for another day. 38. 4. Review - [Case Study] 30 Day GMAT Success Test Spreadsheet: let me go through this spreadsheet with the just quickly so that you know how how this can work give you mind that it's going to look different for for every product. But if you're looking at the retail product than this is probably what you are going to be looking at, so the top I have price point. I tested different price points. Ah, included some free bonus material is shipping and adding everything up to a total. So this is the number that visitors see when they go through the purchase page. I concede through various, very different price point. And then I have my timeline. One of I ran the tests, how many people visited the site and from I wanted to compare that to how many clicks I'm getting. So I got a better idea which number I had my 12 years, and there is information from the takes, and you kind of have to just pick one is gonna think about conversion, and they're not always going to be consistent, but there's still be similar. Next, you are going to be looking at sales. I used this thing called Jumped Foreign, which is a a way for you to integrate. Ah, form into a website and I got three people feeling all too form fitting out their name. I think I had the Carriedo maybe address key thing to remember. Don't ask people for their credit card information. That's really go. But something basic. President fair, you know, treasure thing. Phone number. Sounds like that. See, people fill out the form here, so I consider that an order on four people fraud form at the 100 the price point. Consider that for orders. So that's the sales part. And then here I have my cost cost to run by AdWords campaigns The actual cost. I also included shipping costs because I I was thinking all I need to ship these books to people. I need to include those in my calculation. And I was included credit card charges and various different costs that I think I will probably incur on top of the advertising cost. Anyways, these are all simple, simple math. You just need to put them all together. And then I came to the the Prophet have off the stiff test. If I can sell three at this point at this price point, that's profitability. And obviously the most profitable price point was $100. It was not the final price that I went with I after after release, I tested with a few different price points as well. And also the other hand, I didn't end up creating bonus materials and get national career kid. I just endeavor with a break so that that's the especially that I used to to make a decision pretty straightforward, and I'm sure you can also just adopt or create your own versions of it. 39. 4. Review - The Unknowns: There are things that are kind of out of your control and really can't really test them during a limited time. There are still a lot that needs to be considered and also in things that are outside of your knowledge. So you always kind of have to keep another mine, that there are unknowns that you just you just call. There's no way for you to to to forecast and some of these things could be seething, healthy, nicer, different months, my giving different results with day versions, weekends, certain ideas of definitely more season now in time sensitive another's, for example, with a g. MacBook I and I didn't notice. There's why I was doing the tests. Even their test ran for from Generator April. After the book was launched, I noticed that certain time of the year, but their arm or sales off Guilmette burgs in general, you can observe this on an industry level, and then it sort of slow standing the summer, and then the picks up a little later in the year, and then in the spring, when people sort of starting to crunch for for the test, any particular events that's going on the world. Why are you doing your test? Any big events happen during your test that could have impacted your result details. So these kind of tests are always on approximation and sometimes can lack details so you won't think about. Are there important details that you might have mess that could change? The result on dis is sometimes hard Teoh to do after the test, because afterwards, you don't really want to think about what might have gone wrong. But it's important to review your tests and see how I can adjust your tests for these details. For these adjustments, you need to make for your next test, depending on the industry or in there are industry specific on know about, maybe change in how the technology is going to wear changes in purchasing behavior or how people communicate. These are companies, and you kind of need to keep an eye on, especially if your product it is going to take some time to develop, and I'll explain further next 40. 4. Review - Things to Keep in Mind: Sometimes an idea is very complex and might require additional customer actions instead of just having one landing page and then a few buttons to the next page in the next page, my requirement, convinced before people would take actual actions. So my advice to you would be, if that's the case for your idea, gonna continue running the tests at different levels. The man testing is not an exact science, and it's more of an art. And I think this is the case for a lot of social science in the way where you really wanted to remember that when you're dealing with human behaviors is not always exact, so there are subtle nuances that you might have to consider. And finally, I think this is a new important one, which is that consumer behaviors can change over time. So you really want to think ahead, especially if your idea requires a long product development time. So well, the market changed in two years time. While you develop your product right? Let's say today you get really great conversion, right? Everyone loves the idea of one wants to have it right away, but will that continue to be the case two years from now. If that's how long it's going to take for you to develop this product, trying to guard against this changes behavior and industry and technology by continuously testing even after you've made the decision to go ahead with the idea, I think you just need to keep in mind that continue to talk to your customers, Continue to test behaviors even without a real product. I think that's important to continue to understand the market and how it might be changing . Timing is everything is your industry one that changes rapidly. If that's the case, then perhaps you need to act quickly. Maybe you need to test a version of your idea. That's a lot quickly to build, maybe doesn't offer everything that you wanted to offer and then test that idea and see if that something that you can start with 41. 4. Review - [Case Study] GetFooto: So I want to show you another case study. And this theme was called Fiorito at the time and eventually the product was called Darko. We had this idea what happens if we take all the pictures of food that people were taking? And we can analyze the data and create a new social network face around food and locations and venues Sharps. But before we start building the app, we wanted to get just a rough idea. If there is any interest in this kind of after a specific and its very location focused social media. So we created a landing page. Ah, a longer version of it and several short versions of it. And the longer version. We also had a yellow button version and a an email sign up version. So multiple different versions. We wanted to just understand what people are most interested in about this idea and let me actually show you the landing because I managed to find this on my my computer. Right, So this is they landing page. So we created simple Markov's. So you kind of look, it looks like without the app even there, we haven't really And, um, some key selling points. And at the end we have Donald Button, and these are, as you can see from the very bottom of my screen, these a bit Lee lynx eyes. I'm tracking these installs and you click on them. They just go to the APP store. It doesn't because there's no after you download. All we needed was just to know that how many people have clicked on these buttons? So that's the long version, and this is the same version, but was a une email sign up at the end for here. We used type for four for the email set up, but this one has told the content Israel of the benefits or the markups everything. Then we've created 123455 different versions, short versions. Now this one. I wanted to see what what the demand is like without the markka. Is there enough interest without actually seeing the app or people that interested in the idea that they would go in trying to find out more? All right, so we have again different bit the links to to track different versions of the test off the landing page and then the other four versions are basically the top bit where it gives you an overview. And then one of this one just gives you an overview. And then the other three gifts he one section off the app, right? This is all about sharing of food voters. The next version has doesn't talk about anything else but discovering Fareed around you. And then finally this one. We're testing me how important it is to How important is the community angle off? Yeah, right. It doesn't have anything else, just just a community point. So we tested all these different versions and gotten some interesting results. So from the various different landing pages that we've used to test, we found that two full landing page converted three times better than the short ones. The conversion between decimal button and the email sign ups, a similar committed better on IOS, then enjoyed because there was going to be a free app. So we didn't get to test the profitability side of things. And another thing that we didn't get to test was how, actually, people will respond to our design because at that point we haven't actually designed that product which has had some mark of some ideas of how it would look. But there was no UX design yet 42. 4. Review - [Case Study] GetFooto => Doko: So we did end up creating this product. It took us a while. I think. I think it took us at least a year, if not a year and 1/2 to get the first version out and product was pretty good. I think we had a lot of march for what people wanted out of it. Thea was featured by Adobe, Great reviews and feedback from creatives. However, because we didn't continue to do test him, I sort of got sucked into development, married, really excited about making it a reality, going, going to design and just making it ourselves, who didn't do enough testing in terms of retention in terms of usability. And what we found was we had a lot of Ilya fans, but it takes them a long time to create you content. And that became kind of ah problem area for the app. So we are reworking, trying to figure out what we should do next forward a platform. In the meantime, social media has move onto video a short form videos in particular in the past couple of years. So this goes back to what we discussed earlier, where you kind of have to think ahead and think about where your industry might be moving. Are you going to be able to produce your products in time to capture, to realize your test? On the upside? What we tested all the results of the marketing. Spano, the conversion rates. They were pretty much spot on. Once we released, we utilize the same advertising platforms and we were seeing very similar kind of conversion rates, which I think we did pretty well in terms of pain marketing. I think that was what I was working. I won, you know, we sort of had an idea how that would work out from the test, and then it turned out to be pretty much spot on. 43. 4. Review - Assumptions: at the end of the day. Demand testing and idea validation really is about testing your assumptions. Even during reviewing these results, you might have Teoh make some assumptions as to how people are reacting to the landing page . It's hard, Teoh have a perfect test condition, and we talked about some of the potential unknowns on Noah bows that you might have to consider. Finally, continue to test all of the different assumptions around your idea around business beard, marketing assumptions, demand assumptions or behavior assumptions or design assumptions. You want to keep testing as you go along, even after you've made the decision to go forward with the idea. The more you contest, the more information you can together. It gives you more room to make the right decision. So that's something that, even for me, I need to remind myself to continue doing that because it's too easy to start going into the building phase and then forget about things that you publish. It also do on a prison site 44. 4. Review - Assignments: your final assignment for the course. I want you to Ellis your results. What does your test tell you about your idea? I want you to build a business model based on the result. It doesn't have to be complicated. Put the numbers down, put on a spreadsheet and just let it tell you what you can expect if you actually long should product. And after you've done that, make a recommendation. Is that on idea you want to go forward with? Is it an idea that maybe needs some tree king? Do you need to do more tests? And if so, how would you change your future tests? And even if you make a decision to go ahead with it, what's the next thing you you need to test? Once you've done that, you can go back to the planning stage and then continue this exercise throughout development. Once you've done these things, I love for you to share your test results and discuss with their other students off the quarters. I'll try to come in common as well 45. 4. Review - Resources and Best of Luck!: These are some of the tools that I've mentioned during the course. I will make it the slides available for download this round. Use the tools a lot of them are free on. There are a lot of other tours, and you can use as well so urine on them to to to the set of tours. But these are the ones that have used and the ones that I found really useful. Good luck with your ideas. I am very grateful that you've taken the course. But if you need to find me there a few ways here, in my email address, my Twitter and my website, I would love to see what you have come up with. Somebody has testes. Well, hopefully this has been a useful coins for you. Thank you.