Product Management: User Research

1. Introduction: Hi, Ron. Welcome to the introductory video of this class. This is Shreya Jen. I work in the product team at Phonepa. Phonepa is India's one of the leading Fintech brands that does over 350 million transactions per day and has about 500 million registered users. Prior to this, I have worked across many BTB SAS firms as a platform product manager and has about ten plus years of working experience. I started my career as a data scientist, and I have a PTEch computer science degree from Liplan so in this class, we will learn how to craft user research objectives, which can vary from creating a product strategy to just ideating on a product or idea or a feature idea. We'll also learn how to conduct different b2b and b2c user research frameworks or techniques. The key takeaways from this class would be like we discussed the objectives of user research, which can vary from creating a very fine product strategy or doing a competitive study or trying to improve the user experience. Second, is different steps to conduct user research. How do you define your target audience? How do you form a clear crisp problem statement, the concepts and frameworks you would apply for that specific problem statements and lastly, how to capture the insights. Different b2c and B to B research techniques which include qualitative research, quantitative research, competitor study, secondary research, and so on. Lastly, we'll end up with doing some competitive benchmarking exercise. In this class, along with discussing the concepts and frameworks, we will also go through real life examples so that you can apply that in your day to day life as a product manager. This class ends with a really cool class project that will cover how you will develop a better user interface for the Ride hailing app Uber to appeal to newer generations specific pain points. So this new generation is more digital and tech savvy, and the user interface needs to resonate with their specific problem areas. So let's get started. See you in the first lesson. Thank you. 2. Objectives of user research/feedback: Hi, everyone. Welcome to the new lecture of objectives of user research and consumer feedback. So there can be different types of objectives, and your use case might be one of them or might be one of the granular objectives of these bigger categories. So we will go through this one by one, discuss some use cases as well. So first objective or first type of objective can be just identifying customer needs, right? So in this case, you might not have a very clear cut problem statement when you are conducting user research, but it is to figure out what could be the potential misses or gaps within your product, if you don't have much tribal knowledge, even if you don't have much tribal knowledge about the product. So there can be cases where, you know, imagine it's a food delivery app, and you are trying to figure out cases in one of the let's a payments leg. That means the end user has selected some food order and they have now reached to payments leg. And in this payments leg, you are trying to figure out what you can do in order to make the customer experience even better. Now, you could lead to or this research can lead to, you know, missing the gap or missing some of payment instruments or payment options. It could also be that the people are not able to trust, you know, fitting in their payment details on your app. It could also be the UI or the layout is not very friendly for them to be able to actually tighten those details. Another example, out of payments could be, let's say, when they're trying to figure out what food to order, they're not able to discover or explore as much. So when an end user goes to a food ordering app, the use case or the thought process they might have in mind could be just, you know, I feel like having healthy food today or having chunk today. So with that use case in mind, how should your product listing be, how your menus on restaurants should be listed? Should it be category first? For example, I want healthy options, so I list all the restaurants within the healthy category or should be restaurant first, right? Now, in this case, you are not start with a particular problem statement, but you're trying to gauge what our potential gaps could be. Right? Second is validating your product ideas. Now, whenever you go on to build big product features that requires significant engineering bandwidth, design bandwidth, and product bandwidth. So this is a risk that you are taking against certain impact that you're projecting, right? So it's always beneficial to reduce your risk and increase your upside to validate your product ideas before you actually go and implement right? Even validating product ideas can be done in various steps, starting with just validating the idea as a whole. And once you are set on an idea, the proposal or the solution you have to fulfill that idea can also be validated in many steps in the journey. For example, if you want to, let's say, introduce a big feature of adding customer support in your product, once you have validated that it's a valid use case, post that how you want the UI to look like, how much of automation you want the bot to do. All those things can also be validated before you actually go and build that feature. Third is improving user experience. Now, let's take an example of two very different kind of products. One is, let's say, there is an app to book concert tickets, right? Now, this app should be or is meant to be more quirky or more user friendly because you want users to get a look and feel of the concert or of the other venues you know, other features that you have. You want the user to get a good look and feel, and then they end up, you know, booking tickets for that concert. So in this case, the visuals, the designs, the whole experience matters a lot for the user to be able to use your app effectively, right? Versus, let's say, a payments app or a stock broking app where people are trying to buy stocks or sell stocks. So that needs to be taken in a very different light because your user persona is different, as well as your user needs are also very different. So this is actually trading of real money. So you would want to be very sensitive about that, right? What needs to be shown and what does not need to be shown kind of disclaimers, the kind of regulatory concerns that you have, right? Or the problems or the bottlenecks that the user might feel might be very different from, let's say, a ticket booking app, right? So this is to enhance the overall user experience of the product, which needs to be viewed in very different lights, even though both the products are consumer facing, right? A fourth furness building product strategy. In this case, your end users, your user research is not just limited to end users. It includes involving stakeholders as well, gauging feedback from them. Even your co founders might come into the picture because this is a very important leg that needs to be condensed before you actually go on to build any features, right? Unless I have my product vision or product strategy, right. Convincing everyone to build features that align to the strategy is going to be very difficult. So this is a good place to get user research or feedback from many different areas and domains as possible so that when you shape up your product roadmap, you have taken everything. You have taken all the viewpoints into consideration and mindfully taken some calls of, you know, we are not going to move in this direction, although there might be some need, but this is the call that we have taken because of this in this region. The fifth one is customer satisfaction. Now, it's not only flows or experience within your product, but what's happening outside and around of the product also matters a lot. For example, let's say the customer support of your product is not really great. Now, customer support might not be needed to do one transaction on your product. Let's say a food delivery app. The user comes to the app, books or orders a meal and then goes off. They might encounter some issue, and that's why they reach out to customer support. So if customer support is not dealing with them right, it again, eventually is going to impact your product metrics or your business metrics, right? So it's important to take a look around when you are doing user research as well, and not just to be just focused on just one problem statement. Another example could be in a B to B poll that let's say you have a B to B software that does payroll, right? Now, for a product person, it may seem like the only important details would be when a user lands onto the product and how do they go about, you know, creating payroll options, right? But it is also important from a business point of view. When you sell this product to a client, how seamless is the integration process? Because they might also be using very different softwares within their company. How easy is to include your software in their ecosystem? How easy it is to explain other stakeholders what you're doing with your product, right? So a lot of things around the product also becomes important from a business point of view. So in this lecture, we saw the objectives, different objectives of user research and why it is important to conduct those user research. Thank you. 3. Steps to conducting user research: Hi, one. Welcome to the new lecture on steps to conduct user research. Now, the given list is a generic set to conduct a user research. The main category of points that you need to keep in mind before you actually go and interview users. Starting with defining your clear research goals. Like we discussed in the last lecture, having an objective to your research is really important so that everything, the kind of questions that you ask, the kind of results that you interpret goes back to your objective of the problem statement. So clearly articulate the research objectives. Write if it needs to be written. If your research objective is not that concise, then also you should write that objectively. For example, if your resource is just to figure out the existing caps, then also the objective of whoever is conducting those research should have that problem statement in mind. And the objective should also align with your product, what your broadest product strategy. For example, the objective of your product in the same space as your competitor might be very different because your product strategies are different. Second, is identifying your target audience. Now, firstly, figuring out your user personas. Figuring out the distribution within your user personas is very critical. For an example, let's take two competitors in the same space, A and B. They might be doing the exact same a thing or exact same task, but the user personas are very different. If the user personas are very different, then the product features that you want to build is going to be also very different. For example, let's take a payment app that helps you make payment to a merchant, right? Now, let's say if app A is primarily used by Gens or people of younger generation, then there um their problems or user needs might be very different. They might want more authentic, more user friendly, you know, a more flowing app, right? Versus your app B, which is primary that's used by millennials or people with, you know, different demographics, who might need a very different set of or might have very different set of expectations in the form of a more trust. So millennials or older people will not trust an app so easily or a digital product so easily that they would want to spend huge amounts on. So you will have to work your way around making that product or brand more trusting rather than more user friendly. So that's why user persona matters a lot before you conduct user research. Now, let's say for the same app, you have different types of user persona where Jen si Millennials, everyone is using that app. So then you have to take a very conscious call and see the distribution of different user personas using your app. So it may so happen, let's say, 70% of your audience belong to one category. In that case, do you want to optimize the experience more for them or you want to keep the experience more generic to everyone? You want to introduce a learning curve to all your user personas so that everyone has a rich experience. So those sort of decisions need to be taken. And for that, the identification of user persona becomes really important. Third is choosing a research technique. Now, depending on which industry your product belongs to, whether it's a b2b product, b2c product, even with a b2c, the kind domain it belongs to, your research technique can vary. So in a typical B to B type of setting, where your clients are not that huge a number as compared to a consumer product, right? So your clients might be different companies or SMB companies, right? So if you conduct, user or you have a platform to capture this data, that might not be the or technique. In this case, you might want to go through this extensive process of conducting user interviews to actually see what the gaps within your product are, right? So just capturing of data might not be enough for a B to B product, right. Might have to gauge how they're navigating across the product. Are they spending too much time on one feature because they're not able to understand it or they find the feature to be of prime importance, hence they're trying to figure it out. So in a B to B world, there are more nuances to figure out, and hence you might have to go through that interview in person. Second technique can be surveys. So if you're confident that the scale of the end consumers using your product is really high, and you're confident that, you know, people might end up filling the survey, then it's a really good technique because in a survey, you get both subjective and mathematical quantifiable information. In a survey, you can make people understand or people write subjective answers, as well as you can put out some quantifiable question like an NPS score or how easy is it for you to navigate my app from a scale of zero to five. Right? So if you have a good enough scale, if your objectives are clearly defined, then surveys can also prove to be really beneficial. Third type is data platforms, which is applicable to a more b2c settings, which is actually instrumenting data funnels and calculating the drop offs in each part of the funnel. Example, someone is using let's say, a Ride hailing app, they log onto the app, they figure out they enter their destination and they see the options and then how much time it takes for them to actually go and click on finding that cab, right? And once they've found the cap, how the payment leg drop off funnel is looking like? Fourth type is secondary research. Again, secondary research techniques can vary from different objectives. But in this category, what we want to highlight is that depending on your objective, depending on your industry, your technique needs to vary. Fourth is once you have figured the technique, how do you, you know, figure a base framework or the set of questions that need to be asked to the end user, right? Now this is a very critical exercise that needs to be done because you don't expect your end users to be always telling you the truth. So there is a really good book called Mom Test that explains you that, you know, that basically gives you an understanding that if you want to, let's say, validate a new product idea, and if you ask your mom about it, then she's always going to, you know, lie to you because she wants the best for you. So in this whole process, the objective is solely missed. You won't be able to capture whether your fact, whether your idea is good enough or not. Similarly, when you conduct user research with your end consumers, they might want to appear more knowledgeable. Hence, they might be telling you things which they really don't feel the need to, right? Or they might be highly critical of your product, right? They might say that they will buy a product or they want to buy a product, but in real terms, they might not actually end up buying those it's important to get around this whole picture, get to the point where you are able to ask the right questions or mold your questions in such a way or mold your interview in such a way that gets the most truthful information out of them by not necessarily probing them in a way that gets, you know, most reflects outcome or most reflects response out of them. The last one is capturing those insights or clating these insights. Now, depending on your user technique, the capturing of insights can vary a lot. So let's say if you are doing a user interview face to face, just audio interview. In that case, you might want to use an AI interpreter where it is correlating all your summary, right? So you get to address the main points. If you are using if you are doing an interview where you are actually observing the end user navigate your product, in that case, you want to let's say introduce a heat map where you can actually later on see that where did the user stop where they are taking the most amount of time. Uh, capturing insights for a third type of category where you are instrumenting some data funnel. In that case, you might want to get this data in an Excel or in a database and figure out various cuts to capture your insights, right? So these were some of the, you know, generous steps that you need to address before you actually go on to conduct a user interview. Thank you. 4. B2B & B2C: User research techniques: Hi, Ron. Welcome to the new lecture of detailed view on user research techniques. In this lecture, we will discuss different types of user techniques and some examples of where to use these techniques specifically for. First one is qualitative research. In this first one is conducting in depth interviews with customers to capture very detailed form of insights. Now, these interviews can be very free flowing in nature, where, you know, first, you are making the user comfortable on, you know, a getting to small talks and making user comfortable for them to respond to you in the most truthful way possible. So the important part is that it's going to these user interviews are very long in nature, typically 45 minutes 1 hour. And the idea is to gain maximum amount of information without having them to react to questions rather than respond to questions. Um, so a typical example of these interviews could be in a B to B world. You're trying to capture, let's say, the user problems from the end user, like a developer, right? So in this case, you would want to, you know, get to their comfort zone where you get to know what their daily, you know, daily job is. How do they start their jobs? What kind of things do they do? When do they actually go and use your product? What is the need of your product, right? So it's important to first gauge the user persona. Get them comfortable, ask them about their daily routine, and then go on to, you know, ask your questions in the most indirect way possible. In this, you have a longer time period as well so that the interviews can be used more effectively to get unbiased feedback. Okay. Second is shadow interviews. So a lot of BTB companies where sales people conduct very frequent meetings with their potential clients. This is a good way to capture what the end users or buyers really need. So in these type of settings, the primary objective of sales folks will be to get clients to buy their product, right? So client is already in a setting where they will be most truthful because now they're going to spend money on buying your product, right? So this is an indirect way to just gauge what their real, real problems are, which is preventing them to buy your product, right? Or even if they end up buying the product, what they would want out of your product in the long term. Second category is quantitative research. Like we discussed, surveys is one of the most effective ways to conduct structured surveys, get quantifiable mathematical data, as well as be very subjective about your options. Surveys also act as a good exercise because it can be incremental in nature. You can publish surveys from time to time and it is also involves minimal cost or spending as well. So if problem statement can be solved through service, then it's one of the best techniques to use. Second is usage analytics. Then you analyze your product usage to identify different areas of improvement, right. So whether it's in the form of a daily report or a data funnel that you're observing or you figure where majority of the drop offs are happening on your platform. So a typical BTC product like, say, a dating app where people are browsing who to match with and if they end up matching, are they going ahead and initiating conversation with them? Or they are matching with a lot of people, but the conversations are not happening, or it can be other way around they are not getting a match, and hence the conversations are not happening. So this is a very important aspect to understand where you can actually focus on the problem statement and then go about devising solutions. AB testing is another really important technique if the scale of your users is good enough. So any big features or any contrasting ideas that you want to get validated, AB testing helps a lot in deciding if the majority of the users are preferring this over that, or even for smaller nuances where it's very difficult to get user feedback, a very good way is just to do AB experiments. So let's say if I want to use a certain font, or a certain component or a layout. Whether A works or B works, it is actually getting tested in real time from the end consumers that what's really working for them. Because these cases can be very subjective or creative in nature. There is no way to say that this font works better from, you know, expertise point of view, but it can get really easy and really mathematical if you just conduct AB experiments on these type of solutions. Third is competitive analysis. Now, in comparative analysis, first form is benchmarking where you are trying to gauge whether your product as compared to competitors in your space to figure out to figure out your respective strengths and weaknesses. So in this case, you are comparing your product with your competitors to identify whether, you know, is there a certain aspect in your product that's lacking where competitors are doing a little bit better. So the case in the example that we saw where the app is tickets to book tickets. In this case, if your competitors is being really creative in that process and your product is not, and if that's helping the end consumers really gain trust with that app, then this is something that you would want to inculcate, right? Or other types can be if it's a hailing app. And if you see that, let's say, showing the amount of time that will be needed to book your ride is really important for the end consumers to know. And your competitors are doing it slightly better, then you would want to also optimize for that. So here, the actual change required or the engineering efforts required is not much, but it's important to make a decision whether this certain feature is important or not, right, which can be done easily with your competitor benchmarking. Second type of competitor analysis is feature analysis. Now, this is without taking your product into consideration, right? It is just to analyze what trends are working in the market, to analyze how the government policies are working, to take a glance at what can be potential long term bets that you can take, right? For example, let's take the case of dating apps again. Now there is an increasing trend of, let's say, olas within communities or people are not marrying, right? In cases of Japan, South Korea, there's a lot of set of people who are not, you know, following the traditional trends that our older generations did. So now Bumble is also coming up with a feature called a BFF feature where you are just hanging out with people or making friends or hanging out in a group with a common focus like a book reading club or, you know, going out club to a bird watching club or going out in a nature sort of a setting, right? So now, these are the trends that need to be analyzed long before you actually figure that this sort of category of product is viable for your business or not. Eight Fourth form is secondary research. Secondary research is just analyzing industry reports, a lot of VCs, a lot of future looking private equities actually publish these reports. These reports are also bifurcated as per countries. So for example, economy of US or US is going to see more trends of, let's say, people moving to digital versus a country, let's say in India. They're also adopting Internet or digital products on a daily basis, that trend is also growing, right? When you see trend across countries, you can also take a future looking approach to your product roadmap. So it's not just analyzing what your user needs right now, but what will your user need five years from now? Second type is customer feedback, where you get a lot of useful information from different channels in the form of let's say customer tickets or reviews or let's say, social media standing of your product. Let's say, YouTube is one of the most, best rated apps in the world of about 4.7. So there you can actually see that if there are some issues in the product, reviews can be a good source to capture that feedback. Customer tickets in the form for B to B products. If your customers saying that, for this particular use case, I'm just not able to get a solution from this product or from the support team, then it makes sense for you to actually go through the customer support tickets, then conduct user search techniques to actually figure out what the problem is. The last one is usability testing. So again, this requires for you to actually observe people using your product live in real time, right? So this is, again, a good way when you do not know exactly what your problem statement should be. But you're actually just viewing users in real time without no barrier, no constraints and you are seeing actually users using your product. So in this case, you might end up finding a lot of issues or a lot of strengths even when you see user using your product. You can actually form different patterns that, if most of your users were able to reach to, let's say, generating a report in your B to B product use case. So you can definitely say that, creating a report in my B to B product is something that the users are able to understand. But navigating to the right point is taking more time, right? So you are able to actually capture patterns without having to put a very concise or clear problem state. Right. So this lecture in this lecture, we discussed different user techniques cutting across b2b, b2c domains, as well as different types of industries. Thank you.