Episode 19
· 30:28
If you're looking for a step by step guide for validating your product idea, this show is for you. Jason Evanesh from Kissmetrics is back for part two. First, let me tell you about our friends at sprint.ly. Sprint.ly simplifies the agile development process by making it easy to see what's being worked on and what's done. I'd like you and your team to try sprint.ly free for thirty days.
Speaker 1:No credit card required. Go to www.sprint.ly. Now let's get back to part two with Jason Evanesh.
Speaker 2:Well, is a good time to go through the actual customer development process you use at Kissmetrics. So can you just take us through what it would look like step by step from the beginning straight through to the end?
Speaker 3:Sure. Sure. So we start out with the signal that I already talked about. Like, I am constantly trying to get signal from the sales team. There's one guy in the sales team who helps with integrations that literally pretty much once a month sends out a summary of everything he's learned.
Speaker 3:And he sends out to myself and to Heaton and to all the other C level leaders on the team. And we basically have that as a top level port forwards me any email that they think is relevant to product development, and I read through those, and I read through feedback. And all that's informing me of, like, what's going on in addition to what I'm learning in all my customer development interviews and what we're seeing happening in the And from that information, Keith and I will sit down, sometimes we'll sit down with our director of engineering as well, and we'll talk about product direction. And that product direction comes, know, basically comes down to the three of us, two or three of us talking about what we think makes sense, what fits into our roadmap, what fits into the skills of our engineers that we have available for projects coming up. And we'll basically zero in on like, alright, here's the two or three things we wanna work on next.
Speaker 3:From there, we'll start doing product research directly on just that feature. And so those will be when we'll do things like those surveys I told you about. We'll do a person search in Gizmetrics to see what who's using that feature or related features. And I'll email a bunch of those people and say, hey, will you hop on the phone with me for like twenty minutes to talk about it? And I'll do like seven to 10 customer development interviews about that specific feature and what we're going to build.
Speaker 3:And from there, usually by seven to ten, it's just like usability testing where after a while you hear the same problems from everyone and you're like, oh, gotta fix that. I'm tired of hearing that. And that's when I know to stop. And now we're ready for me to write that thesis I talked about, which is a collection of, why are we working on it right now, which will be all the data driven information, you know, any statistics I can show on it, and and kind of the high level product strategy for why we're doing it. And then it'll also have all the use cases and problems and opportunities that we can solve with this feature.
Speaker 3:And then that's when we kind of bring in design and engineering to get involved and understand, like, hey, here's the thesis, here's what we're thinking about working on. And I'll usually do like a first round mockup. Like I literally have graph paper and colored pencils, and I just draw what I'm thinking is a possibility. And then we have, we actually read this awesome blog post about what it's really like to work with Steve Jobs. And so we've integrated this thing called cauldrons into the process where basically I'll have this first mock up and I'll usually have like a dozen questions of like, well, I know I could have gone this way or this way.
Speaker 3:What do you guys think? This is what I did with it. And you basically just like all ideas are free flowing and everybody talks through what they think is possible, what they think is hard and easy, and we kind of evolve it. And then I'll do a second round of mock ups based on what we learned, and then I'll hand that off to a designer who can now start making it look pretty and, you know, actually think through all the usability questions of like, should we have tabs for the solution or should there be like a drop down that allows you to switch between views or something like that. You know, they'll think through all that hard stuff, and then we'll get engineering involved to actually build it.
Speaker 3:Generally, we'll go and get some feedback from customers somewhere along the way, and then we'll eventually do a limited release to a handful of customers with a brand new feature, or if it's an iteration, we'll show it to a couple of people right before we deployed it, and assuming there's nothing crazy that people tell us, we'll release it to the full customer base. And then, of course, we listen very carefully on the feedback channels to see if they freak out about anything. You know, if they tell us this is like Stitch to the live story I told you guys about, we'll try and make an iteration really fast.
Speaker 2:Interesting. Hey, is that thesis something you can share with people? Just the way you organize that? Or do you have a good example of that?
Speaker 3:Yeah, I can definitely do that.
Speaker 2:Okay, well maybe what we'll do is, if you could send that to me, I'll make that available at productpeople.tv. Because I think people would be interested in seeing that.
Speaker 4:The process you just described, do you guys have it sort of formalized in that way or is it a little bit ad hoc and that's just sort of what's naturally evolved?
Speaker 3:It's definitely been an evolution. The thesis actually came from Josh Elliman. He and I sat down with him shortly after I started at Kissmetrics and that was his advice. And this collagen thing I talked about came from a blog post we found about someone talking about working with Steve Jobs. So we're always looking to iterate and add to it.
Speaker 3:And I would definitely say that it's not perfect. I don't have a Gantt chart that explains when we're gonna do each one of these steps. But in general, it's kind of a checklist of I'm gonna make sure we do all of these things in some way or some form.
Speaker 4:Right. And I'm sure it probably varies based on kind of the size and scope of the feature that you're looking at building.
Speaker 3:Exactly. Yes. Absolutely.
Speaker 4:Okay. Cool.
Speaker 3:Yeah. The more it's like a known problem, known solution, the more we may cut out bits and pieces of it or just do very quick versions of it. Like, I don't always write a massive thesis. Sometimes the thesis is just like, you know, almost an email that goes to a couple of people and just has a sketch with a few things marked up on it. Done.
Speaker 3:Let's just go and do it. If it's something bigger though, then yeah, it's definitely flowing through this whole process. But sometimes we can go through all of this in a matter of a week, and sometimes it'll take a month or two if it's a really big project.
Speaker 2:Right. How would the customer development process be different for someone building a brand new product?
Speaker 3:So I think if you're building a brand new product, obviously you don't have a giant installed customer base like I have the luxury of working with the Kissmetrics. I never run out of new customers to talk to because there's like 1,500 of them. And so, you know, that's a huge luxury that I can pull from all those people. But at the same time, like our mobile app, we haven't used any of our Kiss Messaging products to drive people to that, and so we've kind of done it like a new product. That means that you have to pull things from different areas.
Speaker 3:Like, I've gotten people to talk to because of a blog post I put a form at the end of. Hacker News can be a good place depending on what your product is. You can use AdWords and drive it to a landing page or Twitter ads. You know, you just have to find a place where your customers already are, well, potential customers, and you end up shaping your conversation more about, you know, what are they currently doing to solve the problem you think is important, and find out how dissatisfied they are with current solutions.
Speaker 2:Maybe take us through the actual customer development interview. So let's say, you know, I'm looking to build a web app for hairdressers. What would you know, and I managed to find someone that's willing to give me some of their time, maybe take them for coffee, what would that interview look like?
Speaker 3:Yeah, so I have a three step process I like to follow with my customer development. And at the high level, it's people, it's problem, and it's solution. And so that's the order in which you drive your discussion, which usually takes twenty to forty five minutes, depending on how deep it goes, and how much rapport you have with the person, and stuff like that. So I start out with people, because this is so underrated. People is all about understanding, like, okay, so we're talking to a hairdresser about a software solution for her.
Speaker 3:Well, let's find out all about her. How long has she or he been a hairdresser? You know, are they the manager at a store, or do they just rent a chair? What is their cost structure like? Is this the only job they have?
Speaker 3:Do they do this full time? What kind of hours do they work? Do they have repeat clients, or are they someone who just relies on whoever walks in? What percentage of their base is that? Are they a high end hairdresser?
Speaker 3:How much is a haircut from them? How much is a perm and other girly things I don't understand? You want to learn all this baseline information, because later on, when you start getting a signal, you need to be able to parse out why did this hairdresser love the idea and this one hate it. Oh, it turns out high end hairdressers don't need this, but the low end ones are absolutely in love with it. Cool.
Speaker 3:That means I need to walk into Supercuts. Or Man, Supercuts people don't like this at all. The high end ones love it. I better go to a La Lange something or other fancy You know, yeah, you can tell I know a lot about hair. But yeah, so I spend this time learning more about the person.
Speaker 3:I also find that this actually builds really good rapport with them because they feel like you want to actually get to know them, and that leads to them opening up a lot more about what they do. And so after spend, like, maybe five or ten minutes getting to know who they are, and ideally, you're going to do as much of this behind the scenes as you can. So, for instance, you should be able to Google and find out what the hairdresser place is like that they work at. And so you can ask second level questions about that, because you already know how much it costs, where they're located, do they have multiple locations, are they the manager? There's all these things you could already know if you did your homework, so you can ask deeper questions.
Speaker 3:But from that, then I like to talk about their problems. And so instead of just saying, Do you have the problem I'm trying to solve? I actually try and find out if they'll say it unprompted. So I then will say like, Hey Susie, what are your biggest problems and challenges being a hairdresser? And you'll hear the things that she says, and if they're totally off topic from what you were talking about, like maybe you're like just talking about appointments, then you could say, well, do you have any problems related to the appointments that your people need to schedule or filling, you know, free time that you have that no one's sitting in your chair?
Speaker 3:Those will lead to them giving you all kinds of great problems that they have that you need to solve, and you'll hear great stories around how they think about it, and how they may actually solve the problem currently. And that's why I learned all this information before I finally get to the last part, which is solution, which is whatever solution that you have for them. Now that you've learned all these other things, now you can finally bias the witness by what you thought was important for them and say, hey, I have a solution. I think it solves the problem you already mentioned in our conversation. What do you think it is?
Speaker 3:Can I show you this thing? And whether it's mock ups, some wire things, or an actual app or, you know, web product, can open up your laptop and show them. Now they're much better primed to see your solution. If you would open with a solution, then all they're gonna think about the whole time in the conversation is what you made, and it may or may not even be a fit for them. And what's great is, if it turns out you've heard like people about them as a person, and you've heard the problems, and you know your solution isn't gonna work for them at all, on the fly, you could sketch up something else and say, well, actually, so what I came here to talk to you about today, it turns out probably isn't good for you, but I wanna see if this other thing you mentioned, like, if I did this, and you just draw something up real quick or talk just even talk to them about it.
Speaker 3:You can then learn a lot. And that's basically how I do my process.
Speaker 2:Do you ever do two part customer development interviews where maybe you cover people and problem first and then go away and, you know, drop a solution and then come back? Or do you
Speaker 3:Yes, definitely. Okay. Yes. I do both. It really depends.
Speaker 3:If you have something already and you've got a captive audience, you might as well take advantage of it. Yeah. But it's often actually great validation if they say, oh, I'm super interested in that. I'd love to use that. And you say, cool.
Speaker 3:What's your email address? I'll follow-up with you when I have something to show you. There's no better validation than what people actually do with their feet in action or their finger, I guess, clicking. If you get their email or something and you email them later and say, Hey, I've got this thing, check it out. If they don't check it out, well, you just found out how much they really care.
Speaker 3:Or if they do check it out, you can get great feedback and follow-up and it's further validation of their interest.
Speaker 4:Yeah. So I'm curious when you ask them like the second step about their problem and you ask like, are your biggest problems? That's a pretty open ended question. Yeah. Do you have like some tips for how you can maybe pry or get people really thinking deeply about what their problems really are?
Speaker 3:So actually that's the thing is I don't want them to think too deeply. I wanna know, like, to me, the most valuable businesses you can build are based on the biggest problems top of mind for somebody. So to me, if it's not in their, like, top three to five problems, it's not something they're thinking about enough in their day to day that I think that there's a strong business to be built. So if you're trying to build like a real business, you want it to be something that they're thinking about all the time. Now, the other thing is, though, like I said before, is you can zero it in a little bit.
Speaker 3:So if you're talking to a marketer, and if I'm talking to a marketer about their biggest problems, well, man, they can have a lot of responsibilities. And that's why I'll zoom in and I'll say, okay, When you're doing your email marketing, what are your biggest challenges? Well, now I've got them going in on one specific task in their day. When they're doing their daily emails or their weekly emails or whatever they're doing, now I'm zeroed in on just that, I can find out their problems in that area. And so that's one opportunity, and so that allows me to zero in on it.
Speaker 3:And you could say, you know, you could say the same thing for, what are your biggest challenges around measuring the success of all your email campaigns? Oh, man. It's a nightmare to try to do attribution to, you know, first touch. Oh, well, that's what Chase Metrics is great at. That's why we do it.
Speaker 3:You know? So you definitely can say, what are your biggest problems related to x? You know, if you're the hairdresser, maybe it's related to appointments, or maybe it's related to ordering scissors. I don't know. I don't know.
Speaker 3:So you can definitely you can zoom in a little bit more, but I still like to keep it a little bit high level just because if they can think of it off the top of their head, it means it's something that if you were marketing to them on the Internet, and they saw an ad on, like, a website they're on, they'd probably click it.
Speaker 4:Right. And so sometimes, like, when you get somebody talking about their problems, sometimes they automatically start going into the solution that they're envisioning themselves. How how do you kind of like temper their input for what the solution would look like against like the solution that maybe you'll come up with later?
Speaker 3:Five whys root cause analysis is my favorite tactic. So basically, this came from Toyota manufacturing and Eric Reese has talked a lot about it. You just start asking them why. Oh, why do you Okay, why a solution like that? Okay.
Speaker 3:Why that way? And, like, you start asking why a few times, and you start to get to the core of like, oh, I want it this way because of x or y or z or because of this thing. And I found that that will often get people back on track. The other thing I do too is I ask people things like, if you could wave a magic wand and have the perfect solution to that problem, what would the outcome be? And when you ask that, what happens is people remove all their preconceived notions on what's possible, and just get down to what they really want.
Speaker 3:Like, Man, I really just want to be able to have a full chair all day. You know, I want to be able to have people constantly coming in, so I would love to just have a solution that allow me to push a button and say, I have no appointments for the next two hours. Can I get someone in? Like, right now? And, you know, that would help them in instead of them saying, well, you know what?
Speaker 3:I want this, like, booking solution that, like, shows my availability and allows me to, like, ad hoc email all my customers. Like, all of a sudden, they're just going down this huge pipe of this bloated product. When you ask for them to wave a magic wand at what they really want, that often removes all those biases around what they think is possible and what they've seen before.
Speaker 2:That's a great question. If you could wave a magic wand and solve your problem, what would you do? I could see how that would get right to the core right there. I think that's the challenge with customer development is there is so much noise. When you're talking to people, there's noise coming from the person, but there's also noise coming from yourself because you might be thinking about solutions already when you should just be listening to their problems.
Speaker 2:But that magic wand scenario, that gets right to the core. I think that's really interesting and maybe that's a good place for someone building their own product to start.
Speaker 3:Yes. Yep. That's definitely one of the magic phrases that I always try to use because it works.
Speaker 4:I think it's interesting because it's also kind of defining success for you so you can kind of know like when you've succeeded at solving that problem, right? Like if you're like, Oh well, if they could wave the magic wand and have no idle periods in the day and after you've implemented your solution and tested it out and that's actually where you're going, I mean it sounds kind of silly and obvious to say it that way but sometimes I think that that's the biggest challenge is knowing when something is done or if you're succeeding. Being able to define some specific what success looks like is pretty valuable.
Speaker 3:Yeah, totally.
Speaker 1:If you to
Speaker 2:start Greenhorn Connect again, and like right from scratch, would you start by taking, you know, seven to 10 people out for coffee? Like what would be your approach if you're going to start that again from scratch?
Speaker 3:Yeah. I mean, so here's the thing. Like for me, I was going around the Boston Tech scene trying to network my way to a job, that meant I had to go out and find all events in the tech community. I found that to be a huge pain. And then I was going out to these events and talking to people, and in addition to trying to find a job, I was talking about how much of a pain it was.
Speaker 3:And so I did probably, in all honesty, hundreds of conversations that were ad hoc customer development without realizing it, because I didn't know customer development yet. And so I would do a lot of the same thing, where it's like go to the environment, talk to your customers, and see if they have the same pain you think you feel. Because often, you know, a lot of these ideas come because you're solving your own pain, and so your biggest challenge is figuring out who else has that pain and how they see it manifesting for them versus how you see it happening. Yeah. If I'm starting over and anything I'm doing going forward, I'm always going use that approach of like, Alright, let's validate this with trying to figure out a way to quantify how many people have this problem and understand, know, really deeply understand the problem that you're solving.
Speaker 2:And do you think, you know, asking, like, how many people if because this is a good scenario because a lot of our listeners are building their own product and I think they're thinking, how do I get started? Do you think they should aim for 10 people at first or what's a good number?
Speaker 3:So here's what I would do. Step one is build a landing page that talks about the problem that you're solving. Or you can write a blog post and put a sign up form at the bottom. Either one of those will accomplish the same thing. Then you want to drive traffic to that, whether that be getting people to read your blog post and sign up, or buying AdWords, or Twitter ads, or whatever you need to do to drive traffic to it.
Speaker 3:Maybe you're posting it to LinkedIn groups that are relevant. Whatever it is, you're driving traffic to it, trying to get, like, between a 100 and a thousand people to sign up. Then what you're to do is you're going to try and talk to seven to 10 of those people with a lengthy interview, know, twenty to thirty minutes. From there, you're going get tired of hearing the same thing from those people, and either you're going to start validating your idea, you're going go, Oh, this is all wrong, and you're going to have to change. Assuming that everything's good, though, now you have enough information that you could write a survey to ask some of the same questions that you got from those people that you spoke to in detail.
Speaker 3:And I would then survey their whole user base to find out as a whole. So I found these 10 people and I learned from them. Now let's validate that those thousand people I have on my list feel the same way. Now send them the survey. And from those survey response, both the number of people that respond will tell you how many people actually really care about the problem, and then the responses you get to the survey, which now should be a lot of multiple choice because you know the answers that they're going to say, will really inform you, you know, what this kind of spread of the demographics looks like for the problem.
Speaker 3:And from there, now you're ready to go start and build it, because now you have all the quantitative and qualitative information.
Speaker 2:That's a really good outline there. So just to summarize, you said set up a landing page, drive traffic to that page, and I assume what you're saying is, you're basing it on how many people sign up for your email list.
Speaker 3:Yeah, exactly. How many people sign up on the form on your landing page? Yeah.
Speaker 2:And then talk to seven to 10 of those people in-depth about their Then based on those interviews, write a survey and send that out to the rest of your list. And then based on those responses, both how many people responded, like how many people cared enough to respond, and the actual data you got, maybe summarize all of that and then start building.
Speaker 3:Yep. Yeah, there's an awesome blog post by the guy who started off FOG PHP, which is like Heroku for PHP, I believe. And he literally basically followed this process. He had his own problem. He thought he wanted to do it.
Speaker 3:He was too lazy to build it one night when he wanted to get started, so he threw up a landing page and put it on Hacker News, got 800 sign ups. He then made a really brutally long survey and found that, like, almost everyone took the survey, and his landing page kept growing with number of people signing up. He ended up starting to build features based on all that.
Speaker 2:I'm going to put that outline on the site, productpeople.tv, because those steps, I think, is exactly what anyone building their own product from scratch should follow. That's just a perfect step by step, here's what you need to do to validate this idea before you start building or designing anything.
Speaker 4:It's also a great step by step. The steps basically will give you a taste of all the different aspects of product management and running a business that you'll need like setting up a landing page where you're going to have to learn a little bit about conversions and copywriting and then driving traffic. You might need to do some content marketing or something like that. Then creating these surveys you're gonna have to learn about customer development. And obviously at some point you're gonna have to move into the design and development of the actual product itself.
Speaker 4:But it gives you a good taste of everything that you're going to have to become an expert at if you want to turn this into a business.
Speaker 2:Yeah. I
Speaker 4:didn't know if I cut out there or what, but
Speaker 2:No, no. I think that silence was silent agreement.
Speaker 3:Yes. I got nothing to add.
Speaker 4:Since we lost the video, it's hard to gauge. Jason, we're going to wrap up here pretty quick, but one last thing I wanted to ask you about was, dealing with expectations. Like when you're kind of the hub, like we talked about earlier, you're talking with customers, you're talking with salespeople, you're talking with executives. How and everybody who has an idea for, like, the problems that need to be solved. Everyone thinks that their problems are kind of, like, the most urgent.
Speaker 4:So how do you deal with setting expectations with the sales team who are, like, you know, maybe nagging you that this feature has to get built, it has to get built, or deal with the customers who say, when is this feature going to be built? Are you going to build it? How do you deal with that?
Speaker 3:Honestly, that's part of the job. That's one of the big things you have to do is like being a product manager really means being a great communicator and a people person because you have to have relationships with everything from the most outgoing of engineers to the most introverted engineers, and the most creative out there designer to the most customer driven designer, and the salesperson that understands nothing except dollar bills, and support people that really just want to make everyone happy and are tired of answering the same question a 100 times. So yeah, there's definitely always angry people, and so people skills is a huge underrated part of product management. And so you just have to be really good at kind of making people feel like they're being hurt. So one of the things I go out of my way to do is that when we act on something, whether it be support told me there was a problem they really wish someone would finally fix, or sales tells me, Hey, customers keep asking for this and we don't have it, When we build it, I make sure that I explicitly email them thanking them for telling me about the problem they had, and their patience on waiting for them to get a solution.
Speaker 3:And that goes a long way in building trust, where I can then say, Listen, I hear you. I know this is frustrating, and unfortunately, I can't do anything about it now, but I promise you that I will get to it when we can, and please keep sending me signal because we will do more about it later. And that's gone a long way to build trust and a good relationship. And, you know, in the end, we were talking about cheesy movies earlier. For me, like, I love The Replacements with Gene Hackman and Keanu.
Speaker 3:And, you know, one of the things that Gene Hackman says is, when the game is on the line, winners always want the ball. And I think that for product managers, like, if you build the wrong features, you build stuff people hate, you know, essentially, you're gonna lose the game. So, like, I look at it like, look, this is a high pressure job, but I love it. Like, I get to do so much. I get to interact with so many different people.
Speaker 3:This is the price I pay. Get to talk to people and work with all the different departments, but it also means I have all the pressure of that to deal with. And so you either thrive in that, or maybe you don't want all that pressure and the responsibility of doing the product roll in.
Speaker 2:Man, that's really great advice. Jason, this has been so helpful for colonized product managers. If people out there, like our listeners, want to reach out to you, where can they find you on the web?
Speaker 3:Probably the easiest way I use Twitter a ton. So my handle is just evanish. And I guess you guys would probably put this with the forms. And then Yeah. My about.
Speaker 3:Me page has an email me button on it. So that's about. Me evanish. It's the link in my Twitter profile, actually. I always look for that stuff to follow-up.
Speaker 3:My blog also has my email on there. So, I always love to talk to other product people and compare notes and all that sort of stuff. There's lots of great ways to find me on the web. I listen to all those channels just like I do in my day job.
Speaker 2:Perfect. Perfect. Well, thanks so much for coming in. We really appreciate you being on the show. And yeah, we're definitely going to have you back on again based on any questions we get back.
Speaker 2:If you're out there and you've got more questions about customer development, shoot Jason a line, but also shoot us a line ProductPeopleTV. So that's all for now. We'll see you guys next week.
Speaker 1:Hey friends, if you want the resources that Jason E. Vanish talked about, the product development thesis, the customer development process, and that process for validating your idea, all you need to do is sign up for our mailing list. Go to productpeople.tv/newsletter. And we're going to send all of those resources out in our next email. So go to productpeople.tv/newsletter.
Speaker 1:Now it's time for our shout outs section. This is a chance for you to advertise your project to our audience of product people. To purchase a shout out, go to productpeople.tv/shoutout. I really love this first shout out. It's startup-videos.com.
Speaker 1:Start up videos showcases the best startup videos around, helping you find inspiration and resources for your product demo video. Go to startupvideos.com. And one last shout out for Product People. Follow us on Twitter at Product People TV and let us know what you think about the show. You can also email us at ProductPeople@BizBox.CA.
Speaker 1:And if you haven't already, please give us a review on iTunes. It helps the show so much. That's it for this week. We will see you next time.
Listen to Product People using one of many popular podcasting apps or directories.