There are two broad philosophical approaches to explaining the forces that drive world events. The first one is sometimes called the Great man theory, neatly summarized by the quote ”the history of the world is but the biography of great men.”  This view was famously espoused by the philosopher Hegel and later Nietzche, who called such great people Ubermenchen (“supermen”).

In the second part of my interview with Eric Ries, we discuss (1) acquiring customers without launching and (2) opening up board meetings to the entire company.

At IMVU, Eric and the management opened up board meetings to the entire company. Why?

1. To give people the information they need to do their jobs.
2. To teach everyone in the company to think like the CEO.
3. To prevent employees from gossiping about board meetings.

And more!

I’ve synchronized the audio with some simple slides below. That’s my favorite way to consume the audio. You can also find a transcript and stand-alone audio below. Please let me know if you find the transcript useful.

Read on to learn what kind of employee Eric used to “show the door” at IMVU…

Slides: Opening up board meetings (pdf)

Audio: Opening up board meetings (mp3)

How do people find out about our product if we haven’t launched?

Nivi: And this gets to a second topic, which is you guys weren’t doing any of this really in public, because you had not launched a product, right?

Eric: Amen.

Nivi: Nobody knew who you were, but people were, at the same time, were using the product, so how did you do that?

Eric: I got a question today which was something like, “I’d love to follow your advice about not having a public launch, but we need to get early beta users for our product launch. How can we do that if we are not willing to talk to bloggers? Nobody really knows who we are.”

I think a lot of people have that attitude, that without PR, you just can’t get any early customers. Again, we have got to start with, “What is the goal of early customers? Why do you want them?”

If you are charging from day one, one of the reasons you want them is you actually want to make money. You want to show that your business is viable. But even if you are not charging money, you have a need to find out whether your business is viable, whether you have that minimum viable product, whether the business model, at the end of the day, is going to work.

For that, you do need customers, and you do need to be putting customers through a product experience that will give you that information. But you don’t need a lot of customers. I think that is where people get confused.

For a big fancy launch you can get hundreds of thousands of customers to show up for one day. But for metrics analysis, generally a cohort of 100 people or 1,000 people are plenty to learn from.

If you change your goal from, “How do we get the maximum number of customers,” to, “How do we get the minimally sufficient number of customers to learn what we need to learn,” new possibilities get opened up to you.

Acquiring customers on $5 a day

Eric: For example, at IMVU, we practice the $5 a day AdWords campaign. I was the VP of marketing in those days. If I actually knew anything about marketing, I would have known not to try this. By traditional marketing standards it is considered crazy to spend only $5 a day, but we had a pretty low budget and we really were pretty frugal.

I discovered that in those days you could buy clicks for five cents a click. But to me, $5 a day meant every single day 100 human beings are coming to try my product.

If you think about that from a beta testing point of view, especially if you look back at the old days of software shipped by CD, getting 100 people to try your product is actually a lot and you can learn a lot from that. And at 100 people a day, you are in good shape, just at that tiny, tiny level.

The risks of doing that are really quite low. I think a lot of engineers have this idea that once you put your product out there in public, the investigative journalists are going to find out about it and write about it and we are going to lose control of the story. Let me tell you. You should be so lucky.

IMVU was a top 1,000 website in the whole world before it got any press whatsoever. We were making millions of dollars a year. The press was writing about newly funded, venture backed competitors that had no traction whatsoever; because those were the guys sending them press releases.

It was frustrating, and psychologically you want to have that cover story on WIRED that you can send home to mom, but you know what? We did not start this company to have good vanity covers printed about you in the press. We were there to serve customers and serve them well.

Running experiments under a different brand name

Nivi: How do I run experiments, if I already accidentally got that TechCrunch article and I…?

Eric: Yeah, I am sorry. You are not doomed, but you are going to have to go waste energy later cleaning up the positioning that you put in that article, which is undoubtedly wrong.

Nivi: Right. There is that aspect of it, but do you think you should, just basically pretend that article never existed, or do you run tests under a different brand name?

Eric: That is not a bad idea. Especially on the iPhone, I see this because of Apple’s stringent release process where there is this huge delay before you can actually bring things to market.

And also because people want to get into the top 25. That is where all the action is in the Apps store. There is a lot of competition to make sure that on the day you launch your app you get all the right coverage lined up and all the stuff happening.

People feel like they don’t want to do a bad launch under their real brand name, because that will harm their ability to do the proper launch later and get to the top 25.

But there is no law that says you can only bring out products under one brand name. I strongly, recommend that to people if you are very concerned about your precious brand. I think most startups are way too concerned about the power of their brand. They should be so lucky to get some kind of brand going.

Even still, bring it out under a terrible name. I specifically recommend people bringing products out under brand names that they hate so that they won’t ever be tempted to make that into their real official brand name and then become afraid to experiment with it.

You have got to be bringing products out under a brand that you feel comfortable experimenting in. Then once you find the right formula, there are two possibilities.

Either you will be able to port that product over to your new brand name and it will be great, or the product concept you brought up under that bad brand name will be so powerful, you just can’t get people away from it.

It is too sticky and you are stuck with it. But congratulations! You are successful! Is it really so bad that you personally don’t like the brand if customers do like it? I think it is not so bad.

Running pricing experiments in public

Nivi: A friend of ours has a popular subscription based product that they don’t charge for and now they want to start charging for parts of it for the premium model, and they want to find the optimal pricing strategy. How can they run those experiments in public and in secret? What would you suggest to them?

Eric: I would actually not be afraid to run them in public. It is hard for people who are afraid of what the worst possible thing that could happen is, to do this. But I think it is good to just try it and get over it.

What happens is, it is true that customers don’t generally like the idea that one customer got charged one price for something that somebody else got charged a different price for the same item. So there is some risk when you do different pricing offers in a split test.

But in my experience, there are two mitigating factors that make it not so bad. The first is it is actually incredibly difficult for most customers to figure out that is happening, especially if you only do it in a limited time window.

For example, I am going to tell you a story that may not seem related, but bear with me. When we were at there.com, the virtual world company, we would do a lot of QA. That was a heavy QA company.

For hours every day we had QA testers sitting in a lab together running the virtual world software and testing to make sure that it worked. I remember one day getting called in to see about…

There was one tester. They were around a physical corner from each other. So you couldn’t see each other, but they were not more than 20 feet away. They were both engaged in this activity.

The guy called me down and he said, “I am in this dune buggy riding around with somebody in the virtual world and we are seeing this glitch. We are not seeing the same thing. Something is not right.” They were calling back and forth, trying to pin down what it was.

I remember sitting there really confused about what the problem was, because it looked like the two of them were sitting there in the dune buggy and everything was fine.

I walk around the corner to the other guy. I talk to him about what the problem is. I look at his screen. On his screen, he and the other guy are engaged in a paintball match. They are not in a dune buggy at all.

He was almost a mile physically distant in the virtual world from where the other guy was, yet their conversation was perfectly consistent to them and it never occurred to anybody to ask, “Wait. What planet are you on in this time that we are comparing notes?”

They had no clue that this was happening. I think, we totally tend to underestimate just how powerful the pull of what you see is to most people. They basically can’t imagine the world, any other way than the way that it is.

Entrepreneurs don’t have that problem, so a lot of times they don’t grasp what is true for customers. It is actually very unusual for the customers to go onto a forum and post, “Here are all the offers that I am being offered and exactly what I see. Does anybody else see the same thing?” Our natural assumption is that everybody else sees the same thing.

So you are not totally likely to get caught. That is a mitigating factor. It is actually not as bad as you think when you do get caught, because don’t forget; you have the power of the apology, especially as a startup.

If you screw up… You are going to screw up all the time. If you are a customer of a startup, your general experience is, “These guys are constantly screwing up.” What customers care more about than whether you screw up or not, is how you treat them when you do screw up.

They care that you listen to them and take them seriously more than if you always get it right. If they want to work with a company that always gets it right, they will go work with some premium giant company that really has a very carefully constructed customer experience.

If you get caught doing this thing, you can always say, “We are so sorry. We were experimenting with this pricing. We didn’t mean for this to treat anyone unfairly. And if anyone was treated unfairly, we have gone back in the records and we are going to give them all double the money back for the thing that happened,” or whatever you have to do to make it right.

That is OK! It is really not that bad. What happens then is people say, “Wow. These guys are serious about making sure that we get treated fairly.” Meanwhile, you get to keep experimenting.

Opening board meetings to the entire company

Nivi: Yeah. You have talked a couple times on your blog about how you opened the board meetings up to the entire company and the positive benefits of that, and people’s perceptions of negative benefits.

Eric: Yeah. Well this is not something that a lot of companies adopt. This is considered pretty crazy.

I don’t know if it’s that most people are actually afraid of giving the whole company information they need to do their job, because it might lead them to judge the top management harshly, but people judge you harshly whether you give them the information or not, from my point of view.

Just give them the information! Your pathetic attempts to hide what’s happening don’t fool anybody.

Having been on both sides of that divide, I can tell you I never felt like I was being successfully fooled. And if you do manage to fool me for a limited time, I’m awfully pissed. My point of view is: you want people to have the maximum information possible.

You need to do it in a trust-building way. You’ve got to make sure the people you’re giving it to understand what they can and can’t do with that information, and they understand that they need to keep company secrets confidential.

If you don’t trust your employees to keep company secrets confidential, you’ve got bigger problems and you should go address those problems first.

There is some board business that has to be done in secret for legal reasons, so it’s not true that absolutely every meeting that any time ever happened at the board level is open to the public.

Employees have critical things to say in board meetings

Eric: But the interesting part about board meetings is the strategy conversation where you present progress, show data, and you make discussions about what should happen next. And that’s the part of the meeting I strongly recommend people open up to their employees.

What we did is we actually had a board of advisors and then a board of directors that was a subset of those advisors. We would convene the full set, advisors and board, at nine o’clock in the morning and we would have a maybe two-hour strategy conversation followed by maybe a half-hour or one-hour private board meeting.

For the strategy conversation the rule was: every employee can attend. We did this up until we were a 50- or 60-person company. We actually, physically crammed everybody into one room, and we had the employees sit around in as much seating as we could fit and the board members would sit at the big table.

It wasn’t a free-for-all, most of the employees were encouraged to listen, not to speak. But every once in a while the rule was that if someone had something they really needed to say, they could be recognized by the CEO and say their piece.

It was amazing. We would, occasionally have a board meeting where we would have a moment, where there would be data we were presenting to the board, and it would indicate that on a certain day, a certain metric went up and that was due to us launching that feature that day, or whatever our interpretation of what that data meant.

And not an insignificant number of times we would have an employee raise their hand and say, “Excuse me, but do you also realize that something else happened on that day?” Yada, yada, yada.

And occasionally, I’d be the one presenting! On the one hand, I’m really embarrassed. So I’m like, “No, I didn’t realize that.” This is a critical thing about running my own business I didn’t know.

But once I got over my personal embarrassment, what you would find is the board loved it! They’re like, “Thank God that guy was sitting in the room and could enlighten us about that. That changes our interpretation of what this means.”

And quite a few times I think we saved ourselves months of work by coming to a realization of something way earlier than we would have, because the right guy happened to be sitting in the room.

And yeah, occasionally you had an employee who’d make an off-color comment or say something that really shouldn’t have been said in front of the board, but people learn from those experiences. Most of the time most people had really substantive conversations.

Nivi: Did you ever get in a situation where some of the employees were like, “I don’t even care about these board meetings. I don’t even want to go?”

Eric: Yes, yes! We eventually had people who on occasion would beg me not to have to go to the meeting. And we eventually made them voluntary. For a while I was really rigorous, I said, “No, everyone has to be there. If I have to be at the meeting, you have to be at the meeting. Why do you think I’m any more privileged or unprivileged than you?”

Yeah, because board meetings are actually pretty boring. But when people are outside the room looking in — and you know most conference rooms have some form of glass — people can see what’s going on. They’ll come up with an excuse to walk by, kind of peek in. They will make up whatever crazy conspiracy theories are consistent with the data if they’re not there.

An, in my point of view, that’s such a source of waste: people gossip and there’s rumors and people don’t know. Let them be in the room, let them see how boring and mundane most board meetings are. So that for the occasional one where something actually interesting gets decided, let them be there to hear it themselves.

Everybody in the company has the ability to understand what everybody else in the company has to understand

Eric: There are some costs, definitely some down sides to doing it. One which took us by surprise was that, people can occasionally get confused about who’s in charge, we did occasionally have people — some board member would say, “You guys should really build feature X.”

Board members occasionally would just spout off about what’s randomly on their mind, and occasionally you’d have an employee get confused that that means the company is now going to go do feature X because board member so-and-so said so.

And that was actually good practice for us, to be a constant reminder that no matter who you are, no matter what it says on your business card, nobody gets to decide randomly that the company’s going to do feature X. Right? I don’t care if you’re the CEO or the lowliest person, we’re going to have a reasoned and considered process for deciding what to do.

Nivi: And it’s a learning opportunity.

Eric: It’s always a learning opportunity. The other thing that was hard for me personally was it’s hard to have your people who work for you see you be criticized in public. That was not fun.

Nivi: Hard for whom?

Eric: Well, it’s hard for me. My emotional reaction was like, “Wait a minute! I’m doing the best I can and now you’ve got to watch me get smacked around because I screwed something up.” But once I got over my personal emotional response to it, it was wonderful.

Because it humanized me to the people who worked for me — they got to see, “Oh, I see the pressure that he’s under” — but more importantly, when I needed something from somebody for the purposes of presenting to the board I could go to them and say, “Do you remember what happened the last time I didn’t have the right answers to these questions or I had shoddy this? You’re really going to send me in there with this? Come on, you’ve got to help me out!”

So it made us collaborators in creating solutions for the board rather than I’m constantly asking them for stuff and they don’t know why.

And I think, get over your own infallibility. We all make mistakes and it’s better for people to see what the real stuff is.

Nivi: I think you wrote about this on your site, basically the assumption is that everybody in the company has the ability to understand what everybody else in the company has to understand.

Eric: That’s right.

Nivi: The assumption is I have the ability to understand what the CEO has to understand.

Eric: That’s right. And that makes people uncomfortable, because sometimes we would say, “You have the obligation to understand what the CEO’s going through right now, because it’s going to impact the way you do your job.”

Some people would say, “I just want to sit in my narrow corner, do my little thing, and I don’t want to worry about what the company strategy is.” And we would show those people the door. We were really serious about that.

You really needed to have people who were… they didn’t have to be good at it! We weren’t asking them to be good at the CEO’s job, but we are asking them to understand why is the CEO making the decisions that he’s making. Because they’re going to have to make CEO-level decisions sometimes.

Sometimes the actions that have the biggest impact on the company’s performance are taken by people at the line employee level. They may not realize it’s going to have that big impact, but they are going to make those decisions. By the time the CEO finds out about it, sometimes it’s way too late to do anything about it.

We sure hope that the guy at the line level understands what the company strategy is and how his decisions impact, at least the best that he can.

Nivi: Yeah, I think maybe their decisions impact the company more than the CEO in the sense that if the CEO doesn’t come in to work, who cares? The company proceeds, but if the team doesn’t come in to work nothing happens.

Eric: OK, let me tell you: when the community manager takes a day off, you can have serious, serious meltdowns in the community if it happens to be the wrong day. That can have major impacts on the company.

Should we share bad news with employees?

Eric: I’ll say one more thing because this is a real effect that people are afraid of, which is that if you give people information about how a company’s doing, it can impact morale negatively. Sometime the company’s not doing well.

It makes some people have this idea that part of your job as a manager is to shield people from bad news or shield them from chaos. Because it’s not fair to them to have them have to do their job and also be confused about how the company’s doing. I just think that’s a really paternalistic attitude that we just need to let go as an industry.

If you want people to believe you when you tell them the good news, you have to sometimes tell them bad news. Otherwise, you have no credibility. And when there’s bad news to be shared, yes, it negatively impacts morale. But for a good reason, because things aren’t going well and we now need to rally the company around the fact that we need to change what we’re doing.

And there’s nothing like actually seeing the board say, “You guys have a major crisis on your hands that you have not yet understood,” to get everyone in the company saying, “We’re alarmed. We need to do something about it.”

That can cause some chaos, and that can be disruptive, but if you build trust and rapport with your employees then what you could do is you can sit everybody down for an analysis meeting after the board, which we would always do, and say, “OK, let’s talk about what we heard and what does it mean for the company,” and let people share their perspectives.

Let people say stuff like, “This says to me we need to cancel all our projects and completely retool.”

You need to get that idea out in the open because when somebody thinks that, you don’t want them to just unilaterally go execute on that plan! You want the opportunity to tell them and everybody else who didn’t have the courage to say the same thing: “No, we’re not retooling, but we are going to make some adjustments and here’s how we think about it, here’s what we’re going to do about it and here’s what’s going to happen.” That was pretty powerful.

Nivi: Thank you!

Eric: You’re welcome.

Nivi: I think that was great.

Eric Ries and I recently sat down to talk about minimum viable products: the product with just the necessary features to get money and feedback from early adopters.

The minimum viable product (MVP) is often an ad on Google. Or a PowerPoint slide. Or a dialog box. Or a landing page. You can often build it in a day or a week.

I recorded the interview and synchronized it with some simple slides below. That’s my favorite way to consume the audio. You can also find a transcript and stand-alone audio below. Eric also highlighted some excerpts from the conversation on Lessons Learned and put together his own presentation on MVPs. Let me know what you think — I’m especially interested if you like the synchronized audio and slides.

In Part 2 of the interview, we discuss opening board meetings to the entire company.

Slides: What is the minimum viable product? (pdf)

Audio: What is the minimum viable product? (mp3)

What is the minimum viable product?

Nivi: First of all, this is Nivi from Venture Hacks, and I’m talking to Eric Ries from… where are you from?

Eric Ries: From the Lessons Learned blog.

Nivi: From the Lessons Learned blog, formerly from IMVU, formerly an advisor to Kleiner Perkins. We’re just going to have a discussion on a few topics of Eric’s and my choosing.

Nivi: First topic: What is the minimum viable product? Talk to me about minimum viable products.

Eric: OK, well let’s start with the question. Why do we build products in the first place?

In the end, we hope to be able to launch product to lots of customers and have them give us money so that we build a great business.

One approach to solving that problem would be, let’s build a product with the maximum number of features that will maximize our chance of success in the end. But the problem with that is you won’t get any feedback until you’ve already built all those different products.

All those different features, so you ship this product with a ton of features, and generally, by the time it’s done, it’s too late to make sure that you are on the right track.

The alternative would be, let’s do the release early, release often thing, and let’s get feedback as we go. The issue there is, if you just follow the release early, release often mantra, you find yourself running around in circles, because you ship code, you get some feedback from people, you do a focus group.

Some customers say, “Give me feature X,” “Give me feature Y,” now you’re kind of like, maybe sometimes you do what they want, maybe sometimes you’re going to do what you want, and then they get mad at you, and you’re chasing your own tail a little bit because you’re not operating against a clear, long-term vision of what you’re trying to accomplish.

The idea of minimum viable product is useful because you can basically say, look, our vision is to build a product that solves this core problem for customers, these kind of general feature areas, and we think that for the people who are early adopters for that kind of solution, they will be the most forgiving.

And they will fill in their minds the features that aren’t quite there if we give them the core, tent-pole features that point the direction of where we’re trying to go.

The minimum viable product is that product which has just those features and no more that allows you to ship a product that early adopters see and, at least some of whom resonate with, pay you money for, and start to gave you feedback on.

Nivi: So there is some set of customers out there, we believe, that just with these features alone, this product is useful to them.

Eric: Exactly right. And sometimes it’s useful to them because early adopters have the same kind of visioning power that entrepreneurs do, but because they can see what the end product is going to be.

Getting developers on the IMVU platform with a MVP

Eric: There are cases where you ship them a product that actually doesn’t work very well — at IMVU, the first version of our product worked pretty terribly, but, for example, it was all about user-generated content. We wanted to get developers on the platform.

The problem with developer platforms is this chicken and egg problem. Developers don’t want to develop unless there are customers who are there to buy their products, and customers don’t want to come on the platform unless developers are there selling them something useful.

What we did is we took early adopter developers and we told them a story about how IMVU was going to take over the world and be this really powerful product for mainstream customers and we made them believe it.

And we gave them an economic incentive that said, the earlier you get on board with the platform, the bigger your take is going to be for derivative products that get created down the road.

We shipped a product that basically had almost no customers — certainly no mainstream customers, and the developer tools weren’t that great — but, because we had told that story effectively and we really understood those early adopter developers, we got a ton of them on the platform developing.

Because they felt like they were in the middle of a gold rush, despite the fact that there was really no evidence to support that belief other than their own power of imagining what this thing was going to be down the road.

Luckily, we delivered on that vision, and so they actually were — a lot of them — pretty happy.

A non-MVP: The Kerry vs. Bush avatar

Nivi: I want some examples of these crazy minimum viable products. By which I mean, for example, and you can talk about any of these, the AdWords approach, the approach you talked about in terms of dialogue boxes and just popping those up, and just trying to sell a PowerPoint slide to an enterprise customer.

Eric: Let me start with an example of a time we didn’t know the minimum viable product, to illustrate my point.

It was 2004 — you have to remember 2004, Bush versus Kerry election. At IMVU, we had this idea — it happens to entrepreneurs all the time, you wake up in the shower and you’re like, “I’ve got an idea for a killer product.”

The idea for us — this was probably September/October 2004, debates are happening, politics is in the air — our idea was, we’re going to sell presidential debate avatar set that we would either dress up like Kerry or Bush, and you’ll be able to debate with your friends in this 3-D presidential debate product.

Now, put aside for a second whether that actually seems like a good idea to anybody else. We convinced ourselves it was a great idea, and we spent two full weeks racing to get this thing built, because it was time sensitive, the election is coming, the debates are happening, every day counts.

We thought it was a great idea for that whole two weeks. We built it out and we shipped it and we spent countless hours debating exactly what features had to be in it or not in it, and we were going to sell it for $1.99.

Our theory was, pricing won’t get in the way of anybody buying this thing. We want to make it cheap and easy and it will make lots of buzz and Wall Street Journal and New York Times are going to cover this thing; it’s going to be awesome.

I remember for us in those days, two weeks of development was a lot, because we were a pretty fast team. We did that, we shipped it — cut to the chase, nobody bought it. We sold exactly zero copies of the Kerry vs. Bush avatar.

We tried a bunch of different permutations and different variations of it and added features, and we changed the price, and eventually we gave it away for free, and even at free we couldn’t sell any copies.

Nivi: Well how many bought it for free?

Eric: None.

We literally couldn’t give this thing away. This thing was a dead weight money loser. There was actually nobody in AmErica interested in having a presidential debate avatar.

How to turn the Kerry vs. Bush avatar into a MVP

Nivi: How would you have approached this exact same product by taking the minimum viable product approach?

Eric: Well it’s interesting. We thought we were taking the minimum viable product approach because we had only spent two weeks on it. Right? Where we had made a very early prototype and put it out there.

But, if you think about it, going back to the definition of the minimum viable product, which is the minimum features that are required to learn what customers want, we had spent way too much time on it.

What we should have done, and what we did for a lot of features thereafter, is started with a landing page that promised people that product. Then we should have taken out the AdWords we were planning to take out, drive traffic to that landing page, and offer people to buy the experience that we are talking about.

What we would have found out if we were doing that experiment is 0% of people would have clicked through, which means it doesn’t matter what is on the second page.

The first page is so bad, not because it is badly designed, but because the features are wrong that you don’t need to go through the effort of building out the product. So we wished we had done that, and we did make that mistake really.

Nivi: How would you have marketed that minimum viable product to your existing customers?

Eric: When you already have customers you need to have some way to experiment with making them an offer. In a lot of cases the minimum viable product really is just that offer.

You can crisply articulate to customers what they’re going to get and how much they’re going to pay for it. You can learn a lot by just popping up a dialogue box that says, “Hey, would you like this new feature?” or showing them a banner ad for that feature.

For example, on IMVU we would have a system setup so that we could arbitrarily from the server select a small percentage of customers and make them an offer by inserting a dialogue box into their conversation, and basically it’s a simple “Yes” “No,” would you like this thing, “Yes” or “No?”

When they say “Yes,” we take them to a landing page where we try to sell it to them. Eventually if they really did want it, we would have to make up some excuse why we couldn’t give it to them like, “We’re experiencing technical difficulties right now.” “We’re not quite ready to give it to you. Give us your email address and we’ll email you when it’s ready.”

Again, you’ve got to remember that 99% of the time nobody wants it. Most offers that appear to an entrepreneur as a good idea are actually horrible, horrible ideas. By making the offer and having it be rejected by customers, we learn not to waste time building stuff that nobody wants.

Nivi: Right. Maybe the right definition of a minimum viable product, like you were saying, is, essentially a test to see whether people will actually want the product that you’re imagining in your head.

Rejecting false negatives: “But my customers don’t know what they want!” #

Eric: That’s right. The reason why this is an art and not a science is, I’ll have entrepreneurs come up to me and say, “But hold on, my customers don’t know what they want. If I ask them ‘Do you want this thing?’ They might say no when the answer is really ‘Yes.’”

Unfortunately, that’s an excuse that is used way too often, but there are situations where it’s true. The judgment call is; what really is the minimum set? In some cases like in entertainment products it might actually require you to build an early prototype, or a mockup, or even version one of the product with the minimum possible set of features that you think could go.

The nice thing about minimum feature set is you can always try intermediate points to ask yourself, “Am I at the minimum feature set yet? Am I at the minimum feature set yet?”

As long as you’re not afraid of the false negative, that is, if you don’t get discouraged because you’ve built your first paper prototype of it and shown it to people and nobody wanted it. That can’t mean that you give up because, “Oh, forget it, we’ll never make it.” You’ve got to say, “OK, well then let’s iterate some more.”

If you keep iterating at it, you keep making it a little bit more sophisticated, at a certain point after you’ve been through 10 iterations, that you still got no uptake whatsoever, and the feedback you’re getting from customers is still a yawn, you might say to yourself, “You know what? We’re not moving in the right direction. In fact, we’re past the point of minimum viable product. This just isn’t a viable product.”

Nivi: Right. Back to your point about entertainment, in Hollywood they start off in scripts. If the script looks good, then they will make a pilot. If the pilot looks good then they might order a few episodes from the first season. After the first season, then they make from there. They don’t build all three seasons, and then try to ship them.

Eric: Yeah, which means that some great shows never get made, because the early tests look negative and the people involved don’t have the courage or stamina to see it all the way through.

On the other hand, sometimes people have the courage and stamina to see through a really bad idea. That’s why the concept of learning is so important. This cannot be done on a spreadsheet. You have to keep training yourself on multiple iterations and multiple attempts to start to develop good instincts for it in your particular domain, your particular market space, what’s likely to work and not work.

Then you can do that as an entrepreneur, but even more powerful is if you can get your whole organization, everybody, training themselves constantly to do that kind of learning, so that good ideas will be passed around the whole organization.

Building products like packets get routed on the Internet

Nivi: Also minimum viable product ties into the concept of “build it before you sell it.” Are there any other out of sequence things that you guys did at IMVU or elsewhere that you found helpful? For example; you guys probably wrote tests before you coded to some degree as well, didn’t you?

Eric: Yeah, that’s right. I mean test driven development is the same principle.

Nivi: I’m wondering if there are some other things that you guys have done.

Eric: If you think about the full range of the product development life cycle; from specification to design, to implementation to testing, to maintenance to sales to deployment.

Nivi: Deployment and all that stuff.

Eric: I now think about that rather than as a linear sequence, I actually think about it as a big network. The question is; just for any given feature, in what path should you route it through that network? Different features should be routed differently, just like on the Internet we route packets differently as necessary.

Nivi: I’m wondering if you have some specific examples of paths you’ve taken.

Eric: Yeah, we just talked about writing tests first. We’ve talked about selling first. We’ve talked about deploying first.

Deploy first, code later

Nivi: When do you deploy a product? When have you deployed something before you built it?

Eric: All of these forms have offers of products. For example in a lot of Internet products, how does the customer know that the feature exists? Yes, there might be 100 pages of fancy stuff, but how do they even know that stuff exists?

Generally, it’s because there’s a link in the header. There’s one link on one key page that notifies them that they can go do this other thing or at a key moment they receive an email, even though the feature might be wide and complicated.

Generally, access to the feature is controlled through a simple choke point. We would often add those choke points in a split test just to see first of all, did anybody click on them? So we could look at the click-through rate of people that now believe there is this feature.

We would also see an interesting phenomenon, sometimes the presence of a feature, even if nobody clicks on it, still impacts their behavior in other ways.

For example — I can’t remember the data, we saw an experiment where, this isn’t exactly right but something to this effect, we added a link to the header notifying people that there was a VIP Club for special people only to get access to on IMVU.

Now, it’s not the same as what we have now. IMVU does have a VIP club today that is not anything related to this. But, in those days, the idea was, we were trying to test whether people wanted to have some kind of premium experience.

What happened was few people actually clicked on that link to go find out about the VIP club, but in the experimental cohort for people who were shown that header, their average spending was higher.

By constantly reminding them that there was such a think as a premium experience, we primed them to want to do more spending on IMVU. It was a very unexpected result.

And it’s a good example of why you always do full cohort-based split tests. Always test for the macro effect, don’t look at the micro effect. That’s a case where deploying a feature before we’ve built it actually can give you an insight that maybe you don’t need to build a feature after all.

Nivi: Interesting. Yeah. That reminds me of an example that one of our friends told us about where they couldn’t get people to sign up for a free product, so they put a paid version of the product right beside it, and I don’t think anybody signed up for the paid version…

Eric: It made the free version look a lot more…

Nivi: Yeah. It made the free product look legitimate. And they started to actually convert on that basis.

Design first, code later

Eric: There are also cases where you want to design a product last, rather than design it first.

We would often ship things in a schematic form with horrible design to see if we’d gotten the information flow and information architecture right, and really good interaction designers, if they’re being honest, will tell you that that’s always the sequence.

You always re-factor your design out of specific use cases and out of specific uses, rather than starting with the broad vision of what it’s like.

Nivi: Basically the trails where the people walk, you pave those?

Eric: Exactly right. And you could do that — I remember one time we used to use paper prototypes where you have designers sit down with a focus group of people and show them a fake screen shot that they came up with in Photoshop.

And then the customer says, “I want to click that button,” and you go grab the piece of paper that represents what happens when you click that button, you show them another prototype, and we used to do them that way.

We actually built a paper prototyping website into our website where, for certain customers, the designer could come in and actually replace sections of the website with full-on mockups from Photoshop that were sort of clickable and sort of functional.

Then we would bring a customer in and have them register an account; we would specify that that account should see the paper prototype, and then we would go through with them, do the focus group.

But the customer doesn’t know they’re seeing something that’s a mockup. They think they’re using the product.

And, yeah, the product is a little weird, because sometimes they click things and nothing happens, and sometimes things won’t look a little right, but customers think that about your product anyways, so it’s not particularly abnormal for them to see a product that looks completely incoherent to them. That’s the state-of-the-art for most customers and those products.

That was another case where we would — and we would, occasionally, come up with some prototype that that functioned well enough that we would ship it to real customers while we’re working on the full on, blown, beautiful version of it.

Sometimes customers would see a schematic version from which we would gather more information about whether we’re on the right track, and put things in the right place.

Where did you get the money to experiment?

Nivi: What allowed you guys to do all of this experimentation? Specifically from a financial point of view? Not about the mentality behind it and so on. Time is short and money is running out, or were you basically break-even this whole time, or how did you work that?

Eric: No, we believed in a concept we called “bridge to profitability, ” which is, whenever we raise money, we always raise money on the basis of a plan that we honestly believe, would get us to profitability on just that round.

If you notice, that doesn’t mean that we actually got to profitability on that round. In fact, we kept raising more rounds because we kept believing that we had the opportunity to pursue a bigger opportunity by raising money, but we never had to raise money. We always felt like we could get there on our own.

But that meant that we were constantly a dollar short, because plans to get to profitability on a round generally involve you going negative and then catching up before the money runs out, so we were not eager to learn, eager to experiment.

We did not love metrics. We were very scrappy and very afraid of running out of money, so we only ever adopted practices that we thought would have a high ROI.

The issue is that these practices do have a high ROI, and we got to them because we kept following the traditional model and failing and feeling like, God, we are burning money like there is no tomorrow. We keep building features that nobody wants. We literally don’t have the time and we can’t afford to keep doing the same old thing over and over again.

I don’t know if we were just lucky that we had that mindset, but, for whatever reason, even though at some level we always believed the next feature we shipped was going to save the day and have a step change in our profitability and take us to the moon, because it’s just the best feature since sliced bread.

Because we were so desperate to actually make money, we — I’m trying to think of how to put this — we became obsessively committed to actually being right rather than just believing in the next thing to save us.

If you look at the traditional hockey stick shaped curve, it’s flat for an awful long time. We had been in a company previously where we were always promised, “Yeah, it’s flat now, but we’re on the verge of the hockey stick.”

The problem with the hockey stick plan, when that’s your plan, is you just work for a year and then the 13th month you’re going to go to the moon. For the 12 months, you don’t know if you’re making progress or not, because you’re following the plan.

Everything is flat like it’s supposed to be, because when the final feature comes into place, you’re going to go to the moon.

We just didn’t believe that was going to happen. So we forced ourselves to actually test each feature to see if it was going to be successful.

Then we just stopped being able to drink our own Kool-Aid, because we were just wrong so often, it became harder and harder and harder for us to convince ourselves that it was a good financial investment to just randomly try the next new thing versus actually trying to say, “Hey, is there a way that we can increase the probability of having a successful feature.”

Nivi: Yeah, and I think, there are probably two other things here, right? You charge from day one, so basically every experiment you ran was an experiment to make more money, right?

Eric: That’s right.

A preview of part two

Nivi: Why would you not run experiments if you’re charging from day one. And then, the second thing is — this gets us to the second topic, which is: You guys weren’t doing any of this really in public because you had not launched the product, right?

Eric: Amen.

Nivi: Nobody knew who you were. But people, at the same time were using the product, so how did you do that?