A startup is a human institution designed to create new products or services under conditions of extreme uncertainty.
-Eric Ries

Ad Agencies & Startup Factories

When I first read Eric Ries’ definition of a startup, I laughed out loud because I thought it was the perfect description of a full-service advertising agency – or at least the one where I worked for 10 years.  That’s what we did, every day of the week: create new products and services for big-name clients under conditions of extreme uncertainty.  We were a startup factory – and we were good at it.

A key difference between a successful ad agency and Eric Ries’ idealized startup is the role of the intended customer in the development process.  Reis thinks that little can be learned by talking with end-users and consumers because “(m)ost of the time, customers don’t know what they want in advance.”  (Ries, 2011, p. 49)  A successful ad agency knows better: by asking the right questions in the right way – and by listening with an open-mind – one can, in fact, find out what customers want (and don’t want) and substantially reduce the uncertainty that surrounds the creative development process.

Reducing Uncertainty

The rational response to conditions of extreme uncertainty is … to systematically reduce uncertainty.  Eric Ries’ recommended method is the specification of a “minimum viable product” (MVP) followed by rapid turns through the “Build-Measure-Learn” loop to generate “validated learning” – results that derive from customer behavior and are based on empirical evidence. (See Figure 1) Ries asserts that through this process of repeated cycles of product testing – feature by feature – one eventually will develop a product that is viable in the marketplace.  (Or one eventually will be forced to admit that it’s not going to fly and then “pivot” to a new approach.)

We’re big fans of empiricism, but we hate to see clients running in circles and blindly bumping into walls at the outset of a project.  Why not get customer feedback on the MVP before the “build” process starts?  It it’s not going to fly you can pivot a lot sooner, and with fewer resources expended.

There are two approaches that, taken together, can substantially reduce the uncertainty that surrounds a new product development effort:

  • A qualitative category exploration to learn how potential customers view the category and how they choose to engage with it; what they find enjoyable and what they find frustrating; how they compare the available options; what caught their interest in the first place.
  • A well-designed concept test to identify the potential product features, attributes and benefits that will drive preference in the category, thus providing a blueprint for the product development effort.

Qualitative Category Exploration

A qualitative category exploration is generally the first step in a new product development effort and it can take a variety of forms:

  • Open-ended individual interviews, either in person or online
    Qualitative category explorations with current category users are intentionally non-directive at the outset. The aim is to allow the user to describe, in their own terms, how they participate in the category, what products they currently use, how they feel about the options available to them, and finally, the benefits (and problems) they have experienced in the category.  The lead question is NOT “Tell me what you want…”  It’s more likely to be “Talk to me about how you use (shop for, feel about, etc.) product category X. That’s interesting: tell me more…”
  • Focus groups with current category users
    The focus group format follows the same outline as the individual interview but allows for interactions between participants.  This cross-talk between category users can often lead to serendipitous learning when participants challenge each another’s comments and preferences. Or when someone start building off the comments of other participants to create something entirely new…
  • In-home or on-site observations
    The anthropological approach has similar objectives to the qualitative individual interviews: to allow the customer or end-user to demonstrate – through ordinary language and natural behaviors – how they participate in the category, how they use the available products or services, what they find appealing, what they find awkward or annoying, work-arounds they may have developed, alternatives they are considering, etc.

Avoiding Blind Alleys
In reading Ries’ account of the IMVU development process, it is painfully obvious that his belated interviews with existing IM users ultimately saved his project – in spite of his repeated assertion that “customers can’t tell you what they want.”  As the IMVU project approached the point of collapse, the feedback he received from current IM users eventually forced him to acknowledge that virtually all of the assumptions on which he had based the initial development efforts were, in fact, false.  (Ries, 2011, p. 44)  Those misleading assumptions – and the multiple, costly development dead-ends that followed – could easily have been side-stepped if he had simply started the IMVU development process by engaging existing IM users in a non-directive, open-ended dialogue: “Talk to me about how you use IM…  That’s interesting: tell me more…”

To be fair, Ries does introduce the concept of “genchi gembutsu” (go see for yourself) (Ries, 2011, p. 86) and then goes on to discuss Scott Cook’s random telephone interviews of potential home accounting users  that preceded the launch of Intuit (Ries, 2011, p. 88).  Cook’s initiative is a perfect example of an open-ended, non-directive new category exploration.  But strangely, Ries doesn’t pivot that mirror around to reflect on his own initial stumbles with IMVU – and draw the appropriate conclusions.

Seeing the Forest and the Trees
If you’re allergic to qualitative research, you can still gain insights by analyzing syndicated panel data, but you won’t get the same level of detail or the emotional insight.  Regardless of the approach chosen, the learning and insights come through customer research:  by reflecting on participants’ responses, by reviewing their behaviors, by noting the choices they have made (and may not have made) and finally through those glorious “Aha…!” that hopefully arrive when the patterns clarify. This is not a process of deductive reasoning or brute analytics – though that may be the post-hoc explanation for the conclusions that are developed.  This is an attempt to see the whole by allowing the pieces to fall into place.

Concept Tests

New product concepts can take many forms and, to his credit, Ries mentions several of them.  A favorite example is the video created by Drew Hudson to demo the intended functionality of Dropbox (Ries, 2011, p. 97) – an approach we have used to great effect.

  • Product Concepts
    In its simplest form a product or service concept is a short written statement – generally accompanied by a visual element – that describes what the product or service does, lists a small number of differentiating attributes and (hopefully!) communicates the intended end-user benefit. In more elaborate variants on this theme – which we recommend – one creates a small suite of concept statements or visuals that are crafted to intentionally vary the highlighted functionality and potential end-user benefits.  These concepts are then presented to prospective customers/end-users for evaluation either singly or in rotation depending on the assessment methodology.
  • Presenting the Future
    When presented with these clearly defined alternative versions of a possible new product, target customers can readily tell you which ones (if any) they prefer and why they prefer them. In a very real sense, these alternative concepts statements represent alternative descriptions of a possible future – a future that includes the envisioned new product, but viewed from differing functional and emotional perspectives.  Asking respondents to compare alternatives and indicate a preference is the psychological underpinning of most product development research including conjoint measurement techniques.

Preference Testing
By analyzing preference patterns and their underlying drivers one can identify “what customers want” through inductive logic and inference.  This is possible even in a new category that is largely unfamiliar to the intended target group.  (See q-PFO).  Qualitative, quantitative or combination assessment methodologies can be employed. The final choice depends on the available time, the level of finish that the concepts have attained and the decision-maker’s comfort with specific methods.

Enhancing the B-M-L Validated Learning Loop

Both of the customer-oriented research steps we advocate would occur at the front-end of the “Build-Measure-Learn” loop, prior to the initial “Build” stage.  A diagram for the expanded validated learning loop appears in Figure 2.

[image_frame style=”framed_shadow” align=”center” alt=”Enhanced “B-M-L” Validated Learning Loop” height=”400″ width=”500″]http://starpointgroup.com/wp-content/uploads/2018/04/enhanced-bml-validated-learning-loop.png[/image_frame]

Why Talk to Prospective Customers?

There are three clear and compelling benefits to talking with prospective customers at the earliest stages in the startup development process.

  • To understand the category from the customer’s perspective – thereby quickly dispensing with invalid assumptions that lead down blind alleys at the outset of a project.
  • To quickly isolate the crucial features and essential functionality that prospective uses demand, and to dispense with those they see as secondary or completely unnecessary – thereby creating a blueprint for the MVP and the initial build stage.
  • To identify the most compelling customer benefit and supporting product features (“reasons to believe”) that will constitute the positioning platform for the product’s roll-out. These are crucial elements for developing a marketing campaign that can effectively generate informed customer awareness and demand.

In summary, talking to your prospective customers early in the process can dramatically reduce the uncertainty surrounding the initial stages of any startup activity.  A/B testing and rapid turns of the validated learning loop are great for improving existing features, for prioritizing new features and for enhancing the user interface – once a validated product concept exists.  But at the earliest phases, when it’s all just an entrepreneurial “Idea”, a small investment in upfront customer research will do a lot more to reduce “extreme uncertainty” than will coding hunches and running in circles. Learn more about our Startup Planning process.

Works Cited

Ries, E. (2011). The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radially Successful Businesses. New York: Crown Publishing Group, a division of Random House, Inc.