Now Reading:
How to Know Which Ideas Your Company Should Pursue
Full Article 9 minutes read

How to Know Which Ideas Your Company Should Pursue

Executive Summary

As companies adopt open innovation and crowdsourcing to stay at the forefront of innovation, the challenge of selecting which ideas to pursue is enormous. Research shows that organizations that receive a large number of ideas have difficulty selecting the most original. Idea evaluators often lack the skills needed to recognize the value of an idea. And managers favor ideas from people they know or who somehow resemble them. To evaluate ideas, innovation scholars have often proposed novelty, feasibility, and market potential as the most important criteria for “good” business ideas. However, there is no widely shared notion of what constitutes a good idea or how to rank relative “goodness” because good ideas can have a wide range of benefits. Beyond financial profitability, some ideas can still create value for the company in terms of customer satisfaction, branding, customer loyalty, and so on. But there may be an easier way: evaluate the benefits an idea promises relative to the cost of developing it. A concept that offers only incremental benefit but requires little investment might be more profitable than a fantastic idea that requires a lot of investment. In an approach analogous to value investing, the company’s aim becomes looking for ideas that appear to have a high benefit relative to the cost of development.

Juice Images Ltd/Getty Images

As companies adopt open innovation and crowdsourcing to stay at the forefront of innovation, the challenge of selecting which ideas to pursue is enormous. Research shows that organizations that receive a large number of ideas have difficulty selecting the most original. Idea evaluators often lack the skills needed to recognize the value of an idea. And managers favor ideas from people they know or who somehow resemble them. To evaluate ideas, innovation scholars have often proposed novelty, feasibility, and market potential as the most important criteria for “good” business ideas. However, there is no widely shared notion of what constitutes a good idea or how to rank relative “goodness” because good ideas can have a wide range of benefits. Beyond financial profitability, some ideas can still create value for the company in terms of customer satisfaction, branding, customer loyalty, and so on.

But there may be an easier way: evaluate the benefits an idea promises relative to the cost of developing it. A concept that offers only incremental benefit but requires little investment might be more profitable than a fantastic idea that requires a lot of investment. In an approach analogous to value investing, the company’s aim becomes looking for ideas that appear to have a high benefit relative to the cost of development.

This heuristic, often used to rank investments, is selecting for efficiency, a dimensionless metric that represents how well each assessed unit converts the resources it utilizes (inputs), to outcomes (outputs). Efficiency reflects relative performance, meaning that each unit’s efficiency depends on how competitively that unit ranks against other investments. When adapted to the process of evaluating ideas, efficiency signifies how well an idea converts the resources it needs to realize different types of benefits. Evaluating a large volume of ideas this way requires some investment, but you could automate the process. For instance, you could ask participants on the crowdsourcing platform to evaluate their peers’ ideas based on the different input and output dimensions.

To test how this concept could be used in practice, we analyzed one year of publicly available data from the My Starbucks Idea platform. Starbucks launched this idea-crowdsourcing platform in 2008. Since then numerous customers have submitted ideas for new offerings. (At the time we conducted this research, all ideas were publicly accessible. However, management changed this in 2017; the submitted ideas and comments of other members are no longer visible to outsiders.)

The process starts with members of the platform submitting their ideas, which are then reviewed by a Starbucks team. After the selection is made, the team presents the “champion” ideas to the company’s key decision makers, who are responsible for figuring out how to put the ideas into practice.

To measure idea efficiency, we used a method called data envelopment analysis (DEA). DEA was originally proposed in the seminal paper by Abraham Charnes, William W. Cooper, and Edwardo Rhodes and since has been employed in diverse studies to measure the efficiency of organizational units such as hospitals, police departments, and schools. We used DEA to measure not only the resources each idea requires for its implementation but also how well each idea scores on a variety of objectives relative to other alternative ideas.

For the purpose of this study, we downloaded thousands of ideas but focused eventually on a set of 315 ideas submitted in 2016 in 10 of the 15 categories defined by Starbucks (e.g., “atmosphere and locations,” “building community,” “new technology”). Next, we screened the content and excluded complaints, compliments, and country-related ideas. For the remaining 117 ideas, we collected other metadata such as the date that each idea was posted and the score of each idea. The score represents the acknowledgment that each idea received by members of the My Starbucks Idea platform. More specifically, members can approve or disapprove every idea and the total number of “upvotes” minus “downvotes” received constitutes the overall idea score. (Numbers ranged from –20 to 9,110, with an average score of 366.)

To determine relevant inputs (the resources needed for implementation) and outputs (the desired outcomes) of an idea, we reviewed the academic literature as well as all the selected ideas in our dataset to develop a context-specific coding. Following this iterative process resulted in a final set of five inputs and five outputs. We presented these dimensions to participants of an online survey. Each respondent assessed one set of ideas on their inputs and a different set of ideas on their outputs. Three to 10 different respondents evaluated each idea’s input and output of an idea on a scale between 1 (very negative) and 5 (very positive).

How did our respondents’ judgment compare to Starbucks?

Idea: Allow people to send a gift drink to their friends by text message. “I would like to be able to send a gift drink from my Starbuck’s card or credit card to a friend via SMS or iMessage to cheer them up or tell them I’m thinking of them. Seems like there would be an easy way to do this by generating a gift card receipt.” According to our evaluation, this idea scored particularly high in terms of boosting customer satisfaction and loyalty and improving brand image. Provided that the technological infrastructure already exists to be able to implement this idea and because necessary training of Starbucks employees would likely be minimal, the outputs or benefits of this idea clearly outweigh the necessary inputs or costs to implement the idea, making it highly efficient.
Efficiency survey: Thumbs up.
Starbucks: Thumbs down.

Idea: Vary food menu. “Guy192” felt that the food selection offered by Starbucks was rather repetitive and suggested that it be changed from time to time in order to include both regional and seasonal variations. For example, some regions have their own preferred products that are either part of their culture (e.g., iced coffee) or are produced locally (e.g., pretzels). While this idea scored high relative to other ideas on customer satisfaction, brand image, and brand loyalty, it also scored high on many of the resources necessary to implement the idea relative to others.
Efficiency survey: Thumbs down.
Starbucks: Thumbs up.

Idea: Starbucks mobile truck. One customer suggested that “although Starbucks is available in almost every corner of my city, it would be best if it had a mobile truck, just like an ice-cream truck. This would help workers who do not work near any Starbucks locations, especially for call centers or any type of job that would require the worker to stay up late at night.” People we surveyed on this idea assessed that, relative to other ideas, the resources needed to get this idea off the ground would greatly surpass the possible benefits such as increased customer satisfaction and loyalty.
Efficiency judges: Thumbs down.
Starbucks: Thumbs down.

We don’t know what led Starbucks to reject proposals such as the gift idea — an idea we considered very efficient. Nor do we have information about why a rather inefficient idea — varying the food menu — was implemented. However, after examining the ideas that our respondents considered inefficient that Starbucks did not implement, we can conclude that our method might give judges a more practical way of evaluating ideas consistently — almost like an investment filter.

In other words, our method might not make it easier to pick great ideas, but it should make it easier to avoid picking bad ones. It helps to prioritize ideas or consider which ones should be more thoroughly evaluated. It is also a flexible method: The different input and output dimensions can be adapted as a company sees fit. These dimensions could then be communicated to potential inventors to give them also a better idea of the kinds of ideas that the company considers worth pursuing.

Another advantage of the method is that it enables companies to provide constructive feedback to people who submitted rejected ideas, which is extremely important to encourage future contributions. Prior research by one of us (Dirk) and another colleague has shown that when idea rejections are paired with feedback about the reasons for the rejection, contestants will submit more ideas.

Over the past decade, many companies have come to believe that the ability to innovate is an essential component of their future value. In response, they have tried to encourage stakeholders to think of many new ideas. To create a more functional innovation pipeline, however, they need a more systematic way to filter ideas. Submitting them to an efficiency test could be a good first step.

Input your search keywords and press Enter.