Article: Tuesday, 12 November 2019

Some business leaders, employees, entrepreneurs, and organisations cope with constant change by crowdsourcing ideas from stakeholders. But which ideas are best for business and how should managers select them? A recent study by associate professor Dr Dirk Deichmann, assistant professor Dr Inga Hoever and former researcher Violetta Rodopoulou, all from Rotterdam School of Management, Erasmus University (RSM), measured the ‘efficiency’ of ideas in a database of crowdsourced suggestions from customers of a high street retail business with a strong online brand. The researchers’ aim was to find a formula or tool for evaluating ideas for their benefits in comparison to the cost of developing and implementing them.

The RSM researchers used suggestions from Starbucks’ idea crowdsourcing platform to test their theory, and created a toolkit for filtering ideas – by submitting them to an efficiency test as a first step.

Ideas and comments on the ‘My Starbucks Idea’ platform were publicly visible until recently. The research was conducted while this was still the case; the researchers selected 315 ideas from 2016 focusing on 10 categories; atmosphere and locations, building community, merchandise and music, new technology, ordering payment and pick-up, other experience ideas, social responsibility, and the Starbucks card.

The researchers used data envelopment analysis (DEA) to generate an ‘efficiency score’ representing the relative capacity of each idea to convert inputs to outputs. For this, they determined appropriate input and output criteria which were relevant in and suited the context of the ‘My Starbucks Idea’ platform. In the end, they developed five input criteria which reflect what would be needed to implement ideas, and five output criteria about outcomes that ideas might produce.

The set of ideas was presented to participants in a survey. Respondents assessed the inputs of one set of ideas, and the outputs of another set on a scale of 1 (very negative) to 5 (very positive). The researchers then checked their respondents’ judgements and compared those with the judgements that came from Starbucks’ management.

"The researchers still suggest that having some quantitative method for measuring the value of crowdsourced ideas for innovating businesses might give a more practical and consistent way of evaluating them"

The outcome? The alignment was not perfect:

1. Allowing customers to send a gift drink to friend via text message

The evaluation of the idea using the researchers’ coding formula showed this idea scored well for boosting customer satisfaction and loyalty, and improving the brand image, providing that the technological infrastructure exists. Staff training costs would be minimal so this is a highly efficient idea according to the researcher’s hypothesis. But apparently Starbucks rejected it.

2. Vary the food menu to time to incorporate more seasonal and regional items

The researcher’s formula scored this idea relatively highly for customer satisfaction, brand image, and brand loyalty, but showed that it would need plenty of resources to put it into action. The researchers’ assessment was that it would be an inefficient idea – but it got a thumbs-up from Starbucks.

3. A mobile Starbucks coffee shop

Like an ice cream van or truck calling at workplaces like call centres or where night shift workers are employed. The researchers’ assessment of this idea using their toolkit showed it would demand more resources to get it off the ground compared to other ideas; spending on such resources would outstrip any potential benefits to customer satisfaction and loyalty. The team’s research classed it as an inefficient idea; Starbucks also rejected it.

Practical and consistent evaluation

The researchers don’t know why Starbucks’ management decided to implement ideas that the toolkit had shown to be inefficient. However, the researchers still suggest that having some quantitative method for measuring the value of crowdsourced ideas for innovating businesses might give a more practical and consistent way of evaluating them – almost like an investment filter.

Initially they also looked into visitors’ votes about these ideas on the ‘My Starbucks Idea’ platform. Interestingly, Starbucks’ judgement didn’t differ significantly from that of external visitors to the site. Generally, the ideas launched and implemented by Starbucks also received a large number of votes by users and thus had a high idea score on the ‘My Starbucks Idea’ platform, but some of the implemented ideas had a surprisingly low efficiency score.

But don’t trust the votes

Previous research shows that while votes may be a cost-effective and fast way to screen and select ideas, they are not a good indication of the quality of an idea. The researchers found that the instinct for reciprocity among contestants makes vote tallies a better indicator of the personal popularity of an inventor than of an idea’s quality and innovativeness. Managers should be wary of using votes as a tool to select ideas, they warned.

While the RSM researchers’ method is slightly more complicated than counting votes, it offers a more nuanced picture of the investment that would be necessary to implement an idea, and an evaluation of the benefits it offers. It allows companies to consider ideas relative to each other.

An efficiency score makes it easier to prioritise and if necessary make a more in-depth evaluation of those ideas that scored above a threshold for efficiency. It’s flexible too, because the input and output dimensions can be adapted as a company sees fit. Ideally these dimensions would then be communicated to potential inventors because it indicates the kinds of ideas that the company considers useful.

Easier to avoid the bad ideas

"Our method might not make it easier to pick great ideas, but it should make it easier to avoid picking bad ones,” said Deichmann. “It helps managers to identify the good ideas, and helps to prioritise them. Companies can adapt the input and output criteria to make it flexible. And it’s easy to show the results of the assessments to potential investors.”

Deichmann also thinks the method is a good way of encouraging people who chip in with their ideas not to give up if their ideas are rejected – because it produces a structure that makes it easier to give feedback.

Dr. Dirk Deichmann

Associate Professor

Rotterdam School of Management (RSM)

Profile picture of Dr. Dirk Deichmann.

Dr. Inga Hoever

Associate professor

Rotterdam School of Management (RSM)

Profile picture of Dr. Inga Hoever.
Pile of books with vibrant bookmarks protruding from various pages, symbolizing in-depth research.
Header image for RSM Discovery, representing the research stories and articles section of the Rotterdam School of Management website.

RSM Discovery

RSM's Research stories in a fun and accessible format. Read the latest insights from the best researchers in the field of business. You can also subscribe to the newsletter to receive a bimonthly highlight with the most popular articles.
  • Subscribe to the newsletter

Want to learn more about this subject?

RSM offers Executive Education and Master programmes in various business areas for any stage of your career. For instance:

Last seats

Leading Change

3-day programme

Young woman posing in front of De Rotterdam, Rotterdam's iconic modern skyscraper.

Personal brand effectiveness

3-day programme

Elegant man posing in front of the Erasmus Bridge in Rotterdam

Corporate Responsibility

3-day programme

Elegant man holding a skyline of skyscrapers connected to a globe in his palm, symbolizing corporate responsibility.
Portrait of Erika Harriford-McLaren

Erika Harriford-McLaren

Corporate Communications & PR Manager

Danielle Baan

Science Communication and Media Officer

Erasmus University campus in autumn, showcasing its iconic red trees, viewed from across the campus pool.