Observing Consumers Beats Listening to Them
Bolt, co-founder and chief executive of user research firm Bolt Peters,
argues that while tech entrepreneurs are encouraged to listen to potential
customers, observing is a much better tool to see whether a new product
will succeed in the market.
main problem with opinions is self-reporting bias,” wrote Mr. Bolt in a
column in the venture-capital blog, VentureBeat. “Opinions
are often inconsistent with behaviors or other attitudes, especially when
an example, he noted that Clippy, Microsoft’s animated paperclip helper,
came about after the company’s researchers found that when asked, users
roundly agreed that they wanted help working with their documents.
once people started actually using it in the real world, they hated it
— it might be one of the most hated features in the history of computing,”
said Mr. Bolt.
Bolt offered three ways tech entrepreneurs can gauge whether people would
use a service:
- Test ideas early by watching behavior: He
suggests having eight people interact with a prototype or even wireframes
or design makeups. In the tech world, a number of websites (Chalkmark,
Pidoco, Balsamiq, etc.) allow companies to easily test prototypes. Wrote
Mr. Bolt, “You can still ask all your needy questions about what
they think after the session — just don’t take those too seriously.”
- Get all stakeholders to watch the research: Technical
and business constraints obscure the basic question of whether the interface
is any good.
- Use unorthodox methods: Mr.
Bolt noted that Apple claims it doesn’t conduct user research but releasing
products in generations provides the company with loads of reviews, task-specific
complaints, crash reports, customer support issues, and Genius Bar feedback. “It’s
audacious, large-scale behavioral research,” Mr. Bolt wrote. Similarly,
long-beta testing periods have helped launch services such as Gmail.
Bolt concluded, “Do whatever you need to do to understand how people
use your product. If it’s a device meant to be used in cars, watch people
use it cars; if it’s a video game, avoid sterile lab environments. Just
don’t ask perfunctory, cookie-cutter survey questions to your potential
customers, and expect that to ensure your product’s usefulness.”
Questions: What are the merits of observation versus listening in testing
new products? What are some ways researchers can incorporate observational
research alongside listening techniques?