[Image of: RetailWire Logo and Tagline (for print)]

BUSINESS TIPS

Febreze:
Breathing New Life Into an Under-Served Category
SymphonyIRI Group:
Shopper-Centric Execution
DemandTec:
Demand-Driven Retail Strategies
Nestle Purina:
Center Store/Pet Category
MarketingLab:
iShopper Marketing Evolution
IBM:
Enterprise Marketing Management
Nature Made:
Vitamin Category
Precima:
Shopper-Centric Retailing
AT&T:
Experiencing Mobile Barcodes
[12 comments]

BrainTrust Query: The Hawthorne Effect (Maybe) Debunked

July 2, 2009

By Bill Bittner, President, BWH Consulting

In labor productivity studies, they call it the Hawthorne Effect. In more general terms, it's called the Observer Effect. In both cases, the conclusion is that the mere act of observing something affects the outcome. The Hawthorne Effect has been an important consideration in quantifying the impact of process change for over 85 years. Now The Economist reports that a new analysis of the original data casts doubt on the initial conclusions.

The Hawthorne Effect was derived from observations of workers conducted at a large telephone industrial plant outside Chicago in 1924 known as the Hawthorne Plant. The original goal was to measure the impact of improved lighting, but instead reached the conclusion that just knowing they were being observed affected the way workers performed.

There was never any detailed econometric study of the original data from the Hawthorne Plant. However, because it was so intuitively reasonable, the conclusion was widely accepted, quoted and factored into research methodology. Now two economists, Steven Levitt and John List, at the University of Chicago have discovered the original data and conducted the analysis. It turns out that electricians changed the lighting on Sundays while the plant was closed, so workers arrived to an improved lighting environment on Monday mornings. But looking at other data for periods when the lighting did not change also revealed improved productivity on Mondays. It led the economists to another obvious conclusion: workers are more productive at the beginning of the week when they are fresh. Other factors also impacted the study, so the effect of being observed could not be isolated.

Retailers are constantly trying to extract information from the raw data they receive on labor productivity and customer purchases. This finding reveals how difficult it can be to distinguish between correlation and causation. Few would argue that the conclusions we reach from raw data are often influenced by our own experiences and in some cases this leads to insightful observations. However, in other cases, it almost undoubtedly leads to missteps and the wrong the conclusions.

Discussion Questions: Are workers affected by being observed as postulated from the original Hawthorne project? How should personal experience be factored into conclusions drawn from data analysis? How can we do this without projecting our expectations on the results?

[Author's Commentary] On the first question, I am reminded of a store labor study we did that observed what everyone in the store was doing every 15 minutes. Everyone was asked to work as usual. At one point, when the observers could not find the night crew at around 3:00 AM, they followed the aroma of barbecue steak to the basement where the subjects were happily preparing their lunch while enjoying some local marijuana. (The store was in New Orleans.) Obviously, the observers had allayed their concerns over being observed.

As far as the other questions, I am reminded of an advertising professor in graduate school who pointed out the difficulty of putting yourself in the shoes of your ad audience. For example, he said that less than half the population had read a book over the previous year, a seemingly impossible fact to a classroom of students. It is often easy for us to see patterns in data that support our preconceived notions of the world. But this is also the "art" that a long time category manager brings to their job as they interpret the data. It often takes a combination of raw data and experience to really develop insight.

Whatever the statistical validity of the Hawthorne study, it is still intuitively pleasing.

Discussion Questions



While we value unfettered opinion, we urge you to show respect and courtesy for people or companies about whom you comment. Keep in mind that this is a public, professional business discussion. RetailWire reserves the right to edit or refuse the publication of remarks that we deem unsuitable. We may also correct for unintended spelling and grammatical errors.

Instant Poll
In your opinion, how valid is the Hawthorne Effect theory (i.e. that research subjects are influenced by the knowledge they're being observed)?





To participate in this QuickPoll, please enter your email address:

You may avoid this prompt in the future by registering / logging in.

Comments:

I think the Hawthorne Effect is perfectly valid. We see it all the time in real life and even in some training games we play with participants. In one game, we have the groups conduct an activity several times, without being timed. The moment we tell them they are being timed, they dramatically improve their performance.

But, to have real impact on performance, you can't just observe. You need to add an element of accountability. Once individuals know they are being measured, and that there are consequences (both positive and negative) for their actions, performance always improves.

[Image of: View Braintrust Panelist button]
Kevin Graff, President, Graff Retail

Accountability is more the key. If you have systems and processes in place that give full accountability, it will replace the need for observation. If the associate knows that you can track every motion and every step back to them, it is, in effect, equivalent to have Big Brother looking down and observing.

Susan Rider, President, Rider and Associates, LLC

The question of how being watched changes behavior is self evident for MOST people, but not all. Knowing that there are cameras within most public places prevents certain acts from occurring, but there are still some among us who do not think that will ever catch up with them. Even though cars have windows, there are still some people who tend to personal grooming, hygiene or worse while in full view of other motorists. Being watched DOES change behavior--though it may not account for all of the change seen in the Hawthorne Study.

As far as the second part of the question--removing personal bias--the best way is to have a double blind study where neither subject nor experimenter know what condition is being observed. However, turning businesses into laboratories is impractical in most instances, so having multiple raters or reviewers is a partial solution (in the hopes that they will not be blinded by the same biases...again, it is only a partial solution). For an interesting discussion on how scientists unintentionally skew results of experiments or allow cultural assumptions to color their findings, read, "The Structure of Scientific Revolution" by Thomas Kuhn. The discussions around Paradigm Shifts is illuminating.

Questions: Are workers affected by being observed as postulated from the original Hawthorne project? How should personal experience be factored into conclusions drawn from data analysis? How can we do this without projecting our expectations on the results?

[Image of: View Braintrust Panelist button]
David Zahn, Owner, ZAHN Consulting, LLC

The author's commentary at the end of the article is a solid summation of how to analyze data. Having begun my work in sociology I understand the need to maintain objectivity when instincts want to direct the conclusions.

Unfortunately many retailers don't have this background and prefer to mold data to meet their preconceived and often preferred conclusions. It's why "dashboards" are so popular. They minimize the need to think about the numbers.

With regard to the Hawthorne Effect itself, I'm not sure that the Chicago professors saw all the data, or know enough about the actual study to form their conclusions.

Coincidentally, it was in Chicago when I worked for Quaker Oats that we conducted an experiment to see if workers would perform better with "white noise" or Muzak. The company had just eliminated cubicles and employees had complained about hearing their neighbors and being overheard, along with the general distractions the new format created.

Unobtrusively (over the weekend) Muzak was installed one week and the "white noise" alternative the following week. The follow up interviews helped us determine the preference, but also revealed that the employees knew what was being done, even though the intent was to hide it from them.

So I'm wondering if the Hawthorne plant employees were really unaware of what was happening.

When I've conducted face to face interviews over the years, I've seen how delighted respondents are that someone wants their opinion, their personal opinion based on their own experience. They are so willing to provide their point of view; they often forget they said they only had a few minutes and want to stay longer. It's not a duplicate of the Hawthorne effect, but you can see the similarities.

Supervisors have learned that paying more attention to individual members of their staff can pick up morale and productivity. Yes, these examples are anecdotal, but at the same time I don't believe we have the whole story on Hawthorne. Or maybe I just want to believe what I learned sooooo long ago.

[Image of: View Braintrust Panelist button]
Joan Treistman, President, The Treistman Group LLC

Ever hear of "home field advantage?" Teams are more likely to win because they have 50,000 observers most of whom are projecting good intentions into their observation.

What happens when your guests all focus on your three year old? Often the child will start acting silly in reaction to the attention coming from the act of observing. How many mothers have said about a child who is acting out: "Just ignore him and he'll stop?" In other words even stopping the observation changes behavior.

As we know everything is energy and our entire lives are one constant engagement in universal energy. So our energy cannot help but impact on the whole; that's just a principle of nature. When something happens in nature, all nature adjusts. What we're learning in the quantum realm now is that this impact on the energy around us can be deliberate and done with intention. Your "observation" of your own health, relationships, prosperity can make a huge difference to how your reality unfolds. One who observes the world as lack and misery experiences lack and misery. One who observes it as filled with abundance and joy will make that reality happen.

If those who work in retail would observe their world differently, they'd see changes almost instantly.

"Observe" a wonderful July 4th!

[Image of: View Braintrust Panelist button]
Ian Percy, President, The Ian Percy Corporation

This truth extends all the way to the world of quantum physics. There is an axiom that says you can either know a particle's velocity or its location. You cannot know both. By observing that particle, you change one or the other.

Makes sense it would extend out to the world of human nature.

[Image of: View Braintrust Panelist button]
Paula Rosenblum, Managing Partner, RSR Research

Of course, observation affects behavior or outcomes. If it did not then we could eliminate many management positions at retail or elsewhere. This is as others have noted a well established principle in physics, but surely applies here. It is also why the old saw of "managing by walking around" if often effective. Workers want to do the right thing and if they see management doing and encouraging those things, and know that they will be seen...the results improve. Of course, this works both ways, and if management projects poor habits then their workers will do so as well. Execution is most everything.

Robert Heiblim, Principal, RH Associates

Simple question: does observation change behavior? Simple answer: when was the last time you were speeding and saw a police car on the side of the road? What was your behavior?

[Image of: View Braintrust Panelist button]
Mel Kleiman, President, Humetrics

The Hawthorne Effect is alive and well. Consider the Reality Television shows all over the spectrum from the big 4 networks to the cable and satellite channels. These shows are everywhere and the more they are promoted via advertising the more outrageous they become. The things people will do to be noticed are endless.

The things people will do if they know they are being observed are aligned with their environment and expectations of their performance. At work, people will do their jobs at an effective level aligned with productivity expectations. Performing karaoke in a bar observed by numerous, delirious (by whatever means) lovers of music is like throwing gasoline on a raging bonfire.

Steven Collinsworth, National Category Manager Petfood, Advantage Sales & Marketing

Drivers pick their noses with impunity (and with phalanges). Ring a bell? They don't think they're being observed, and they'd never do it if they knew someone was watching. When our children were little, the game of spotting these drivers during our road trips was the favorite of all in-car travel games. Very appealing to kids for obvious reasons. Hilarity ensues. Of course observation influences behavior and performance. Do we really need research to verify that up is the opposite of down?

[Image of: View Braintrust Panelist button]
M. Jericho Banks PhD, President, CEO, Forensic Marketing LLC

As a former District Manager and then Corporate Store Operations executive I can say that the Hawthorne Effect is alive and well. Any time we would travel to a store the store teams always had their best associates scheduled and spent extra time cleaning the store and making sure it looked "right." No matter how much we instructed the teams to maintain their current business practices they always went a step above.

The trick was getting those same teams to want to put on the same "show" for the clients that were there every single day.

Scott Knaul, Director, Retail Strategic Services, Workforce Insight

There is no question that the act of observation influences the observer--after all, I have never been in a focus group where parent admitted that they fed their kids sugared cereals. It is clear that the peer pressure and the act of being observed (and recorded) influenced their answers. Otherwise there would never be a market for Sugar Pops!

At the same time, as quantitative marketers, we have to be aware of the trap of assuming that we represent the marketplace. Correlation is NOT causality, and we can easily find the relationships in data, particularly if the data is large enough, that can justify our belief and actions.

To reduce that chances of spurious connections in data leading to major marketing decisions, we need two things: (1) a healthy skepticism about relationships (always question assumptions, especially our own!), and a skillful data analyst who is willing to point out the fallacies of our ways.

Even then, be careful--the world does not work the way we want, no matter how much we want it to....

[Image of: View Braintrust Panelist button]
Mark Price, Managing Partner, M Squared Group, Inc.

Follow Us...
[Image of:  Twitter Icon] [Image of:  Facebook Icon] [Image of:  LinkedIn Icon] [Image of:  RSS Icon]

Welcome to the new RetailWire!
Send your FEEDBACK so we can keep the improvements coming.