BrainTrust Query: The Hawthorne Effect (Maybe) Debunked
By Bill Bittner, President, BWH Consulting
In labor productivity studies, they call it
the Hawthorne Effect. In more general terms, it’s called
the Observer Effect. In both cases, the conclusion is
that the mere act of observing something affects the outcome. The
Hawthorne Effect has been an important consideration in quantifying the impact
of process change for over 85 years. Now The
Economist reports that a new analysis of the original
data casts doubt on the initial conclusions.
The Hawthorne Effect was derived from observations
of workers conducted at a large telephone industrial plant outside Chicago
in 1924 known as the Hawthorne Plant. The original goal
was to measure the impact of improved lighting, but instead reached the conclusion
that just knowing they were being observed affected the way workers performed.
There was never any detailed econometric study
of the original data from the Hawthorne Plant. However,
because it was so intuitively reasonable, the conclusion was widely accepted,
quoted and factored into research methodology. Now two
economists, Steven Levitt and John List, at the University
of Chicago have discovered the original data and conducted the analysis. It
turns out that electricians changed the lighting on Sundays while the plant
was closed, so workers arrived to an improved lighting environment on Monday
mornings. But looking at other data for periods when
the lighting did not change also revealed improved productivity on Mondays. It
led the economists to another obvious conclusion: workers are more productive
at the beginning of the week when they are fresh. Other
factors also impacted the study, so the effect of being observed could not
be isolated.
Retailers are constantly trying to extract information
from the raw data they receive on labor productivity and customer purchases. This
finding reveals how difficult it can be to distinguish between correlation
and causation. Few would argue that the conclusions we
reach from raw data are often influenced by our own experiences and in some
cases this leads to insightful observations. However,
in other cases, it almost undoubtedly leads to missteps and the wrong the
conclusions.
Discussion Questions:
Are workers affected by being observed as postulated from the original
Hawthorne project? How should personal experience be factored into conclusions
drawn from data analysis? How can we do this without projecting our expectations
on the results?
[Author’s Commentary]
On the first question, I am reminded of a store labor study we did that
observed what everyone in the store was doing every 15 minutes. Everyone
was asked to work as usual. At
one point, when the observers could not find the night crew at around 3:00
AM, they followed the aroma of barbecue steak to the basement where the
subjects were happily preparing their lunch while enjoying some local marijuana. (The
store was in New Orleans.) Obviously,
the observers had allayed their concerns over being observed.
As far as the other
questions, I am reminded of an advertising professor in graduate school
who pointed out the difficulty of putting yourself in the shoes of your
ad audience. For example,
he said that less than half the population had read a book over the previous
year, a seemingly impossible fact to a classroom of students. It
is often easy for us to see patterns in data that support our preconceived
notions of the world. But
this is also the “art” that a long time category manager brings to their
job as they interpret the data. It
often takes a combination of raw data and experience to really develop
insight.
Whatever the statistical
validity of the Hawthorne study, it is still intuitively pleasing.
I think the Hawthorne Effect is perfectly valid. We see it all the time in real life and even in some training games we play with participants. In one game, we have the groups conduct an activity several times, without being timed. The moment we tell them they are being timed, they dramatically improve their performance.
But, to have real impact on performance, you can’t just observe. You need to add an element of accountability. Once individuals know they are being measured, and that there are consequences (both positive and negative) for their actions, performance always improves.
Accountability is more the key. If you have systems and processes in place that give full accountability, it will replace the need for observation. If the associate knows that you can track every motion and every step back to them, it is, in effect, equivalent to have Big Brother looking down and observing.
The question of how being watched changes behavior is self evident for MOST people, but not all. Knowing that there are cameras within most public places prevents certain acts from occurring, but there are still some among us who do not think that will ever catch up with them. Even though cars have windows, there are still some people who tend to personal grooming, hygiene or worse while in full view of other motorists. Being watched DOES change behavior–though it may not account for all of the change seen in the Hawthorne Study.
As far as the second part of the question–removing personal bias–the best way is to have a double blind study where neither subject nor experimenter know what condition is being observed. However, turning businesses into laboratories is impractical in most instances, so having multiple raters or reviewers is a partial solution (in the hopes that they will not be blinded by the same biases…again, it is only a partial solution). For an interesting discussion on how scientists unintentionally skew results of experiments or allow cultural assumptions to color their findings, read, “The Structure of Scientific Revolution” by Thomas Kuhn. The discussions around Paradigm Shifts is illuminating.
Questions: Are workers affected by being observed as postulated from the original Hawthorne project? How should personal experience be factored into conclusions drawn from data analysis? How can we do this without projecting our expectations on the results?
The author’s commentary at the end of the article is a solid summation of how to analyze data. Having begun my work in sociology I understand the need to maintain objectivity when instincts want to direct the conclusions.
Unfortunately many retailers don’t have this background and prefer to mold data to meet their preconceived and often preferred conclusions. It’s why “dashboards” are so popular. They minimize the need to think about the numbers.
With regard to the Hawthorne Effect itself, I’m not sure that the Chicago professors saw all the data, or know enough about the actual study to form their conclusions.
Coincidentally, it was in Chicago when I worked for Quaker Oats that we conducted an experiment to see if workers would perform better with “white noise” or Muzak. The company had just eliminated cubicles and employees had complained about hearing their neighbors and being overheard, along with the general distractions the new format created.
Unobtrusively (over the weekend) Muzak was installed one week and the “white noise” alternative the following week. The follow up interviews helped us determine the preference, but also revealed that the employees knew what was being done, even though the intent was to hide it from them.
So I’m wondering if the Hawthorne plant employees were really unaware of what was happening.
When I’ve conducted face to face interviews over the years, I’ve seen how delighted respondents are that someone wants their opinion, their personal opinion based on their own experience. They are so willing to provide their point of view; they often forget they said they only had a few minutes and want to stay longer. It’s not a duplicate of the Hawthorne effect, but you can see the similarities.
Supervisors have learned that paying more attention to individual members of their staff can pick up morale and productivity. Yes, these examples are anecdotal, but at the same time I don’t believe we have the whole story on Hawthorne. Or maybe I just want to believe what I learned sooooo long ago.
Ever hear of “home field advantage?” Teams are more likely to win because they have 50,000 observers most of whom are projecting good intentions into their observation.
What happens when your guests all focus on your three year old? Often the child will start acting silly in reaction to the attention coming from the act of observing. How many mothers have said about a child who is acting out: “Just ignore him and he’ll stop?” In other words even stopping the observation changes behavior.
As we know everything is energy and our entire lives are one constant engagement in universal energy. So our energy cannot help but impact on the whole; that’s just a principle of nature. When something happens in nature, all nature adjusts. What we’re learning in the quantum realm now is that this impact on the energy around us can be deliberate and done with intention. Your “observation” of your own health, relationships, prosperity can make a huge difference to how your reality unfolds. One who observes the world as lack and misery experiences lack and misery. One who observes it as filled with abundance and joy will make that reality happen.
If those who work in retail would observe their world differently, they’d see changes almost instantly.
“Observe” a wonderful July 4th!
This truth extends all the way to the world of quantum physics. There is an axiom that says you can either know a particle’s velocity or its location. You cannot know both. By observing that particle, you change one or the other.
Makes sense it would extend out to the world of human nature.
Of course, observation affects behavior or outcomes. If it did not then we could eliminate many management positions at retail or elsewhere. This is as others have noted a well established principle in physics, but surely applies here. It is also why the old saw of “managing by walking around” if often effective. Workers want to do the right thing and if they see management doing and encouraging those things, and know that they will be seen…the results improve. Of course, this works both ways, and if management projects poor habits then their workers will do so as well. Execution is most everything.
Simple question: does observation change behavior? Simple answer: when was the last time you were speeding and saw a police car on the side of the road? What was your behavior?
The Hawthorne Effect is alive and well. Consider the Reality Television shows all over the spectrum from the big 4 networks to the cable and satellite channels. These shows are everywhere and the more they are promoted via advertising the more outrageous they become. The things people will do to be noticed are endless.
The things people will do if they know they are being observed are aligned with their environment and expectations of their performance. At work, people will do their jobs at an effective level aligned with productivity expectations. Performing karaoke in a bar observed by numerous, delirious (by whatever means) lovers of music is like throwing gasoline on a raging bonfire.
Drivers pick their noses with impunity (and with phalanges). Ring a bell? They don’t think they’re being observed, and they’d never do it if they knew someone was watching. When our children were little, the game of spotting these drivers during our road trips was the favorite of all in-car travel games. Very appealing to kids for obvious reasons. Hilarity ensues. Of course observation influences behavior and performance. Do we really need research to verify that up is the opposite of down?
As a former District Manager and then Corporate Store Operations executive I can say that the Hawthorne Effect is alive and well. Any time we would travel to a store the store teams always had their best associates scheduled and spent extra time cleaning the store and making sure it looked “right.” No matter how much we instructed the teams to maintain their current business practices they always went a step above.
The trick was getting those same teams to want to put on the same “show” for the clients that were there every single day.
There is no question that the act of observation influences the observer–after all, I have never been in a focus group where parent admitted that they fed their kids sugared cereals. It is clear that the peer pressure and the act of being observed (and recorded) influenced their answers. Otherwise there would never be a market for Sugar Pops!
At the same time, as quantitative marketers, we have to be aware of the trap of assuming that we represent the marketplace. Correlation is NOT causality, and we can easily find the relationships in data, particularly if the data is large enough, that can justify our belief and actions.
To reduce that chances of spurious connections in data leading to major marketing decisions, we need two things: (1) a healthy skepticism about relationships (always question assumptions, especially our own!), and a skillful data analyst who is willing to point out the fallacies of our ways.
Even then, be careful–the world does not work the way we want, no matter how much we want it to….