Experiments in Business

For this blog post, I had to read two articles on experiments and how they should be conducted in a business setting. I also watched a brief video explaining the benefits and disadvantages of a scientific study, more specifically it discussed the differences between randomized clinical trial and an epidemiological study. Essentially, a randomized clinical trial is not always possible due to many factors, such as potential dangers to people's health, so epidemiological studies are conducted to search for correlation between two things, such as smoking and cancer.

The first reading, "Finally, A Majority of Executives Embrace Experimentation" (2010) by H. James Wilson and Kevin Desousa, discussed differences between experimentation styles: prototyping, simulation and test groups & A/B testing. It was interesting to consider what types of experiments would be the most useful for different business ideas. I found it interesting that there is a difference between prototyping and simulation, and that prototyping could be done quickly without wasting much time on trying to come up with features that might not work. The second reading, "How to Design Smart Business Experiments" (2009) by Thomas H. Davenport, explained a general way to conduct an experiment for a business idea, starting with how, what, when and why to do experimentation.

I learned that simulation works best with big projects, such as cars, and it enables many people to participate in data gathering. Prototyping works best when there is the possibility to recreate the product again after certain tests, and it is fairly easy to modify. I also learned the steps that should be included in an experiment: creating a hypothesis, designing the test, conducting the test, analyzing the test and acting according to the results.

My question is that why so many companies have not engaged more in experimentation? The piece by Wilson & Desousa claims that in 2010, around 51% of surveyed executives claim experiments are their preferred research method. I find that percentage to be quite low, even though I understand that some business ideas are harder to experiment with. I personally find experiments a great way to study how people see different products and how they work in different situations.

For the Google Online Marketing Challenge, an experiment I would conduct is to see whether our client gets more click-throughs from an ad using the words "gift registry", rather than "wish list". This would be a sort of a A/B test, and it would benefit our client by establishing an AdWords ad that provides a better click-through rate, and which could be showed more often to potential visitors.