The Four Elements of Balancing Big Data, Intuition, Decisions, and Results

The Four Elements of Balancing Big Data, Intuition, Decisions, and Results Intelligenthq

The four elements of balancing big data, intuition, decisions, and results
By Dr. Amine Ayad and Dr. Emad Rahim

The publication of “Managers not MBAs” by Henry Mintzberg intrigued our curiosity because in our practical and academic journey, we encountered MBA graduates who were assigned to lead corporate initiatives struggling in managing and leading people. Beyond MBA programs, there was a question that needed answers: How could we solve the strategic predicament of balancing analysis, decisions, action, intuition, and results?

According to the Library of Congress (n. d.), as of March 2014, the Library has collected about 525 terabytes of web archive data (one terabyte = 1,024 gigabytes). The web archives grow at a rate of about 5 terabytes per month. In theory the availability of data should guide decision making, but in reality today’s data can create “blind spots” that lead to errors in judgment. In 1980 Michael Porter used the term “blind spots” to refer to conventional wisdom which no longer holds true, but which still guides business strategies. In today’s data environment, “blind spots” may encompass the complexity of data analysis where managers make decisions relying on partial data and / or incomplete analysis due to the amount of available data. Some propose “big data” as a solution but “big data” is still in its infancy and until big data is within reach to managers and leaders in a manner that Micorsoft’s excel is, managers may need a more realistic guidance to deal with the dual dilemma of balancing analysis and intuition as well as balancing accuracy with speed and results. To that end, we propose the following:

1. Critical thinking and the science of complexity

In 2008, the first author argued that there are numerous examples of catastrophic failures that occurred as a result of selective blindness on the part of senior management personnel. The Esso Longford gas explosion, the Chernobyl disaster, and the loss of both the Columbia and Challenger space shuttles are just few examples of such failures. In life, let alone business, people may see what they want to see, and ignore “evidence” that does not correlate with their own view of the world. In 2006 during an interview with MSNBC, Drew Westen, director of clinical psychology at Emory University, said: Everyone from executives and judges to scientists and politicians may reason to emotionally biased judgments when they have a vested interest in how to interpret the facts.

Sanders (2002), argued that by ignoring the science of complexity and nonlinear thinking small companies are taking big companies by surprise. She wrote in the Washington Post: Linear thinkers tend to rely on past experience to travel from Point A to Point B. Nonlinear thinkers tend to look for changes since the last time they made the trip. A failure to recognize those changes is why blue-chip giants often get caught off guard by small, innovative companies such as Southwest Airlines, Starbucks, IKEA, Old Navy, DirecTV, Hotmail and NetBank. Furthermore, statistics are essential to data analysis but statistics without CT can be fatal. Best (2001) cites the following quotation “Every year since 1950, the number of American children gunned down has doubled” and stated: “Accepting these data without CT leads to mistakes, as the statement is “impossible”. What makes this statistic so bad? Just for the sake of argument, let’s assume that “the number of American children gunned down” in 1950 was one. If the number doubled each year, there must have been two children gunned down in 1951, four in 1952, eight in 1953, and so on. By 1960, the number would have been 1,024. By 1965, it would have been 32,768 (in1965, the F.B.I. identified only 9,960 criminal homicides in the entire country, including adult as well as child victims). By 1970, the number would have passed one million; by1980, one billion (more than four times the total US population in that year). Only three years later, in 1983, the number of American children gunned down would have been 8.6billion (nearly twice the earth’s population at the time). Another milestone would have been passed in 1987, when the number of gunned-down American children 137 billion would have surpassed the best estimates for the total human population throughout history 110 billion. By 1995, when the article was published, the annual number of victims would have been over 35 trillion; a really big number of a magnitude you rarely encounter outside economics or astronomy.

In addition, in business statistics and numeric data cannot tell the whole story; not only because results can be achieved at the expense of long-term sustainability, or at the expense of factors that are not a measured during the analysis such as morale, culture, and/or corporate politics.

Interestingly, Jim Phillips, while chairman and executive director of FedEx Institute, recalled his conversation with Bill Gates: “In the early days I was having lunch with Bill Gates, and we were talking about how hard it was to find people, and he said he was having great success with philosophy students”. They think logically (Sherard, 2002).

The Four Elements of Balancing Big Data, Intuition, Decisions, and Results IntelligenthqDiversity and inclusion

2. Diversity and Inclusion

To that end, diversity of thought and personality preferences should be emphasized. Imagine a team where all members are inclined to behave in a collaborative manner at the expense of data and / or a team where all members are inclined to emphasize data at the expense of the emotional need of the situation or at the expense of collaboration.   When teams conform to certain behaviors as they deal with information and data “blind spots” are inevitable and can be very costly. Without a strong culture that welcomes constructive conflicts and constructive criticism of data analysis, a culture of opinion justification may thrive. We define opinion justification as the consciousness and unconsciousness inclination to select data and / or analysis that justifies self and / or leaders’ opinions. Diversity and inclusion help promote a culture of openness, trust, and tolerance that are needed to challenge popular opinions that are no longer valid as well as erroneous analysis and opinions.

3. Training

In 2011, Manyka et al. predicted: “There will be a shortage of talent necessary for organizations to take advantage of big data. By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions”. Managers are not born with sophisticated analytics, and many analysts that we’ve encountered are not trained managers. When statisticians are hired to analyze data without proper context they tend to make severe errors because they may include irrelevant data and / or ignore critical relevant information beyond the data. Emotional biases that we mentioned earlier is only one form of biases that skew decision and we are not aware of MBA graduate programs that emphasize biases such as belief bias, confirmation bias, anchoring bias, self-serving bias, status quo bias, survival bias, etc. Furthermore, training can define the boundaries and domains of intuition to complement data analysis and decision making.

4. Speed and Quality

Finally, we propose results as the balancing mechanism between speed and quality. In other words, stop gathering and analyzing data once you feel what you have in terms of data can actually improve the situation within your expected return of investment guidelines. In today’s big data environment, successful leaders seek improving the situation and strong correlations may work faster and better than delays attempting to find the root cause of the problem and designing solution for complex “root causes” just to discover that one set of data was missing and the presumed root cause was not the real cause of the problem missing opportunities to improve the situation based on the strong correlation. If the problem is severe and gradual improvements may not justify return on investment, then critical thinking coupled with functional and intuitive knowledge may help define a transformational approach and / or a gradual change management approach.

In summary, the tools of the industrial revolution including total quality management as well as mechanical and linear thinking can still be useful especially if coupled with critical thinking, diversity and inclusion, and proper training that allow for fast, intuitive and data driven decision making while delivering improved results within defined return on investment.

References

Ayad, A. (2010). Critical thinking and business process improvement. Journal of Management Development. Vol. 29 Iss: 6, pp.556 – 564

Best, J. (2001). The Chronicle Review. Retrieved from:

Library of Congress (n. d.). Retrieved from:

Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A. (2011). Big data: The next frontier for innovation, competition, and productivity. Retrieved from:

MSNBC (2006). Political bias affects brain activity. Retrieved from: www.msnbc.msn.com/id/11009379

Porter, M. E. (1980). Competitive Strategy. NY: Free Press.

Sanders, I. (2002). To fight terror, we can’t think straight. Washington Post. Retrieved from:

Sherard, S. (2002). Technology center to serve as hub for U of M academics. Memphis Business Journal. Retrieved from: www.bizjournals.com/memphis/stories/2002/07/22/story3.html