Decision Management: Are we hitting natural human limits?

It is interesting to observe that it is increasingly difficult to follow a linear decision process where complex institutional projects and initiatives are involved.  We see that there are growing risks and an accompanying tendency to fail when decisions are driven too quickly.  Likewise, when allowed free-reign, decision processes have a tendency to return to primary premises, to re-frame, to re-question, to doubt: analysis paralysis.

The majority of ‘best practice’ decision methods advocate a linear process (i.e.  Hammond’s decision steps outlined in the book ‘Smart Choices’).  However, given increasing scale and complexity, information gleaned later in decision processes can lead to re-framing of fundamentals.  This can lead to ‘analysis paralysis’, something everyone who has worked in large institutions has seen and/or experienced.

Perhaps the methods for software and IT project development offer sound decision management guidance.  Whereas the traditional linear ‘waterfall’ method for software / IT project development has been increasingly discredited, especially given increased scope and complexity in institutional IT projects, new ‘non-linear’ methods of development have emerged: i.e. AGILE, RUP, Extreme Programming.

Perhaps the time is ripe for non-linear decision process methodologies for large, complex organizations and projects which provide for iterative movement, yet admit to the ability to re-frame fundamentals along the ‘decision development path’.  Prospects include such methodological advents as multi-criteria analysis for complex stakeholder goal identification and Real Options Analysis-driven management decision making.

The real challenge, I believe, is not technological or methodological, however.  There are a wealth of insightful methods and powerful decision tools available.  As many commentators have identified (i.e. Pfeffer & Sutton, 2006), the deep and underlying challenge is: ourselves!  Namely, the culture of institutional ‘decision management’ in Western society is based upon a traditional foundation of ‘charismatic experts’. The growing reality is that many institutional projects are outstripping the cognitive and communication capacity of such, albeit often gifted, individuals to effectively steward. It is simply a reality of growing scale and complexity outstripping the native capacity of individuals and small groups to effectively encompass.  With increasingly global and impactful scope (witness the U.S. Mortgage Crisis and subsequent global Financial Crisis), can we afford to leave institutional decision management as it is: a largely ad hoc, idiosyncratic affair?

Scale and the number of stakeholders involved complicates the capacity of vested experts to drive high-quality decisions to fruition: larger organizations and broadened stakeholders (a natural outgrowth of increasingly interlinked processes) increase ‘noise’ and threaten to fundamentally derail linear decision processes.  The reality of modern institutional life is that indeed the scope of organizations and stakeholders are expanding: both in real terms, i.e. larger and broader organizations enhanced by globalization and communication technology; and in virtual senses, i.e. expanded insight and vantage points enhanced by the computationally-driven ability to store, assess, and ‘crunch’ bigger and more complex sets of variables.

Put another, anthropocentric, way: humanity is reaching levels of interactive project complexity which strain the inherent cognitive and communication capacities of naturally evolved ‘tribal’ group decision methods (for a solid look at our origins in this respect, see C. Stringer’s ‘Lone Survivors’ ).  If we look at how many research scientists were involved in particle research (atom smasher) experiments 80 years ago, it was typically 3 – 5 scientists, and perhaps around 40 – 80 worldwide (if that).  The CERN Large Hadron Collider project alone involves more than 10,000 scientists from >100 different countries.  Similarly, most national tax offices are thousands of times larger that they were 80 years ago.  Here in the Netherlands the national tax office had around 60 staff members in 1930.  Today the Dutch government employs almost 250,000 bureaucrats to tax a country of 17 million.  Small and random examples, but meaningful when considering just about ANY impactful institutional endeavor in modern life: the sheer scope, scale, ambition, and number of people involved simply staggers the imagination (of the individual).  This is exactly the central point: we are involved increasingly in endeavors beyond the capacity of any one of us, even the most talented, to realistically encompass in all their rich complexity.

A humble proposal might be for large institutions to consider, more universally, establishing  ‘Centers of Decision Process Expertise’.  Such centers could be both process and practice driven, viewing decisions as ‘projects’ that evolve in iterative but circular way (and overlapping, in a portfolio sense).  The comparison with AGILE or Extreme programming software development becomes more literal in this sense: each decision is a developing initiative that re-frames within a portfolio of overlapping decisions (much as IT projects often overlap and affect one another in terms of scope and objectives).  Indeed such ‘Centers of Decision Excellence’ are an existing trend in high-risk industries such as petroleum exploration, bio-pharma, and semiconductors.  Such centers are responsible for teaching decision methods, proctoring decision processes, and tracking ‘portfolios’ of key decisions, especially where there are overlaps.

Real Options Analysis (ROA), as a methodologogy, is also conducive to this type of ‘overarching’ framework:  the RO premise is based upon the notion that as time moves forward, uncertainty reduces and new possibilities emerge.  The notion is that a manager that adopts the ‘culture’ of Real Options decision making is a manager that is in a constant process of tracking, re-framing, expanding or contracting decision scope to meet the evolving revelation of new information.  This is as opposed to the tradition of the ‘charismatic expert’ that makes a final decision as the result of a linear process and then drives the decision to implemented fruition or ‘dies trying’, which is increasingly the case across a number of spectacular institutional decision ‘fails’ in the last decade…