Noise and Bias in Architecture Decision Making

A recent book written by the Nobel Prize winner Daniel Kahneman and his colleagues is titled “NOISE: A Flaw in Human Judgment”. When I read it, it triggered thoughts about my past experiences doing big technology migrations in telecom and banking sectors when enterprise core systems became subject to replacement rather than routine upgrades. Looking back, what led us to decide to migrate instead of doing incremental upgrade?

 Was I, as the architect helping to oversee these large systems, the only one responsible for the situation or was it related to multiple factors like outdated systems technology, lack of functionality needed to provide new services to our clients, or because we have engaged in extensive customization of the system, including its data architecture that disabled us from normal upgrades?

Most of the time, a multiplicity of factors created considerable complexity in the decision making, such as regarding risk evaluation, lack of support and business knowledge from the software supplier, corporate politics, or lack of funding that led to unwieldy core system evolution. Nonetheless, why do different enterprises with nearly the same enterprise core system evolve with their digital ecosystems differently? Different integration patterns have been used, different tools have been purchased and implemented, different principles have been applied, business and IT skills differed or were just missing, different contractors were influential in the decision-making, licensing policies of suppliers, etc.

The answer is related to architecture governance — who is making such decisions, and how? In short,  some enterprises have more effectively managed their design  and technology debts, and have been upgrading their core systems regularly but some got stuck with an old extensively customized version of the core system (often with a constellation of applications around the system pumping data to supply integrated information for the Business). Such divergence away from the original design of the core systems would infect the flexibility of the  entire architecture landscape. Therefore, at some point it became necessary to undertake a complex migration project. These migration projects are often risky and very demanding on everyone not only on everyone in the enterprise. Unfortunately, some such projects are also abandoned after two or so years of effort (one of my former clients is still left with an old legacy system with hundreds of interfaces as their attempt to replace mainframe-based system almost 40 years old has failed due to various pressures including encroaching legislation requesting constant system updates).

Who is at fault — stakeholders, architects, or both? It is usually hard to pinpoint blame because of the nature of the decision making process. From an architectural perspective, we need to analyze which biases can impact objectivity and rationality, but we also need to understand this noise factor that Daniel Kahneman and his colleagues explain and demonstrate through various case studies in justice, medicine, insurance, or other sectors where decisions taken by people with similar qualifications resulted in unwanted variability and clouded organizations’ judgments. In the digital ecosystem context, this leads to increases in the Total Cost of Ownership of core systems, design and technology debts, delays in the delivery of new services down the line, all of which impact an enterprise’s ability to evolve and adapt to new market requirements.

Most people understand how biases can creep into important decisions but most of them are less aware of how much the noise factor can affect their choice-making, which is highlighted in the book and revealed by noise audits. Can you also imagine that temperature or changing weather conditions may have on the decision-making, risk perception, and risk appetite of stakeholders or architects?

Also in the context of the creeping Agile@Scale transformation of many enterprises, typical decisions are delegated down to self-managed teams, but important and rare decisions remain under the purview of executive management. More thought needs to be given to the roles and responsibilities of architects, to who should be making architecture decisions, to who should have a veto right, to the processes leading to decisions regarding which tools, principles, techniques, and models are used.

More transparency in this area of governance is necessary to lead to more rational decisions, especially given a corporate culture where architects are still only making recommendations. In any respect, those making decisions or even just architecture recommendations, need to upgrade knowledge, skills, and abilities, including regarding tools of the trade such as architecture modeling and diagramming.  Whether we are talking about informal diagrams/graphics or formal models created using leading modeling standards (i.e., ArchiMate, BPMN, UML, SysML or and emerging one such as LabNaf), the key is to enable more effective collaboration, communication, analysis, and decision making. It is important to keep in mind that formal models are less useful for the decision making of decision makers not knowledgeable about such standards. Instead, one should incorporate the use of more informal visuals, such as represented by the Business Model Canvas, or ones customized for various classes of stakeholders. 

Authored By Alex Wyka, EA Principals Senior Associate and Consultant