The cost of (not) testing!

During the late 1990’s the Post office, in an attempt to modernise and streamline its operations embarked upon a programme ofupgrading its ageing computer network with an aim to replaceoutdated technology with a state-of-the-art system purported to streamline processes and enhance efficiency. This system is Horizon, and it was deployed across the network between 1999 and 2000.  

Soon after the rollout commenced Postmasters and Postmistresses (essentially independent small business owners who operate community Post Offices) started reporting discrepancies in financial records when using the system.  This led to hundreds of them being accused of being responsible for these discrepancies and led to criminal charges being laid against them with tragic consequences. Approximately 900 workers were wrongly convicted of theft, fraud and false accounting. Some went to prison, others were bankrupted. Suicides and attempted suicides occurred in extreme cases and many marriages fell apart. Many of them were shunned by their communities and have not recovered their standing since.

In 2021 a public inquiry was launched by the UK Government to shed light on the events that have unfolded, one of the key witnesses, a Post Office IT specialist named Jeremy Folkes gave compelling testimony on how systemic errors and lack of rigour around the clarity of requirements and testing of the system led to the software triggering these tragic events. (For those who like to dive into the details the transcripts of the inquiry are here:  https://postofficeinquiry.dracos.co.uk)

Had the Post Office conducted comprehensive testing, it is likely that several key issues could have been unearthed:

·       Software Errors: Adequate testing would have revealed software defects and bugs within the Horizon system, enabling developers to address these issues before deployment. Many of the errors that contributed to false accounting accusations stemmed from underlying software defects that went undetected.

·       Data Integrity: Testing procedures would have scrutinised the integrity of data handling within the Horizon system, it would have highlighted discrepancies in financial records and flagged erroneous transactions.  

·       User Experience: User Acceptance Testing (UAT) would have provided valuable insights into the usability of the Horizon system. Post Office staff could have flagged interface complexities or operational inefficiencies, prompting refinements to enhance user experience and mitigate potential errors.

·       Compliance and Regulatory Standards: Thorough testing would have ensured compliance with regulatory standards and industry best practices. It would have identified gaps in security protocols, data protection measures, and financial auditing procedures, thereby reducing the risk of legal liabilities and regulatory violations.

All of these are seemingly obvious with the benefit of hindsight however achieving these requires careful planning and work in setting up an environment that can succeed.  The key elements for fostering a culture of quality include:

·       Invest in Testing Infrastructure: Allocate sufficient resources and expertise to establish robust testing infrastructure, encompassing a clear and dynamic test strategy, automated testing tools, dedicated testing teams, and quality assurance protocols.

·       Embrace Agile Principles: Adopt agile principles that emphasise iterative testing and continuous feedback loops. Agile frameworks enable organisations to detect and address issues in close to real-time, minimising the likelihood of systemic failures.

·       Foster Collaboration: Cultivate a collaborative environment wherein developers, testers, and end-users work closely together throughout the testing process, ensuring comprehensive test coverage and holistic problem-solving.

·       Prioritise Quality Assurance: Instil a culture of quality assurance wherein testing is viewed more of a strategic priority rather than a formal stage gate before shipping to production.  This will enable a culture that is ideally placed to deliver reliable, resilient, and user-centric solutions.

As organisations navigate technological innovation and operational transformation, they must be ready to continuously evolve and grow by placing users and customers at the heart of everything they do.  By prioritising rigorous testing at every stage of development and deployment, they can uphold stakeholder trust, mitigate systemic risks, and establish a reputation founded on integrity, resilience, and accountability.

At the end of the day, its all about understanding the true cost of failure if your implemented system does not work. For some, it’s a few disgruntled complaints that may not be materially important, for others such as with Horizon there may be far more serious and long-lasting consequences. So far, this case has resulted in GPB 58million, several deaths, failed marriages and ruined community standings. How much is it worth to you not to test?

‍