Mastering Digital Quality: A Quadrant-Based Approach to Engineering Excellence
By 
Very often organizations get too focused on one approach or the other, especially when it comes to having the right quality engineering practices. The paradox of missing the forest for the trees is quite common.
Our biases, organizational priorities, individual competencies, and culturally accepted norms significantly influence how we approach challenges. Over the years, our experience in creating better software has led us to adapt the “Four Quadrants of Knowledge” framework into the realm of software quality and reliability at QualityKiosk. Engineering for exceptional user experiences demands a multidimensional approach that integrates validation, customer insights, and the discovery of outliers while maintaining robust security.
In quality engineering, identifying and addressing potential issues isn’t just a technical task—it’s a strategic imperative. Quality is not confined to functional behavior outlined in requirements; it’s an inherent driver for shaping requirements and delivering reliable, engaging, and innovative digital products. The Quadrants of Quality framework empowers engineering leaders, product managers, and business owners to transform their approach to quality, transcending the focus from features to a holistic view of the customer experience.
The Four Quadrants of Quality
Let’s begin by revisiting the “Four Quadrants of Knowledge,” popularized by Donald Rumsfeld, within the context of app quality:
- Known-knowns These are app quality issues that we’re aware of and understand. We design tests and can define specific expected results.
- Known-unknowns: These are app quality issues we know exist, but don’t fully understand. We can design tests around these but cannot define what results to expect.
- Unknown-knowns: These are app quality issues that we don’t identify beforehand but instead come to realize based on insights from production data, user feedback etc.
- Unknown-unknowns: These are app quality issues that we’re not aware of and for which we can’t predict the outcomes.
1. Known-Knowns: The Foundation of Predictability
Key QE Strategies to Navigate the Quadrant
- Case-Driven Testing: Writing test cases for well-documented functionalities ensures predictable validation. Approaches like behavior-driven development (BDD) and test-driven development (TDD) enable this effectively.
- Automation Testing: Automating repetitive test cases for APIs, regression, sanity, and contract testing improves efficiency and reduces toil. The quadrant’s clear definition of inputs, outputs, and user behavior makes it ideal for automation.
- SAST & DAST: Tools like Static Application Security Testing (SAST) and Dynamic Application Security Testing (DAST) validate known security vulnerabilities based on predefined inputs (e.g., source code for SAST or application runtime for DAST).
2. Known-Unknowns: Navigating Uncertainty
Key QE Strategies to Navigate the Quadrant
- Performance Engineering: Testing with known inputs such as load and duration while analyzing how the system responds provides actionable insights to improve scalability and reliability.
- Observability: Setting up logs, metrics, and traces helps monitor system behaviour in real-time. Tools like New Relic, Dynatrace, and AppDynamics provide valuable real customer insights. Using the said approach in lower environment enables teams to conduct A/B tests and iterate safely in controlled environments and release better software in the first place.
- End-User Monitoring (EUM): Understanding how users interact with the software allows product teams to prioritize meaningful enhancements based on actual usage patterns.
3. Unknown-Knowns: Uncovering Hidden Insights
Key QE Strategies to Navigate the Quadrant
- Analyzing Historical Data: Revisiting past project data and lessons learned can reveal overlooked patterns or issues.
- Investigating Blind Spots: Observability tools can uncover performance bottlenecks in neglected areas of the codebase.
- Accessibility and Demographics Analysis: Considering factors like user accessibility and demographic impact can uncover areas for improvement, especially in highly interactive software like gaming platforms.
4. Unknown-Unknowns: Embracing Exploratory Innovation
Key QE Strategies to Navigate the Quadrant
- Chaos Engineering: By intentionally introducing failures (e.g., simulating network latency or database crashes), you can identify and fix gaps in system resilience
- Monkey Testing: Through randomized testing, you can uncover unexpected behaviors, often using tools that simulate erratic user inputs.
- Penetration Testing: Conducting both white-box and black-box penetration testing to identify and address unknown security vulnerabilities, such as misconfigurations, unpatched systems, and zero-day exploits.
- Concurrent QE: By embedding quality engineering practices earlier in the development lifecycle through Concurrent Quality Engineering (CQE), you can manage complex interdependencies and interactions, preventing issues from cascading downstream.
Bridging the Quadrants: A Holistic Approach
- Automate Known-Knowns for a solid foundation
- Monitor and Refine Known-Unknowns continuously
- Investigate Unknown-Knowns to unlock hidden potential
- Experiment with Unknown-Unknowns to build resilience
AI’s Role in Optimizing Quality Across Quadrants
As artificial intelligence continues to evolve, it presents remarkable opportunities across the quality engineering spectrum. However, it’s crucial to understand AI’s limitations and strengths in each quadrant.
In the Known-Knowns quadrant, AI excels at optimizing repetitive testing processes through intelligent automation, enhancing test case generation and providing predictive insights. For Known-Unknowns, AI-powered observability tools can uncover performance bottlenecks and provide real-time system insights.
In Unknown-Knowns, AI can analyze historical data to identify overlooked blind spots. While AI has limitations in the highly exploratory Unknown-Unknowns quadrant, it can complement human efforts through AI-generated test scenarios and exploratory suggestions.
Engineering Reliable, Sustainable and Evolving software with Quality at the Core
Like a meticulously designed high-performance vehicle, software systems require a delicate balance of power, adaptability, and user-centric design. The Quadrants of Quality framework provides decision-makers with a strategic lens to distribute focus and resources effectively.
By embracing this multidimensional approach, organizations can transform quality engineering from a reactive process to a proactive, innovative driver of digital excellence.
About the Author
