A number of federal government initiatives in recent years have pushed federal agencies toward data-driven decision-making. Following passage of bills such as the GPRA Modernization Act of 2010 and the Foundations for Evidence-Based Policymaking Act, the GAO found an increase in the use of performance information by agency managers. More recently, the Biden Administration’s Executive Order on improving the federal customer experience included a requirement that agencies must annually assess and report their performance using a number of measurements such as experiential data analysis.
These efforts are music to the ears for those working in the federal information technology space, as data-driven decision making helps address some of the more challenging discussions around prioritization and budget allocation at tactical, organizational, and strategic levels. However, the process to become data-driven is rarely clear cut.
To help, this post will discuss the ideal data-driven approach to decision making: augmented intelligence.
The Federal Agency Decision Matrix
When acting on the laws and orders mentioned above, federal leaders make decisions based on the information they’ve received to date. That information can be broadly classified along two axes:
- Subjectivity, which relates to an individual’s intuition based on their pedigree, perspective, and personal/professional experiences
- Objectivity, which relates to a machine’s algorithms, data, data quality, and computing power with near zero of a human’s emotions
Understanding the above allows us to plot a hypothetical federal agency’s data-driven decision making capability into a classic 2x2 matrix:
In the top right quadrant, you’ll see that Augmented Intelligence represents the ideal decision-making approach. This decision-making style blends human intuition and machine logic into a balanced decision-making framework, but it is not easy to achieve. Let’s discuss each quadrant to understand what improving data-driven decision making looks like.
The Quadrants
Happenstance
Organizations in the lower left quadrant have not yet gathered a significant amount of input from humans or machines, and so decisions, or the lack thereof, are primarily based on hope. Without focus from either humans or machines, organization’s “hope” the outcome is in their favor. This “dart-throwing” approach results in decisions made with little understanding of how the outcomes will affect the larger picture. Leaders that employ Happenstance decision making run the risk of budget overruns, costly rework, or cycles wasted on irrelevant tasks. As a result, decision makers in the lower-left quadrant may find themselves in a reactive state with frequent disagreements about priorities, requirements, and spending.
The solution to happenstance decision making is to gather more data. However, doing so is not the cure-all that it might seem.
Too Objective/Too Subjective
When approaching the challenge of data gathering, one instinct may be to build or install tracking and analytics systems in order to gather as much data as possible. Examples would include installing an analytics suite to track website usage and drop-off, or a ticketing system to record the volume and resolution status of customer requests. Once a baseline was established, decisions on how to improve an agency’s customer experience could then be made based on the highest-volume issues observed.
An agency could also seek to rely more on human input. This approach relies solely on the human experience where their biases, perspectives, professional/personal experiences drive the decisions. The output will likely contain blind spots and an organization would still find itself in a reactive state.
Crucially, agencies operating in both of these quadrants have added more input but have not necessarily improved decision making.
Moving too far down the machine-driven axis can create outputs where there is a lack of context and understanding. The machine only calculates the data you provide it, and therefore may provide an answer but it will lack the necessary ethics that only humans can bring to a decision. Decisions driven purely by data metrics also run the risk of mistaking action for improvement. A task may be assigned and completed based on its perceived severity on a scorecard, but accomplishing that task may not have advanced the agency toward its strategic goals.
Gathering too much human input carries similar risks, but different arguments. In an environment where a large number of stakeholder opinions exist, the tendency can be to serve those who speak loudest first. You may also run into group-think or deference to the ranking individual. Prioritization discussions can devolve into zero-sum arguments where one party feels that they must “win” the discussion in order to show that they were right, but all parties have an equal claim on data validity due to the subjective nature of human input. Or as Jim Barksdale, the former CEO of Netscape said, “If we have data, let's look at the data. If all we have are opinions, let's go with mine.”
Both of these situations allow decision makers to point to process improvements, but they have not yet achieved true augmented intelligence. For that, we need to unite both quadrants.
Achieving Augmented Intelligence
Agencies reach the top right quadrant when they combine human and machine inputs to create what I call Augmented Intelligence. Augmented intelligence is the convergence of human intuition with machine logic, and it is achieved through several steps:
- Define the critical business objective the agency must meet
- Identify and validate the machine and human inputs you have that show performance against the objective
- Fill any data gaps (via incorporation of additional constituency perspectives, the addition of data sets from various related systems, etc.)
- Define your decision process and begin its execution.
Augmented Intelligence benefits federal agencies in several ways:
- Win-Win Prioritization: having data on the largest customer satisfaction issues an agency faces at a leaders’ fingertips makes the prioritization of improvement initiatives much more straightforward. It also helps disparate stakeholders like administration representatives, agency leaders, project managers and federal accountants come together to find the most efficient solutions.
- More Focused Budgeting: Augmented intelligence’s ability to surface and prioritize the largest challenges an agency faces makes budgeting more focused. Gone are subjective arguments about which programs should be budgeted and whether they’ll be an impact. Instead, priorities can be quickly understood and slotted into budgets.
- Improved Conflict Resolution: large projects aren’t easy, and when challenges arise augmented intelligence helps decision makers focus on the data, not the person. Being more objective and less subjective in these instances helps people see and understand the issue, why a decision is needed, why a choice was chosen, and what the second, third and fourth order effects of a decision might be.
What all these elements lead to is something we call the optimized organization.
“Optimized” agencies are those that continually leverage intelligence from both their data and their decision makers in order to better manage agency resources like staff and budget. These agencies have a pre-defined understanding of how to maximize the value of each dollar spent and will have a rolling list of priorities based on machine logic.
Perhaps most importantly, the optimized organization improves the speed, cost, and quality of their mission or business outcomes.
Augmented Intelligence in Action: an Example from the Veteran’s Administration
The GAO recently recognized the Veterans’ Administration as one of its most-improved performance-driven decision makers. When reflecting on how his agency was able to achieve this milestone, the VA’s Director of Measurement and Data Analytics, Evan Albert, pointed to the combination of veteran feedback and other machine-generated listening data into a data analysis system called “V-Signals”.
Albert noted that the agency-wide V-Signals system collected as much data from as many different lines of business as possible and consolidated that data into a central repository that the VA used to pinpoint gaps in service delivery for improved operational focus.
Therein lies the power of augmented intelligence. It’s a challenging road, but one that any agency can reach as long as it keeps its end goal of marrying human and machine-generated data.
To learn more about data-driven decision making in government, or to discuss how you might benefit from these systems, please contact me: James Eselgroth
Leave Comment