Artificial Intelligence: Complementing, Not Replacing Human Intelligence
Many of us remember our “Schoolhouse Rock” days and that tune that continues to play through our heads “I’m just a bill, yes I’m only a bill….” Well, that song takes us through how a bill becomes a law, but what happens to that bill if there is language in there that provides authority to an agency to do the work? And how do the funds to support that work become available to the right agency and program? The answer is more complex than you would think and includes creating a “warrant”.
At the Bureau of the Fiscal Service, a team of accountants is responsible for creating warrants which requires comprehension and application of the appropriations language. The accountants look at legislative documents and extract data including elements such as title, purpose, amounts, and periods of availability. Some of the legislation is simple, but much of it is complex, requiring interpretations, checks, double-checks, and approvals. This is one of those processes where both timeliness and accuracy are equally important. Once the data are identified, a warrant is created which allows agencies to spend the appropriated funds. Our project is to determine whether Artificial Intelligence (AI) techniques could accelerate the process to create Treasury warrants.
So, what was the solution that was tested? It was a hybrid of AI and non-AI technology. As for the AI solution, that included a machine learning algorithm which essentially mimicked the “brain” and will continue to “learn” over time as the model is used. In this case, we showed the model previous appropriations bills along with the information that the accountants extracted from them, teaching (or “training”) the model what text is meaningful, what to ignore, and even what numbers require some calculations. There were more complex situations where values had to be captured from previous sections of the appropriation and the model had to interpret the context.
Although some equate AI with full automation, that is not always the case. In this instance, full automation was not possible because legislation can be complex, and interpretation is not always an easy task. Some examples of this complexity include discerning which numbers are relevant in warrant creation or understanding the context so that information from different sections of the bill can be pieced together. Although the AI model was able to achieve high accuracy, this is a process that requires 100%. So, full automation was not possible, and determined by the team to be less optimal.
For this project, the team discovered that the best use of AI could be if used as a tool, but not a full end-to-end replacement for the accountants. Rather than pouring over the legislative text to interpret and extract the information needed, the AI solution could do that for them and provide recommendations. The accountants could then review those recommendations and either approve or correct them. Although the solution during this test was accurate on the test data, it will continue to get smarter and more accurate as it processes more test data that is either approved or corrected by the accountants. The test results showed a more efficient use of the accountants’ time as the tedious and repetitive tasks for interpreting the data is helped by technology.
Beyond the benefits of automation, the solution could also be used as a training tool for this team of accountants as they bring on new staff who are unfamiliar with the warrants process. A model that suggests correct inputs could allow new staff to check their work as they tackle the learning curve and improve their familiarity with the legislative text and its nuances. Fiscal Service is frequently requested to analyze draft legislation, which is voluminous, so the tool can be used efficiently for planning of operations.
Another lesson learned from this test was that lower tech solutions should be considered along high-tech solutions. During this test, the team uncovered some pain points that may be better addressed by optimizing the process rather than applying technology. So, before moving forward with the next stages in building the model, the Digital End to End Efficiency team in FIT will work with the operational team in identifying opportunities for process improvement. Stay tuned for another update on the AI warrants project and that DEEE work at https://www.fiscal.treasury.gov/fit/.