A little AI can go a long way

Despite breathless pronouncements otherwise, artificial intelligence is not, in fact, the universal tool that can and should be applied to every single thing so that it can take care of everything without you having to think about anything, at least not right now. 

Processing Content

And so while it might be intimidating to hear about all those companies who’ve installed AI everywhere and use it for everything, as time has gone on companies have learned that it’s not so much about AI itself but how, specifically, it is being used at an organization—and for many everyday uses, a little can go a long way. 

Speaking at a webcast today, Jessie Kanter, an audit partner with Top 25 firm Citrin Cooperman, said that while many think of AI as some sort of super-automater, this framing is a mistake as the probabilistic nature of many AI models, even agentic ones, means they can struggle with providing consistent, repeatable results. And so while AI can certainly be used to automate a process, Kanter said it’s not really ideal compared to other non-AI solutions. 

“I do think there is a lot of confusion between automation and AI and we’ve had people going down the wrong path, they throw all this stuff into Copilot. Back up. We have tool that do that right now, automation tools, that perform the same consistent tasks over and over. Agentic AI is not going to produce the consistency you’re looking for. So this tool is better suited for this task and agentic AI is better suited for other tasks,” she said. 

Jason Bradley, director of methodology and content at audit solutions provider Caseware, agreed: while AI certainly has its use, it is better used not as the thing performing automated processes itself but coordinating them using non-AI tools. Part of it is because of the difficulty with consistency that Kanter mentioned, but also because slathering every single thing with AI can get expensive in the long run due to token spend. Still, AI and automation are not mutually exclusive either, the former can powerfully support the latter if used correctly. Rather than slathering every single thing in AI, he said a little can go a long way. 

“People think of agents as not just doing the work but managing, and there are many instances where you don’t want the agent performing the task. You want it to know when the task needs to happen and to trigger the right script. This lets you build agentic workflows that are very powerful, but you’re not overdoing it and making it more expensive than it needs to be. You can take a long running automation, turn it agentic, and make it worse and more expensive overnight. The real challenge is adding the AI elements to existing automations or using traditional automation in new work flows to get the best of both worlds,” he said. 

This does not mean, though, that professionals will not need to adjust. Even if AI is using the same tools the humans have used for years, those same humans will now need to develop management and evaluation skills on top of the technical capacities they had before. Accountants are used to buying software, “plugging it in and it just working” according to Kantor but even if they are still using processes they’re already familiar with, the addition of AI means adjusting their approach. 

“It’s not the same with agentic AI that does not have consistent outputs, every time you put in the exact same worded prompt you will still get different results, so you need to be very careful not to overrely on it,” she said. 

This is part of the larger issue of governance. Kantor emphasized that AI governance should not be seen as an IT-only issue. Ultimately, she said, the buck should stop somewhere in the c-suite. But this does not need to be the only layer. At her own firm, for example, beyond the firm-wide AI governance policies, each practice area has its own additional set specific to them and how they use AI. For instance, as an auditor her own practice’s rules and procedures need to account not just for the technology itself but how it plays into things like quality management, peer review, regulation and professional standards. 

Key to how her practice approaches AI is traceability and explainability. Any AI use must have a clear reasoning trail so people can determine how exactly a given model came to its conclusions. Another policy bans the use of any AI tools that have not already been tested and evaluated, which she said goes a long way in heading off other problems. Further, specific use cases need to be approved for use, usually by the national office. 

This relates to another observation she made later in the webcast, that one should choose their AI use cases carefully. There are certain areas where it can clearly deliver value, such as document extraction, data mining, and producing first drafts (she noted AI is better at writing than most auditors, who are not trained for it.) 

“You’ve got a lot of good use cases around that: client request lists, getting those drafts written, analyzing what has been given by the client in those request lists, it can really help with reducing a lot of the back and forth,” she said. 

Conversely, she said, they tend to stay away from risk identification and assessment because that is a judgment-heavy task with high stakes. It is important, she said, not to “over-AI things” because it can make things very complex very fast, even in areas where a human would have been able to perform the task in a simpler and more accurate way. 

This speaks to a larger point Bradley mentioned: while many in the AI space emphasize efficiency, this may not be the best measure of results. Something done fast may take more time to review and correct than if a human had done things right from the start. This, in turn, means that just because a process was performed efficiently does not necessarily mean it was done cheaply, as reviewing and revising also takes money.

“Someone having done a job quicker does not necessarily lead to a good outcome… It’s also about doing a better job,” he said, adding later that “sometimes the work taking the same amount of time but being three times better is as valuable as taking one-third of the time.” 

Kantor added that it’s one thing to save hours, it’s another to ask which hours do we save and how. 

“It’s redeploying hours in a way to drive value… Focus not just on quantity but also on quality. We’l get better quality, better consistency, better results from the work people do and our people will be happier because they’re not doing that really low level stuff like moving data from one system to another to another,” she said.