Taxes show AI cannot always substitute for human judgment

Artificial intelligence is transforming every aspect of our lives. From business to shopping, school work to creative industries, AI’s broad adoption is making life easier for millions. 

Processing Content

But it’s not foolproof, and overreliance coupled with complacency can lead to legal speedbumps, ethical quandaries and financial losses.

Human intelligence is often required to bridge the gap and negate risk, especially for tasks with particularly high stakes. 

Mind the gaps

McKinsey highlights that currently 78% of organizations use AI in at least one business function, up from 72% in 2024 and more than a 20% increase from 2023. Equating increased income with increased AI use isn’t guaranteed though, with other investigations showing 95% of companies have received zero ROI from generative AI pilots. 

Likewise, while most employees are using AI tools, more than 65% cite inaccurate results as a top concern. 

There is no doubt gaps exist between exploration and execution of AI. Experimenting with it is one thing — embedding it properly and using it effectively is another.

This is especially significant for items that are high stakes: activities that need meticulous human oversight where even slightly skewed results can burden a business with heavy costs.

Why taxes are AI’s kryptonite

AI tools can accelerate processes, spot trends, automate repetitive tasks, and provide insights, but outputs are only as reliable and accurate as the instructions given to the AI and how these are implemented. If datasets that train the AI models aren’t up to date, or not in line with changing laws or regulations, or if there are gaps in the data or inaccuracies, they can impair the final outcome. 

This is where taxes become our cautionary tale. Heavily context dependent, complex, and requiring precise calculations, the consequences of getting them wrong are significant — especially with millions of enterprise dollars at risk.

There are innumerable exceptions and exemptions within the tax system. To stay on top of the moving feast of tax law or its permutations requires a depth of knowledge not currently available to any AI system. How an accountant prepares and files a client’s taxes is founded in legal requirements but open to a level of interpretation depending on the individual’s level of risk. AI will not have this level of access or insight, nor would you trust one to learn it. The stakes are so high, the margins of error so small, it would be ill-judged to rely on AI to take on the IRS.

If numbers are slightly off, if data can’t be justified in an audit, if financial reporting isn’t completely waterproof, if an AI miscalculates liabilities, applies incorrect rates, or overlooks deductions or deadlines, there will be fines, late fees and interest demanded from businesses.

This isn’t to say AI doesn’t have a place for improving functions within tax teams. Interpreting data, classifying transactions and assigning reasoning, etc., are all crucial ways AI can make efficiencies. Only when it’s relied on for the end-to-end tax calculation is there a problem. 

Taxes are where AI finds its limit.

The AI ceiling

There are other domains where AI should be deployed with care.

Think of diagnostic medicine, where an incorrect conclusion or a condition missed could be fatal. AI models in medicine have also been known to suffer from ‘“black box” issues, where treatments or errors can’t be tracked or caught.

There is also the problem of automation bias where clinicians might over-trust AI and override their own judgment. If a doctor is under time pressure, this could be especially dangerous, with a recent study finding a 7% automation bias rate, where correct assessments were overturned by erroneous AI advice. 

Overreliance on AI will also pose risks for the legal and compliance sectors. Artificial intelligence drafting legal documents can insert errors, ambiguous language and clauses, or even refer to old or even made-up regulations. The U.K. Solicitors Regulation Authority’s Risk Outlook report warned of cases where AI-generated arguments included nonexistent legal references. 

In compliance,  AI can misinterpret rules or fail to detect regulatory changes, which could lead to people or companies being fined, breaches in data privacy and increased third party exposure.

Where AI excels

There will always be potholes with any new technology, but the road is only going in one direction. In all the sectors mentioned previously, AI adoption is booming. Indeed, businesses that don’t keep pace will get left behind. 

Therefore, how AI is harnessed is even more crucial. If there are degrees of error involved, the consequences of this need to be rigorously assessed, and the risks mitigated accordingly. When this is done, the potential value in executing AI tools successfully is immense.

AI can interrogate and corroborate intelligence, provide insights, and empower nontechnical teams to make data-led decisions. The AI of tomorrow can allocate resources, provide strategic business counsel and enhance the skill sets of the human workforce.

As AI becomes more pervasive, the challenge isn’t whether we should use it, but in what capacity. AI thrives when combing through vast datasets, harvesting insights and removing friction from repetitive work. When combined with the human ability to discern fact from fiction, critically assess and apply ethics, it creates a platform for growth.

From an organizational perspective, leaders should be introspective and honest. Ask what tasks or functions would truly be elevated with AI, and try software to see what benefits are unearthed.

AI at its best is able to amplify our capabilities, as individuals and as businesses or industries. Invoking AI where it makes sense, and exercising caution in other areas until proven otherwise, is the sensible and methodical path to progress.