Intuit, H&R Block tax AIs critiqued on accuracy of answers, many inaccurate

The rollout of generative AI for taxpayer assistance has been a little rocky this season as some have raised concerns over outright wrong advice being given to users.

An article in the Washington Post tested the accuracy of the tax advice given by AI models recently deployed by Intuit’s TurboTax and H&R Block. Both companies were faulted for providing answers that were either irrelevant or flat out wrong. For instance, when asked about whether there were tax credits available for having installed a new air conditioner, TurboTax’s AI responded with information about educational expenses. Meanwhile, when asked about whether wash sale rules apply to cryptocurrency, H&R Block’s AI responded that, yes, they do, when, in fact, they do not.

Heather Watts, senior vice president of consumer tax products at H&R Block, said in an email that AI Tax Assist is but one of many ways the company assists with the preparation process and tax questions, which includes the ability to speak live to a human professional at no extra cost. The generative AI, she said, is meant to supplement the tax preparation process and not be the final word when it comes to one’s taxes.

Real-time monitoring and evaluation of customer interactions with the AI tool, she said, do not suggest a significant number of incorrect or irrelevant answers. She added that the errors produced by the bot during the Washington Post’s testing could be the result of insufficiently detailed prompts.

“As with any AI tool, when individuals provide detailed, contextual information, AI Tax Assist generates more accurate and helpful responses. Regarding The Washington Post’s coverage, our experts at The Tax Institute tested the responses deemed inaccurate, finding that alternate phrasing and use during the guided interview led to more accurate responses,” she said, adding that H&R Block overall is backed by guarantees including audit support, maximum refunds and 100% accuracy.

Karen Nolan, senior earned media manager with Intuit Consumer Group’s TurboTax communications team, similarly, said the generative AI tool was never meant to be the be all and end all of tax advice, but one of many parts of the overall experience that helps people file with confidence. The use of AI, she said, is less about doing all the tax work for the user and more about automating repetitive tasks to save time and ensure accuracy.

“For example, AI reduces data entry errors by extracting and categorizing data from forms, improves the identification, classification and placement of data in the correct fields, and auto-populates tax form information from tax documents,” she said.

Contrary to what the Post reported, she said Intuit has not found a significant number of people experiencing issues with its AI tool. Intuit, she said, understands the stakes involved in an accurate tax filing, which explains their guarantee policy.

“At TurboTax, we understand the seriousness of taxes and finances, and we provide every filer with a tax return lifetime guarantee, including 100% accurate calculations, audit support and a maximum refund,” she said.

This is but the latest controversy regarding the accuracy of claims made by generative AI bots. The Associated Press, for example, recently reported that, when asked about relevant election information such as where to vote, 40% of answers were considered outright harmful, including perpetuating dated and inaccurate answers that could limit voting rights.

Meanwhile, there was also the matter of a generative AI chatbot used by Air Canada giving someone inaccurate price information, saying a customer could claim a refund after purchasing tickets when, in fact, the specific policy was to not give refunds once the flight has been booked. While the airline tried to claim the chatbot was a separate legal entity and that, further, the customer should have never trusted what it said anyway, a court nonetheless ordered the company to give the customer a partial refund plus additional damages to cover interest on the airfare and tribunal fees.

Upcoming research data from Accounting Today’s parent company, Arizent, found that inaccurate or irrelevant information was the No. 1 concern accountants had about generative AI: 50% reported they were very concerned about this and 35% said they were somewhat concerned. The incidents also speak to another major concern found in the data — the degradation of client trust and transparency — with 42% of respondents saying they’re very concerned and 29% somewhat concerned.