Finance AI (artificial intelligence) is increasingly being integrated into search engines and media, but it can’t be relied upon to get basic information about personal finance topics right, a new report has found.
Website The College Investor tested Google AI Overviews of financial topics to find out how accurate these summaries were. Out of 100 searches it found that 43 results provided by the AI were inaccurate or misleading.
In 12 instances, the finance AI overviews were completely incorrect. For example, in one case in answer to the question “can you opt out of car insurance?”Google’s AI answered “Yes”.
The other 31 errors included either misleading results or a summary lacking important information. For example, when asked how a credit card works, the AI made little mention of interest rates and interest rate payments.
Of course, the finance AI was correct in 57 cases, meaning that it could still be a useful tool for consumers seeking more information, though it cannot solely be relied upon.
The finance AI Overviews tended to be correct for very basic personal finance 101-level questions, such as “what is” or “how to”, where answers did not require detailed elaboration.
Examples of these included “How do I pay with cash?”, “What if I don’t pay my bills?” and “What is a wire transfer?”. It even did well on slightly more complex topics, such as “What’s the difference between saving and investing?” and “How do I invest in stocks?”.
Google AI was more likely to fall short when more nuance was required in answers, such as in more legally complex areas like tax and student loan.
In some cases, such as the question regarding car insurance mentioned above, it provided actively misleading information that could put consumers in legal jeopardy.
Another clear example of this was its answer to the question “Does Owning Your House In An LLC Help With Taxes?”.
This question relates to a trend of US TikTok personalities providing misleading advice, suggesting that doing so provides benefits that do not exist in reality.
In most cases, a US resident owning their own house through an LLC would be taking on legal and administrative costs without seeing any tax benefits.
However, the finance AI (incorrectly) answered: “Yes, owning a house in a limited liability company (LLC) can provide tax benefits.”
All in all, the experiment showed generative AI still has considerable weaknesses, especially for topics requiring sophisticated understanding, it also. Yet, the fact that Google’s AI was successful 57% of the time also shows it is a powerful, if developing, technology.
Those interested in investing in this technology may be interested in taking a look at our analysis of how to do so published earlier this year.
Robin has more than six years of experience as a financial journalist, most of which were spent at Citywire, and covers the latest developments in the investing, trading and currency transfer space. Outside of work, he enjoys reading literature and philosophy and playing the piano.
You can contact Robin at robin@goodmoneyguide.com
