The high cost of low-cost financial advice

New research from Stanford Graduate School of Business finds that financial advice from AI chatbots can push consumers toward sounder, life-cycle-based financial decisions, like broader diversification and bigger savings buffers. But the quality of that advice depends heavily on who's asking and what they're asking, and for some groups the outcomes were worse, not better. Financial advisors are seeing this play out in real time with their own clients, who bring AI recommendations to meetings, with some advisors raising concerns that their clients might not be getting the full picture.

The Stanford researchers collected real prompts from real people, then built a simulation in which 1000 "virtual people" followed advice doled out by general-purpose LLMs, not tools built specifically for financial advice. The researchers simulated life events like market, income, and job changes for each subject, then compared outcomes across simulations, slicing and dicing their data across factors like gender and financial literacy rates. They found that prompts simulating low financial literacy led to advice that left the subjects nearly $50,000 poorer by age 60. Prompts from women skewed toward words like "family," "grocery," "credit," "loan"; while men's skewed toward "portfolio," "equity," "strategy," "crypto,” resulting in advice that left women nearly $60,000 poorer by retirement.

What advisors are seeing

Advisors are seeing a similar trend--that users' own financial literacy can impact the quality of advice they get from chatbots.

“Clients often don’t know which facts are financially relevant,” says Sarah Cicero, a financial advisor based in Maryland. “They may not know what information the AI needs in order to produce a complete analysis.”

Cicero says that consumer-facing AI tools “lack context, judgment, and coordination,” and sees them as more generalists than a specialists. In one example with a customer, “the AI addressed general benefits of Roth conversions, like tax-free future growth and required minimum distributions, but it didn’t fully evaluate the timing of the strategy,” which could result in higher taxes or Medicare premiums. Getting it right requires a full understanding of the client’s income.

DC-based financial planner Matthew Koppelman agrees that LLMs “have a hard time generating the most current information. If you don’t know what you’re looking for, you won’t spot the mistake.” Even then, some clients still prefer their AI-generated responses to their planners’ guidance – an experience that can be frustrating “when someone takes their search results as gospel rather than a starting point for a discussion.”

Though the advice might be flawed or lacking nuance, these clients are still able to bounce their ideas off a trusted expert. The same cannot be said for those who, like the simulated study participants, skip the human advisor and put all their eggs in the LLM basket.

Not all AI is bad, but experts warn people should be careful

That’s not to say that people who use AI are sure to fall flat on their faces. Danielle Darling, a St. Louis-based financial advisor, says that "AI can be incredibly helpful for learning concepts and generating questions,” though she concedes that it can also confidently hallucinate. The Stanford study also found that users were overall pushed toward better behaviors, including increasing their savings, investing in stocks, and creating cash buffers – but ultimately, the quality of the questions mattered. Low financial literacy resulted in poorer outcomes.

The Stanford findings aren’t an indictment of the use of AI in financial planning, but they underscore the importance of context, guardrails, and judgment. An expert can shed light on a bad assumption or ask follow-up questions, which is why having access to quality financial advice is so important.

Where advisor-grade AI is different

In fact, the use of safe AI tools in financial services and wealth management can actually empower advisors to focus on conversations instead of taking notes or when they can look back at their data to surface insights that would have otherwise gone unnoticed.

Unlike generalist LLMs used by consumers, the trusted tools advisors use are specifically constructed around financial planning and regulatory requirements. This means that when advisor AI surfaces patterns in data and helps advisors do their jobs better, it’s doing so with an expert lens and with a human in the loop. Though some consumers might opt to take their money queries straight to AI and end up worse for it, the right AI in the right hands can actually improve outcomes for both advisors and their customers alike.