Trouble With LLMs as Google's CEO admits "we got it wrong," with Genesis
Are LLM's ready for your bank when users can't know what's "baked in?"
This week’s news stream was full of the biggest fail yet of a Large Language Model (LLM), as Google’s Genesis model shocked users with pictures that didn’t quite meet their expectations.
The blow-back was severe and reminded many of Microsoft’s ill-fated Tay chatbot that spewed offensive and inflammatory tweets in 2016.
This begs the bigger question of whether AI is ready for use in banks or financial services without an attitude adjustment.
The response from Google was swift with CEO Pichai saying:
“I know that some of its responses have offended our users and shown bias – to be clear, that’s completely unacceptable and we got it wrong.”
Pichai was at least honest in saying that the company’s LLM showed clear bias, but only after Elon Musk reveled in calling Google “woke” and the LLM “racist.”
Whatever your thoughts on Google’s Genesis, this episode highlights the inherent dangers of LLMs whose training is a “black box” to all users, personal or corporate.
Google’s PR disaster was perfectly timed for IBM’s paper on Foundation Model risks and mitigations! The paper shows that the risks are many, and mitigations are few!
👉TAKEAWAYS
A quick review of IBM’s “guardrails and mitigations” gives us a few hints of what Google got wrong but few clues on how they will fix it.
🔹Filtering undesirable data
Using curated, higher-quality data can help mitigate certain issues.
👉Ironically, I don’t think that Google had a filtering problem. Google owns the world’s data and likely got fine input; instead, the problem was one of AI ideation. The AI’s portrayal of historical facts was grossly biased, as though it was trying to correct bias in the existing data.
🔹Human oversight and human in the loop
Human oversight and review can help identify and correct errors and biases in the generated output.
👉This is precisely why so many are inflamed. It is impossible to believe that Google didn’t test the AI. Were Google’s testers that insensitive or oblivious to the results? What on earth were the humans thinking?
🔹AI Ethics review
Assessment of capabilities, limitations and risks in AI projects helps ensure the responsible development and use of the technology.
👉Ethics reviews are designed to find biases. It’s clear that human testing aside, Google didn’t seek an outside validator like an IBM or Big Four auditor. Perhaps if they did, they could have caught this problem. At a minimum, it would have removed some of Google’s culpability! This is why AI audits even for the likes of Google are important!
👊STRAIGHT TALK👊
Google is the US’s premier tech company, and its problem with Genesis shows how their internal controls were severely lacking.
While Google can patch Genesis, they’ve just done major damage to their brand and to future plans to use Genesis to write actual news reports! (Here)
So what can an unwitting bank or financial service provider do to ensure their LLM doesn’t come off the rails like Genesis?
IBM suggests “domain adaptation,” which requires training a foundation model to a specific domain or industry to help minimize the scope of risk the models can give rise to and generate outputs that are tuned to be more relevant to that domain or industry.
That’s great, but financial institutions must train the models, which will take time and money.
This is why I have my doubts when reading consulting reports suggesting that GenAI will transform financial services overnight.
Most don’t talk about domain adaption and imply that these models are so advanced that they can be used off the shelf. That’s simply wrong.
I see a great future for GenAI in financial services, but if your bank wants to avoid a PR disaster like Google, prepare for lengthy training of your new AI.
And please, please pay for an ethics review!
Thoughts?
Hey got anything to say about this article? I and others in our “Cashless” community would love to hear it!
Don’t pull your punches. Tell me why I’m right or wrong, but don’t miss this opportunity to leave a comment!
A big thank you to all my subscribers! I appreciate your notes telling me how much you enjoy reading this newsletter!
Join the community by subscribing! You’ll be glad you did!
