Ouroboros, an apt symbol for AI model collapse

by Ariella Brown

AI hits the ouroboros (sometimes written uroboros) stage. You've likely seen it in the form of a snake in a circle, eating its own tail. The ancient symbol also sometimes showed dragons or a wyvern, so I chose this engraving by Lucas Jennis intended to represent mercury in the 1625 alchemical tract "De Lapide Philosophico," for my illustration instead of just going with something as prosaic as "model collapse"

To get a bit meta and bring generative AI into the picture (pun intended, I'm afraid) here's an ouroboros image
made with generative AI. ked Google

Model collapse is what the researchers who published their take on this in Nature called the phenomenon of large language models (LLMs) doing the equivalent of eating their own tails when ingesting LLM output for new generation. They insist that the models should be limited to"data collected about genuine human interactions."

From the abstract:
"Here we consider what may happen to GPT-{n} once LLMs contribute much of the text found online. We find that indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. We refer to this effect as ‘model collapse’ and show that it can occur in LLMs as well as in variational autoencoders (VAEs) and Gaussian mixture models (GMMs). We build theoretical intuition behind the phenomenon and portray its ubiquity among all learned generative models. We demonstrate that it must be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet."

Shumailov, I., Shumaylov, Z., Zhao, Y. et al. AI models collapse when trained on recursively generated data. Nature 631, 755–759 (2024).

Let me know in the comments which illustration you like more. 

An Apology to Generative AI

ChatGPT spelled out in Scrabble tiles

By Ariella Brown

I'm not a generative AI fangirl. If anything, I'd consider myself more of a skeptic because people tend to not just use it as a tool to improve their writing but as a tool to replace the work of research, composition, and revision that is essential to good writing.

It is generally embraced by people who consider online research to be too much work and who believe that anything that comes out of a machine that will charge them no more than $20 a month for writing to be too good a deal to pass up. 

For those of us who actually read, the output of ChatGPT and similar LLMs is not exactly something to write home about. Unless you know how to prompt it and train it to write in a truly readable style, it will default to the worst of wordy, opaque corporate style text. 

But this isn't the fault of the technology. It's the fault of the mediocre content that dominates the internet that trained it. Below is one example that I pulled off  the "About" section of a real LinkedIn profile (first name Kerri maintained in the screenshot that proves this is real and not something I made up):  

As a strategic thinker, problem-solver, and mediator, I thrive in managing multiple, sometimes differing inputs to achieve optimal messaging and positioning. My proactive nature drives me to partner with leaders across marketing teams and internal business units, aligning efforts, connecting dots, and adding context to enable flawless execution of communication strategies and tactics.

In fast-paced, fluid environments, I excel in effectively prioritizing tasks and ensuring they are completed efficiently. I have a proven track record of setting and meeting strict deadlines and budgets, leveraging my ability to navigate dynamic landscapes seamlessly.

Driven by natural curiosity, I am constantly seeking to understand and implement the latest trends, technologies, and tactics essential for driving B2B sales opportunities. My keen interest in exploring new channels for messaging and content distribution fuels my passion for innovation and continuous improvement to not just meet but exceed expectations.

Let’s connect to explore how we can drive success together.

You know what sounds exactly like this? Cover letters you ask ChatGPT to compose for you. 

I've tried those out a few times and never been happy with the results because they always sounds like the text above. Trying to tell it to sound less stiff doesn't make it sound any less canned, and forget about getting it to copy my own writing style.

It's possible that Kerri used ChatGPT to create her "About" section. Given that she's been in the marketing biz for some time, though, I'd think she had to have had something filled out for years before ChatGPT was available, and it likely sounded very much like this even if she did let some LLM or something like Grammarly tweak it for her.  

People like Kerri, who ignore all writing advice from the masters like Orwell, White (watch for a upcoming  blog about him), and others made this the public face of corporate communication who are to blame for the bombastic and soulless style that LLMs replicate at scale. 

That's the reason for this apology too ChatGPT for mocking its output. You're not the one at fault. You had no way of knowing better. Humans do, and they should have provided you with better models for writing. 

Note on the title: I thought of giving this post the title "Apology" intended in the classical sense of a defense or justification for something others take as wrong with the hint of an apology to AI. Knowing that that wouldn't be clear to some readers, I opted to make this just a straight apology instead. 

Related:

A new generative AI comparison

The Perils of prompt engineering

Bard vs. Bard
Add a pinch of salt to creative claims for AI

An A/B test of generative AI
Bard brings in the clowns

A new generative AI comparison

Photo by Andrew Neel: https://www.pexels.com/photo/monitor-screen-showing-chatgpt-landing-page-15863066/

In the past, I've ncomparted Cha GPT to Google Bard (now Gemini). For today's exercise, I put the same query to Claude, ChatGPT, Gemini, and Copilot. I put in the name of a real person and identified the location for that individual in Queens, NY.  None could give me the information I sough. 

What's interesting, though, is noting the differences in how they contextualized their responses. Out of the four, Claude shows the greatest self-awareness, and I'd say that Copilot shows the least.  

Claude's response

As this appears to be asking about a relatively obscure individual, I want to remind you that while I try to be accurate, I may hallucinate or generate incorrect information when asked about very specific people who are not widely known public figures. If you're looking for factual information about this person, I'd recommend checking other reliable sources.

I award Claude points for warning me about the possibility of hallucinations occurring when generative AI attempts to answer questions in the absence of data. See 

ChatGPT's response

I don't have specific information about an individual named ____from Queens, NY. It's possible that she is a private individual or someone not widely known beyond her personal circles. If she is known for something specific like public contributions, achievements, or notable activities, that information might be available through local news sources or community records.

ChatGPT's response failed to warn about hallucination, though it's interesting that it inferred the person in question was a she (based on the first name, I suppose), a gender identification that Claude managed to avoid.

Google Gemini's response

Gemini just bowed out, practically chiding me for giving it this query that it could not answer:

 
I do not have enough information about that person to help with your request. I am a large language model, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions, but my knowledge about this person is limited. Is there anything else I can do to help you with this request?

Copilot's response

Copilot went in a completely different direction. I wouldn't call it a hallucination, but it is still inaccurate. It wanted to find a match, so in one case it named a business with the last name appearing twice as a partnership, though it admitted that was Brooklyn rather than Queens. Its second suggestion was a male doctor in Queens with a similar last name, though it is spelled differently. Honestly, I find it hard to believe that these are the only two results for the last name in all of NYC. I mean once you're expanding the parameters, there really should be a lot more than two possible results. 

Related

The Perils of prompt engineering

Bard vs. Bard
Add a pinch of salt to creative claims for AI

An A/B test of generative AI
Bard brings in the clowns

Follow Write Way Pro on LinkedIn and on Facebook.

You can also follow Ariella Brown.  

Building foundations for castles in the air

By Ariella Brown

“If you have built castles in the air, your work need not be lost; that is where they should be. Now put the foundations under them.”

― Henry David Thoreau 

Two results of the Gemini-generated pics for my prompt based on the quote. Let me know which one you prefer in the comments

This is the essence of creative content marketing. You can start with the creative idea to construct something truly different and unique. But if it is not grounded in reality, which is the actual experiences of the target customer, then it will never be a sustainable structure.

HT to Alfredo A. Lopez for making the comment on Tom Goodwin's post that made me think of this.

I'm wondering if Tom's dichotomy was sparked by reading the "bothism" advocated by Mark Ritson. https://www.marketingweek.com/ritson-bothism-cure-marketers-fascination-conflict/in which he says:

We’ve had the decade-long custard-pie fight over ‘digital’ versus ‘traditional’ forms of communication. I remain completely bemused as to what these terms actually mean in 2020, given almost every form of communication from radio to outdoor is now demonstratively digital in delivery. And yet we have spent all this time pushing the benefits of one side and then defending the honour of the other.

Twenty minutes with a decent data set and an open mind would demonstrate to any marketer that when you adopt a Bothist view of communications and add a dash of traditional media to the digital cake mix, the whole confection improves dramatically as a result.

Related

Don't just aim for different
Visualizing the customer journey
What makes content marketing effective
Trying to stand out

Bard vs Bard: why you can't just accept the first AI prompt response

 

by Ariella Brown

Lately, I've been thinking about business terms that only came into being in the past decade or two. Among them is RevOps, short for revenue operations. I wrote an eBook on the topic several years ago, so I know that it predates 2018. But I wanted to pinpoint the term's origin.

I asked Bing, and this is where it gets interesting. What it said in answer to the first prompt, "Who coined the term revenue operations?"  was contradicted by the second.  Here's the response to the  first prompt: 

The term "revenue operations" was coined by Chris Goede, the former Chief Revenue Officer at HubSpot. Goede first used the term in a 2016 blog post, in which he argued that the traditional sales and marketing functions were no longer sufficient to drive revenue growth. He proposed a new approach, which he called "revenue operations," that would bring together sales, marketing, and customer success under a single leader.

Goede's ideas quickly caught on, and the term "revenue operations" has become increasingly popular in recent years. Today, there are a number of companies that specialize in helping organizations implement revenue operations practices.

Here are some of the key principles of revenue operations:

• Alignment: Revenue operations requires close alignment between sales, marketing, and customer success. All three functions must work together to create a seamless customer experience that drives revenue growth.
• Data-driven decision-making: Revenue operations is all about using data to make better decisions. By tracking key metrics and trends, revenue operations teams can identify opportunities to improve the customer experience and drive revenue growth.
• Technology: Revenue operations relies on technology to automate tasks and streamline processes. By automating repetitive tasks, revenue operations teams can free up time to focus on more strategic initiatives.

Revenue operations is a relatively new concept, but it has the potential to revolutionize the way organizations drive revenue growth. By bringing together sales, marketing, and customer success under a single leader and using data-driven decision-making, revenue operations teams can create a seamless customer experience that drives revenue growth.

Second attempt

Not being a novice at generative AI, I decided to corroborate what it said and asked" Did Chris Goede coin the term revenue operations?" One would assume Bing would say "Yes," to be consistent with its previous answer. But then one would be wrong. This is what it responded:

No, Chris Goede did not coin the term revenue operations. The term was first used in a 2012 article by Jason Lemkin, the CEO of SaaSquatch. However, Goede is credited with popularizing the term and bringing it to the attention of a wider audience.

In 2016, Goede wrote a blog post for HubSpot in which he defined revenue operations as "the alignment of sales, marketing, and customer success around a shared goal of driving revenue." He argued that revenue operations is essential for organizations that want to grow their revenue and compete in today's competitive marketplace.

Since then, Goede has continued to speak and write about revenue operations. He is also the co-founder of Revenue Collective, a community of revenue leaders who are committed to driving revenue growth.

Third time not a charm

It then continues to plug RevOPs, but that's not of interest to me with this query, so I put in another one: "Was Jason Lemkin the first person to use the term revenue operations?"  I find it truly fascinating that this stumped Bard, as indicated by this response: 

I do not have enough information about that person to help with your request. I am a large language model, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions, but my knowledge about this person is limited. Is there anything else I can do to help you with this request?

Round 4

  So I gave it another shot (and if this doesn't prove I'd be an amazing prmpt engineer, I don't know what does). I put in the query: "When did the term "revenue operations" first appear in print?" That got an answer that reinforced the second one with some relevant context:

The term "revenue operations" first appeared in print in a 2012 article by Jason Lemkin, the CEO of SaaSquatch. In the article, Lemkin argued that revenue operations is a new discipline that combines sales, marketing, and customer success to drive revenue growth.

Here is the excerpt from the article:

"Revenue operations is a new discipline that combines sales, marketing, and customer success to drive revenue growth. It's about creating a seamless customer experience that drives more sales, more upsells, and more renewals. It's about using data to make better decisions about where to allocate resources and how to improve the customer experience. And it's about automating tasks to free up time for employees to focus on more strategic initiatives."

Since then, the term "revenue operations" has become increasingly popular. There are now a number of companies that specialize in helping organizations implement revenue operations practices.

Here are some of the key benefits of revenue operations:

• Increased revenue: Revenue operations can help organizations increase revenue by improving the customer experience and driving more sales.
• Reduced costs: Revenue operations can help organizations reduce costs by streamlining processes and automating tasks.
• Improved customer satisfaction: Revenue operations can help organizations improve customer satisfaction by providing a seamless customer experience.
• Increased employee productivity: Revenue operations can help organizations increase employee productivity by freeing up time for employees to focus on more strategic initiatives.

Related posts:

AI's early attempts at screenwriting

The perils of prompt engineering

An A/B test of generative AI

Bard brings in the clowns

AI's early attempts at screenwriting

OpenAI's ChatGPT has made everyone aware of the potential of generative AI. Well, some of us knew about this tech before it became trendy. As a writer reporting on AI, I wrote about one of the early attempts to use it for a film script. The results were not exactly typical entertainment.

"I'd Like to Thank the Algorithm that Made this Film Possible" was published on 7/11/2016 on All Analytics where it garnered 63 comments. I saved a PDF of the text that you can read here.

The film, "Sunspring" is still up on YouTube here:

In 2017, the film producers came put out "It's No Game," starring David Hasselhoff. It's not mentioned in the original blog that predated it. You can see the sequel here:

Generative AI has improved substantially since 2016. But is it ready to write a film that will keep people's interest as well as those written by human counterparts?

We already have a commercial made by AI that is rather strange -- some would say scary -- but sill very effective at driving human attention. I have nou doubt we'll be seeing more films in future.

An A/B Test of Generative AI

Yet another brand that uses this slogan is featured on Amazon

I put the same query to Bing, ChatGPT, and Google Bard. This was the prompt: "Let's Build the Future Together": How many businesses use this slogan?


Want to guess which one gave the best answer?

❌ For Bing, I made sure to be in the chat mode and got even more disappointing results than I got from its search mode. This is the New Bing's response to my query:
"I’m sorry but I couldn’t find any information on how many businesses use the slogan “Let’s Build the Future Together”. However, I can tell you that a company slogan is a word or phrase used to represent a business as a whole or its products and services. Slogans typically follow a company or brand’s name in marketing materials such as business cards and advertisements."


❌#OpenAI's ChatGPT response came faster, but it wasn't any better than Bing's. This is what it said:
"I'm sorry, but as an AI language model, I do not have access to information on the exact number of businesses that use the slogan "Let's Build the Future Together." However, this slogan is a common one that has been used by many different businesses and organizations over the years. It is a popular marketing phrase that is often used to convey a sense of partnership and collaboration with customers or other stakeholders, as well as a focus on innovation and growth."


✔ Google's Bard was the clear winner out of the three. It admitted it didn't have full information and it went off on the tangent of why businesses use that slogan, but it did at least name some that use it:

The United Nations
The World Bank
The Bill & Melinda Gates Foundation
The Coca-Cola Company
The Ford Motor Company

It added that it was used by Barack Obama's 2008 presidential campaign.

While I don't claim to be an "AI Prompt Engineer," I do know that you sometimes need to try the prompt again to get more thorough responses, so I regenerated the response on Google Bard. It offered a slightly different contextual background and repeated the UN and Coca-Cola but it also had 3 additional names of brands to add to the list: Amvets, LEGO, and Siemens.

It also plugs Google, noting that "a quick Google search reveals that there are hundreds, if not thousands, of businesses that use this slogan."

A third run on Bard yields 2 names already mentioned plus 3 new ones:
The American Society of Mechanical Engineers
The National Association of Home Builders
The World Economic Forum





So you get the picture. It can take multiple rounds to build up a list that is not going to be comprehensive but at least gives you some idea off the range of organizations that apply the same slogan.

I'd recommend that all brands do searches on the slogans they're considering. While it may make sense for toys and those in the building trade to adopt it despite its wide adoption by other organizations, a new startup will likely not get much traction from something this abstract.


Related:

AI's early attempts at screenwriting

Bard vs Bard: why you can't just accept the first AI prompt response

The Perils of Prompt Engineering