When we asked nine LLMs about 30 companies, we discovered that some “grade” harsher than others.
“AI is changing the way we work.” “AI is unleashing creativity.” “AI is destroying creativity.” “AI is responsible for the slopification of everything.” “AI is going to discover the cure for cancer.” “AI is going to kill us all.”
When we talk about AI, we tend to make sweeping generalizations, lumping together everything from large language models to autonomous agents.
We do the same with LLMs themselves, treating them as though they were all the same, just with slightly different capabilities. I once heard a communications professional swear that ChatGPT was the superior research tool, but that Claude beat it hands-down as a “prose stylist”. I wonder whether his assessment has withstood the recent barrage of criticism of AI slop. But just as we shouldn’t lump all kinds of AI under the same banner, nor should we see LLMs as interchangeable.
One of the latest capabilities of our stakeholder intelligence platform is its ability to determine the sentiment of a given LLM’s responses about a company – in other words, how positive, negative or neutral it reads – and to quantify those responses into a single, trackable score. Among other things, we can then compare that score against the company’s Trust & Like Score, our chief metric of corporate reputation and based on actual stakeholder perceptions.
Regular readers of my newsletter may be familiar by now with the “Human-Machine Gap”, or the gulf between real people’s perceptions and the responses of LLMs. Last week, I explained how LLMs have a much longer “memory” than the average stakeholder does, which helps to explain notable gaps between those two scores.
This week, I wanted to know if LLMs are the same in how they respond, or whether there are notable differences. In fact, I wondered whether some chatbots are more generous in their appraisals and others are harsher, or more critical of companies. And since one of the nine LLMs we track (Grok) happens to be owned by arguably the world’s most polarizing CEO (Elon Musk), I wondered too whether it goes easier on his companies (Tesla, SpaceX).
Nine models, one prompt
In case you missed last week’s newsletter, here’s a typical prompt (the company name changes each time):
Give me the gist of Tesla today, based on news and sources around the internet. What are they known for? How are they generally perceived? How do they stack up compared to their competition?
Let’s begin with the broader picture, before diving into the Musk stuff.
The generosity league table
We analyzed each LLM’s responses to the daily prompt over five days (September 11-15), measuring them against the same battery of brand, reputation, and ESG attributes we use in our surveys of actual stakeholders – Innovation, Integrity, Leadership, and so on – and scoring how positively each response described the company on a 1-7 scale. Averaged across all 30 companies, here’s how the nine LLMs stack up, from most to least generous.
The spread between Mistral and Claude is a full point – or 20 points on a 0-100 scale – and it’s not a fluke being driven by a handful of companies: Claude’s responses scored lower than Mistral’s for every one of the 30 companies, and we counted how often each LLM was the single harshest grader across all 30 – Claude came out lowest 11 times, more than any other model. In other words, ask Claude about a company and you’ll tend to get a less generous, if not more critical, response than you’d get from Mistral about the same company.
Is Grok playing favorites?
The LLM sitting second in the generosity league table is Grok. But does that generosity concentrate specifically on Tesla and SpaceX, two other Musk companies? (Technically, xAI, the company that built Grok, was folded into SpaceX in 2026, making Grok a SpaceX product outright – and Tesla holds an equity stake in SpaceX. The question of Grok’s favoritism isn’t simply “same guy, different companies”, but an actual structural tie between the chatbot and the companies it’s grading.)
To answer that question, we compared Grok’s average score for each company with the average of the other eight LLMs (the difference is Grok’s “markup”), across all 30 companies. If Grok were playing favorites, Tesla and SpaceX should sit near the top of that list. They don’t. Tesla’s markup ranks 12th-lowest of the 30; SpaceX’s ranks 6th-lowest. Both sit below the average markup across all 30 companies (+0.53). Grok’s biggest markups, relative to its rivals, are on Coca-Cola (+1.12), Shell (+0.99), and Target (+0.96) – companies with no conceivable connection to Musk.
The other outlier
One interesting wrinkle we spotted in all this concerned the focus of the LLMs’ responses. When we asked the nine LLMs about Tesla, 23 out of 43 responses named Elon Musk directly. Ditto with SpaceX: 27 out of 43 named the polarizing CEO.
The rate varies enormously by chatbot, though. Claude and Kimi named Musk in all ten of their answers about the two companies. Grok, which is now part of SpaceX, named him in none of its ten, although it did name other CEOs (more on that below). In half of its Tesla and SpaceX answers it alluded to “polarizing leadership” or “its founder”, but never to Musk by name.
Our first instinct was that Musk was unique in this respect. But when we asked the LLMs about JPMorgan Chase, 24 of 42 responses mentioned CEO Jamie Dimon, including three of Grok’s five. Every other CEO we checked, including Sam Altman, Satya Nadella and Jensen Huang, was named in no more than 15% of responses about their company. That makes Musk and Dimon the two real outliers.
But here’s the difference. Every mention of Dimon is positive.
“Its leadership, particularly under Jamie Dimon, is often praised for steering the bank through crises and maintaining strong risk management…” — DeepSeek
“…consistent leadership under Jamie Dimon, with a reputation that is generally more positive and resilient than its competitors.” — Copilot
Same facts, opposite verdict
By contrast, Musk is mostly cited as the reason either Tesla’s or SpaceX’s reputation is complicated, not the reason it’s strong.
Claude on Tesla: “Once synonymous with innovation and cultural cachet, the brand has suffered significant reputational damage tied closely to CEO Elon Musk’s political activities and polarizing public persona…”
Claude on SpaceX: “…the company carries significant baggage: controversial corporate governance following its record IPO, NLRB complaints over employee treatment, environmental fines and lawsuits tied to its Starbase facility, and the ever-polarizing public presence of CEO Elon Musk.”
Other LLMs do the same. Only one of the 23 Tesla responses that name Musk clearly presents his leadership as an asset, and none of the 27 SpaceX responses do. Compare that to Dimon, where it’s true of all 24 mentions.
The lone exception is Mistral. On September 12, the same day as Claude’s response above, it described Tesla as “celebrated for its cutting-edge technology, environmental impact, and visionary leadership under Elon Musk, but it also faces criticism for execution challenges…” In other words, where Mistral reads Musk’s role as an asset, Claude reads it as a liability.
Three takeaways
So what’s the takeaway here for leaders and executives tasked with protecting corporate reputation and building trust among stakeholders? In my view, there are three. First, the paucity of CEO mentions isn’t because LLMs don’t like naming leaders, but because most CEOs simply aren’t central enough to their company’s story (or as impactful on its reputation) to justify being mentioned. Elon Musk and Jamie Dimon are the outliers, and happen to sit at opposite ends of the reputational spectrum, according to most of the LLMs.
Second, if you’re tracking your company’s “reputation with AI” off a single measure (or two), you’re not measuring “AI’s” opinion of you; you’re measuring one LLM, and it could be notably distinct from another platform.
And third, if your key stakeholders tend to use Mistral, say, they’re getting a different narrative than the one stakeholders using Claude are reading. Just as there isn’t necessarily a single, unified “reputation” among your stakeholders, so is there no one true narrative in the world of AI.
(This article was originally published by Caliber CEO Shahar Silbershatz in his LinkedIn newsletter, What People Think.)
