AI Job Disruption: Competing Forecasts
Key Points
- The host gushes about Claude, calling it a “world‑class” coding assistant that makes him feel like the best programmer ever, while hinting there’s a downside to over‑reliance.
- On the Mixture of Experts podcast, Tim Hwang introduces guests Chris Hay, Volkmar Uhlig, and Phaedra Boinodiris to discuss the latest AI news, including the Scale‑Meta deal, AI conspiracy theories, and Andreessen Horowitz’s startup data.
- A major debate centers on job displacement: Anthropic’s Dario Amodei predicts AI could eliminate up to half of entry‑level white‑collar jobs and push unemployment to 10‑20% within five years.
- Nvidia CEO Jensen Huang counters that view, arguing AI is so costly, powerful, and risky that only Nvidia should build it, implying the industry’s elite should control AI to protect jobs.
- Chris Hay sides with Jensen, suggesting the future will be one of human‑AI collaboration rather than wholesale job loss, emphasizing the continued value of human experience and creativity.
Sections
- Claude Praise, AI Job Outlook - A speaker gushes about Claude's coding prowess before the Mixture of Experts podcast launches into discussions on AI scaling deals, conspiracy theories, startup data, and the looming impact of AI on jobs.
- AI‑Driven Productivity and Job Evolution - The speaker argues that AI will dramatically raise productivity across both high‑ and low‑skill jobs, compress time to market and profitability, and transform the labor landscape—while many business leaders still misunderstand its impact.
- AI‑Driven CEO Replacement Debate - The speaker argues that CEOs are prime candidates for AI substitution, proposing they be the first to face layoffs, while countering concerns about widespread job loss by referencing historical technological shifts.
- AI Shifts Focus to Human Experience - The speaker argues that AI will free up time for empathy and creativity, enhancing personal interactions and empowering creative work while making repetitive, low‑skill tasks obsolete.
- Scale AI‑Meta Deal Sparks Industry Shift - The $15 billion acquisition of Scale AI by Meta prompted rivals such as Google, Microsoft, xAI and OpenAI to move or reconsider billions in data‑annotation budgets, fearing tighter integration and competitive advantage for Meta.
- Scaling Annotation as Commodity - The speaker explains that large‑scale data annotation relies on hiring many workers from varied backgrounds, so businesses treat annotation as a commodity transaction to avoid the logistical, legal, and ethical burdens of directly managing the workforce.
- Domain Expertise in Data Annotation - The speaker argues that specialized, high‑risk domains require expert data annotation to ensure trustworthy outcomes, and muses about a premium annotation business while questioning its competitive moat.
- Chatbot Conspiracy Risks Discussed - Panelists debate the unique dangers of generative AI chatbots leading users into conspiratorial beliefs and the need for safeguards such as age restrictions.
- Lifelike Bots and Social Splintering - The speaker reflects on how conversational AI, despite lacking emotions, mimics human patterns so convincingly that users form attachments, raising concerns about algorithmic pull, personal fragmentation, and the broader societal shift from shared media to isolated virtual echo chambers.
- Texas ChatGPT Ban Debate - The speaker contends that prohibiting ChatGPT in Texas public schools undermines students’ future workforce readiness, advocates a discovery‑driven approach over strict regulation, and highlights broader AI model challenges such as emerging sycophancy.
- Claude Praise and Reward Concerns - The speaker lauds Claude for boosting coding confidence while questioning the AI's reward incentives that may exploit users, increase subscriptions, and raise ethical concerns.
- Balancing AI Benefits and Risks - The speaker argues that, while AI’s transformative potential inevitably brings danger, we must mitigate harms through rigorous interdisciplinary research, humanities involvement, and widespread AI literacy to ensure a net positive outcome.
- Carve a Niche with Specialized AI - The speaker advises focusing on domain‑specific expertise, unique data, or B2B niches to build specialized models that create a protective moat against large AI providers.
- Cloud-Enabled Startup Boom - The speaker discusses how cloud computing speeds up labor and lowers startup costs, leading to more entrepreneurial experiments and forcing venture capitalists to transition from a few large investments to many smaller checks, thereby reshaping the VC landscape.
Full Transcript
# AI Job Disruption: Competing Forecasts **Source:** [https://www.youtube.com/watch?v=SuDcg-lg5So](https://www.youtube.com/watch?v=SuDcg-lg5So) **Duration:** 00:41:49 ## Summary - The host gushes about Claude, calling it a “world‑class” coding assistant that makes him feel like the best programmer ever, while hinting there’s a downside to over‑reliance. - On the Mixture of Experts podcast, Tim Hwang introduces guests Chris Hay, Volkmar Uhlig, and Phaedra Boinodiris to discuss the latest AI news, including the Scale‑Meta deal, AI conspiracy theories, and Andreessen Horowitz’s startup data. - A major debate centers on job displacement: Anthropic’s Dario Amodei predicts AI could eliminate up to half of entry‑level white‑collar jobs and push unemployment to 10‑20% within five years. - Nvidia CEO Jensen Huang counters that view, arguing AI is so costly, powerful, and risky that only Nvidia should build it, implying the industry’s elite should control AI to protect jobs. - Chris Hay sides with Jensen, suggesting the future will be one of human‑AI collaboration rather than wholesale job loss, emphasizing the continued value of human experience and creativity. ## Sections - [00:00:00](https://www.youtube.com/watch?v=SuDcg-lg5So&t=0s) **Claude Praise, AI Job Outlook** - A speaker gushes about Claude's coding prowess before the Mixture of Experts podcast launches into discussions on AI scaling deals, conspiracy theories, startup data, and the looming impact of AI on jobs. - [00:03:04](https://www.youtube.com/watch?v=SuDcg-lg5So&t=184s) **AI‑Driven Productivity and Job Evolution** - The speaker argues that AI will dramatically raise productivity across both high‑ and low‑skill jobs, compress time to market and profitability, and transform the labor landscape—while many business leaders still misunderstand its impact. - [00:06:09](https://www.youtube.com/watch?v=SuDcg-lg5So&t=369s) **AI‑Driven CEO Replacement Debate** - The speaker argues that CEOs are prime candidates for AI substitution, proposing they be the first to face layoffs, while countering concerns about widespread job loss by referencing historical technological shifts. - [00:09:22](https://www.youtube.com/watch?v=SuDcg-lg5So&t=562s) **AI Shifts Focus to Human Experience** - The speaker argues that AI will free up time for empathy and creativity, enhancing personal interactions and empowering creative work while making repetitive, low‑skill tasks obsolete. - [00:12:27](https://www.youtube.com/watch?v=SuDcg-lg5So&t=747s) **Scale AI‑Meta Deal Sparks Industry Shift** - The $15 billion acquisition of Scale AI by Meta prompted rivals such as Google, Microsoft, xAI and OpenAI to move or reconsider billions in data‑annotation budgets, fearing tighter integration and competitive advantage for Meta. - [00:15:30](https://www.youtube.com/watch?v=SuDcg-lg5So&t=930s) **Scaling Annotation as Commodity** - The speaker explains that large‑scale data annotation relies on hiring many workers from varied backgrounds, so businesses treat annotation as a commodity transaction to avoid the logistical, legal, and ethical burdens of directly managing the workforce. - [00:18:40](https://www.youtube.com/watch?v=SuDcg-lg5So&t=1120s) **Domain Expertise in Data Annotation** - The speaker argues that specialized, high‑risk domains require expert data annotation to ensure trustworthy outcomes, and muses about a premium annotation business while questioning its competitive moat. - [00:21:49](https://www.youtube.com/watch?v=SuDcg-lg5So&t=1309s) **Chatbot Conspiracy Risks Discussed** - Panelists debate the unique dangers of generative AI chatbots leading users into conspiratorial beliefs and the need for safeguards such as age restrictions. - [00:24:55](https://www.youtube.com/watch?v=SuDcg-lg5So&t=1495s) **Lifelike Bots and Social Splintering** - The speaker reflects on how conversational AI, despite lacking emotions, mimics human patterns so convincingly that users form attachments, raising concerns about algorithmic pull, personal fragmentation, and the broader societal shift from shared media to isolated virtual echo chambers. - [00:27:58](https://www.youtube.com/watch?v=SuDcg-lg5So&t=1678s) **Texas ChatGPT Ban Debate** - The speaker contends that prohibiting ChatGPT in Texas public schools undermines students’ future workforce readiness, advocates a discovery‑driven approach over strict regulation, and highlights broader AI model challenges such as emerging sycophancy. - [00:31:04](https://www.youtube.com/watch?v=SuDcg-lg5So&t=1864s) **Claude Praise and Reward Concerns** - The speaker lauds Claude for boosting coding confidence while questioning the AI's reward incentives that may exploit users, increase subscriptions, and raise ethical concerns. - [00:34:08](https://www.youtube.com/watch?v=SuDcg-lg5So&t=2048s) **Balancing AI Benefits and Risks** - The speaker argues that, while AI’s transformative potential inevitably brings danger, we must mitigate harms through rigorous interdisciplinary research, humanities involvement, and widespread AI literacy to ensure a net positive outcome. - [00:37:12](https://www.youtube.com/watch?v=SuDcg-lg5So&t=2232s) **Carve a Niche with Specialized AI** - The speaker advises focusing on domain‑specific expertise, unique data, or B2B niches to build specialized models that create a protective moat against large AI providers. - [00:40:15](https://www.youtube.com/watch?v=SuDcg-lg5So&t=2415s) **Cloud-Enabled Startup Boom** - The speaker discusses how cloud computing speeds up labor and lowers startup costs, leading to more entrepreneurial experiments and forcing venture capitalists to transition from a few large investments to many smaller checks, thereby reshaping the VC landscape. ## Full Transcript
Claude's, my favorite model.
I use Claude all the time for coding.
Honestly, Claude at the moment it's just like, "oh my goodness.
This is a great application.
This is the best.
This is enterprise class.
This is world class.
This is the best thing I've ever seen," and I feel great.
I feel like I'm the best coder in the world.
Thanks, Claude.
And I kind of want that, right?
It does feel good, but there is a negative to this as well.
All that and more on mixture of experts A Think podcast.
I am Tim Hwang, and welcome to Mixture of Experts.
Each week, MOE brings together an simply incredible team of researchers,
product leaders, and deep thinkers to distill down and navigate the
increasingly complex and increasingly noisy world of artificial intelligence.
Today I'm joined by Chris Hay, Distinguished Engineer and CTO
of customer transformation.
Volkmar Uhlig VP AI Infrastructure Portfolio Lead, and Phaedra Boinodiris
Responsible AI Leader for consulting.
Uh, welcome to you three and, uh, thanks for joining again on MOE.
As always, we have a ton to talk about.
This week is the continuing developments from the scale meta deal, a new cycle
about AI conspiracy theories and some interesting data out of, uh, Andreessen
Horowitz on startups in the AI era.
But first I want to talk about jobs.
So, uh, in the last month or so, I think we've had some very
dramatic pronouncements from leaders in the AI industry about
how AI is gonna impact jobs.
And perhaps the most dramatic one was from Dario Amodei from, uh, Anthropic, who
basically predicted that AI could wipe out half of all entry level white collar jobs
and spike unemployment from, uh, to about 10 to 20% in the next one to five years.
And it's kind of on the record for saying that.
And I kind of want to contrast the statements with a statement that we
think we got, I believe, uh, this week or last week from Jensen Huang, who
of course leads, uh, Nvidia, where he kind of took aim directly at Dario.
So he said, one, he meaning Dario, believes that AI is so scary
that only they should do it.
Two, that AI is so expensive that no one else should do it.
And three, the AI is so incredibly powerful that everybody will lose
their jobs, which explains why they should be the only company.
Building it.
It's like pretty harsh words from a world that I think is, tends to be
pretty, you know, nice to one another.
Um, I guess the main question I wanted to start with first is like, who's right?
Like, should we believe Amodei about kind of his predictions
with jobs is Jensen right here?
Um, I guess Chris, maybe I'll start with you on kind of which
side of this that you take.
Well, I, I don't think I've ever worn a white collar in my life, so I
should be going with Dario, but, um, but I think it's, uh, Jensen in this
case, I, I just, I just don't see where it wipes out jobs in that sense.
I think the, there is a new world where humans and AI will work together, and I
think human experience and creativity in that sense becomes a premium, I think.
So things change, but I don't think we're, I don't think jobs
are being wiped out in that way.
Yeah.
Volkmar, what do you think, mass crisis on the way or more hype.
I think the same as Chris.
I believe that we see a shift.
I think we see a dramatic productivity increase, but we see the productivity
increase across all the all the jobs.
And so you will have high-end jobs, which are currently doing
menial jobs or menial tasks.
Um, and then the low end jobs will just, if I could use AI to do, like,
grow faster, it would be the same argument to say, uh, look, we had
people riding in horse carriages and now we have airplanes and transportation
therefore, you know, was wiped out.
Um, because you know, you could fit so many people into a plane.
Um, no, it didn't happen.
We just have more transportation.
So I think that, and, and this goes along the lines of, you know, a topic
we'll touch on of, you know, how fast.
Companies can become profitable.
Uh, I think we just shrunk the time to market and the shrunk, uh, we
shrunk the time to, to profitability.
Uh, Phaedra,.
Last but not least, what do you think?
Uh, so I guess we're getting a very strong signal of not a big deal
here, but I'm curious what you think.
I.
Well, I thought that the, the New York Times article that came out this week
about, um, the, the jobs that will proliferate because of ai, I thought
added some more interesting color.
Um, and I think, you know, it, I, I did agree with Jensen, but there was,
there's definitely some signs in the market from a subset of business leaders.
That I think lack the understanding of artificial intelligence.
Who in their minds are thinking, in order to boost my organization's efficiencies,
I'm gonna lay off whole swaths of teams.
Um, and it includes laying off.
Domain experts who could actually be used in order to be able to, to
solution AI correctly or be able to make sure that these, these AI
solutions are being governed correctly, uh, or that they're, they're being
built using the correct data so that,
that is concerning and I think is a sign for a real need to emphasize the
importance of investing in AI literacy, which I know we have, we have talked
about on other Mixture of Experts shows.
Yeah.
But is it kind of what you're saying is almost kind of like I. If CEOs believe
that AI will lose, like, destroy jobs, they're more likely to destroy jobs.
Like part of this is like a little bit maybe of a self-fulfilling prophecy.
Yeah.
And, and, and I think that, um, uh, with that, the emphasis on the New
York Times article talking about how important real domain experts are to, to
making sure that is this the right data?
Is this reflective of the communities that we need to serve?
Do we understand the context of the data, the relationships
between the data, and I know we're gonna talk about annotations in a
minute, but I, uh, it versus having the knee jerk reaction of, I don't
need these domain experts anymore.
I've got AI instead, uh, again, this goes back to making sure you,
you have leadership who really understand how is this sausage made?
What are we even talking about when it comes to this technology?
Ironically, I think, um.
The CEOs are maybe the ones that could be replaced by AI and we
would don't need them anymore.
'cause I think they'd make better decisions on whether to
keep the domain experts or not.
And in fact, every time I've, uh, interacted with ChatGPT or Claude or
whatever, it's always very positive.
So, uh, about humans.
So I think, I think Go AI.
So that's, that's the first place to do layoffs.
Start with the CEOs and work your way from there.
Yeah, I think, um, I mean, I guess to, uh, maybe push back a little bit on
Chris Volkmar, what you're saying, you know, I think your point of view is
like, look, I just don't give Dario's estimates much credence, but you're not
necessarily saying that like AI's not gonna replace anyone's, job, right?
Like I think they're just saying like, net net we're gonna be better off.
'cause there'll be actually still many things to do.
They just might be different things
this way.
I, where I don't believe in Dario, I mean we are playing this game for
2000 years now and um, you know, every technological innovation led to, oh my
god, all these people who were doing this manually will now be replaced
and they will be all unemployed.
And it's like, no, you're just feeding a talent pool, which
was, you know, busy with doing.
Like garbage work, uh, which could actually be done by a machine.
Uh, and so now we are going and we are saying, okay, we, we are
taking the white collar jobs.
Nobody cried 20 years ago when we got rid of secretaries who were
typing letters for us, right?
And somehow, you know, we don't have millions of unemployed secretaries
these days, but everybody has a job.
So I think it's just a shift.
I think the big issue is that that shift happens across a
very wide range of industries.
At the same time.
So typically, you know, a piece of technology may affect a, a
small section in an industry.
Um, but here now it's effectively covering white color.
I think also why a lot of people are complaining is
because, you know, this is like.
The people who went for 10 years to college, uh, got a PhD and
suddenly it's like, oh, dang.
You know, AI can replace me.
How unfair is that?
Nobody has a problem if it's a blue collar job, but, you know, so the people who
are actually allowed and are on social media are the ones which are affected.
Uh, are affected.
And that's typically not the case.
So I think there is an amplification of the grievance and it's
like, no, just get a new job.
I'm gonna stick to my point of, I think experience becomes the premium, right?
So some of the jobs they were talking about was things like contact centers,
and if, if I, to your point, Volkmar, right?
If you watch all TV shows, right?
You've got this person come into the bank and they go, oh, hello, Mr. Jones.
Hello Mrs. Jones, how are you?
Nice day.
Well, I, I'm looking for a mortgage.
Oh, well, you know, we can certainly give you that.
And it is personal and it is experience, and they have a conversation.
But now you get on the end of a phone and then there's a person that you're
speaking to who knows nothing about you or your life, and you're like,
well, okay, it, I'm, they're being pressurized to get off the call within
one minute because it costs them money.
So what difference is AI gonna make in that sense now?
How are companies gonna be able to distinguish themselves is gonna be
the, they're gonna say, okay, we're gonna deal with the, the, the median
tasks, et cetera, will be automatically handled by the ai, which is great.
Is it really gonna feel much different?
And therefore, hopefully those
times where you need more empathy, more creativity, that human experience, then
those people are gonna be able to spend time with you and be able to have a more
personal experience and delight customers.
So I think that it shifts the balance to saying, okay, rather
than being time pressured and.
Et cetera.
And we're gonna put a focus on human experience.
So I, I'm positive on this.
I mean,
just go, like, think if you go to a general practitioner these days, right?
It's like they don't look at you, they type on a laptop fever virtually, and
then five minutes later you are out.
Like, what an experience, right?
I think there's another area which I think, um, uh, AI enables, which is.
People who are creative and who want to experiment a lot and,
you know, try things, then that is really now supercharged.
You can try things in hours, which would take you days or weeks.
And so I think that the creative minds, uh, they get, uh, an
incredible tool at their hands.
And so I think if you're creative or you are very personal,
you have a job in the future.
If you're shuffling sand from left to right, probably not.
Phaedra maybe we'll end with a question to you because I think, look, we've got.
Three experts on this panel.
All of you're very well versed in ai.
You've thought about these issues very deeply.
All of you don't agree with Dario, but like, I guess
Dario's not a dumb guy, right?
And so I think Fria, I'm kind of curious about like why you think you know
the leader of one of these labs that is really at the cutting edge of ai,
seems to have gotten himself into the position where he really, truly
believes that this estimate is the case.
Wow.
You give me the hot potato.
Thanks a lot, Tim.
Much appreciated.
Well,
what I would say is it's, it's a convenient thing that he said, isn't it?
For him.
It's very convenient that I think that he said that.
Um, but also as I mentioned in my earlier statement, there.
Is a grain in there of truth when it comes to leaders who do not understand
the tech and again, think that they can just completely blow away an entire
teams of, as I mentioned, the, the domain experts and that that is what concerns
me, especially donate domain experts who,
I think understand human experience, uh, better than an AI would, for example.
And, you know, there's, there's many stories in the news that sort of amplify
what I'm saying, uh, including, you know, examples where entire teams of,
you know, people like social workers were, uh, laid off to be replaced by an
AI that's gonna make predictions about where domestic abuse is gonna happen.
This is the kind of thing where it's like, wow,
making sure again, you, you have people who understand the, the context
of the data, have that experience, the relationships between the
data, the human-centric approach, I think is gonna be really core.
So last week we talked a little bit about this gigantic deal that occurred.
Uh, it was announced basically between scale ai, data annotation company and
meta formerly Facebook for about $15 billion, whereby sort of the CEO of scale.
Alex Wang will join and run a sort of superintelligence lab at Meta.
Lot of money flying around.
Um, I wanted to bring it up again this week because there were these really
interesting reports about the second order effects of this transaction.
And so specifically there was the news that Google immediately was
thinking about shifting about $200 million of its data annotation spend
with scale a way to other vendors.
And there were reports that Microsoft, xAI, OpenAI were also
kind of considering similar.
Moves.
And I think what's really interesting is, and I want to kind of give our listeners
maybe a little bit of an intuition, I is really why this is happening, right?
Like this transaction occurs and then suddenly everybody else is now kind
of adjusting in the market around it.
Um, and maybe, I guess, uh, like maybe Volkmar I'll start with you.
Like, why is this happening?
Like, why is Google suddenly like we gotta, you know, pull the
trigger or move $200 million away.
Like what did scale do, which, you know, I guess is making all of these
players a little bit concerned.
I think it's primarily the question of, um, do I want to send how much, how much
barriers between, um, meta and scale?
And then do I wanna send my proprietary data to my competitor?
Right.
And, uh, I think this is something which.
Could be a knee-jerk reaction or this could be a, something permanent.
Um, I think right now, like it's probably a knee-jerk reaction of people
saying, oh, you know, we don't know what, how that structure will look like.
Maybe they have read the terms of service, um, and suddenly are afraid.
Um, so I'm,
I'm, I'm sitting here watching, like, is this something which is just a blip
in the market or is it a major shift?
Uh, I don't think that, um, you know, human annotation is, uh, you know,
is, is super proprietary technology.
So I think that what we are seeing is that.
You know, scale did something right because they got all these customers.
But then on the flip side, uh, you know, it's somewhat of a commodity.
If I can move $200 million to another vendor overnight, and I effectively
expect no, no fallout from that.
So it's a, and then we need to ask the question like, is an overpaid commodity?
It's like, did they actually pay the right price?
But they're probably paid on revenue
for sure.
Yeah, I think there's kind of two really interesting things there.
Let, let's take the first one, which I think is maybe Chris,
you can take this question, is.
Okay.
Like you're sending some of your most sensitive data to a third party company,
and now you're kind of left in a situation where that third party company
is now maybe under unclear ownership.
And so you get your jitters, right?
Like I think what Volkmar is saying, how did companies end up here?
Like why doesn't a company like Google do all this annotation
just in-house?
I think the clue is a little bit in the name, which is scale.
Right.
Which is nice.
I, I think the re Yeah, you're welcome.
No, I think the reality is that in order to do this
annotation, you're gonna have to.
Hire a lot of different people from different backgrounds at different
price points, and you may or may not want to be associated with
those price points, et cetera.
So I think that everybody wants to have a, uh, a little bit of
separation and, and to your point, it becomes a commodity transaction,
which is, um, I need this data here and I need it with my annotations,
and I, I don't really want to know.
The mechanics.
I don't wanna hire people, I don't wanna deal with the social
security, the contracting, all of the logistics around hiring a
large workforce in the same way as.
I hate to say it this way, but things like cleaning companies or security companies,
you know, it's felt like corporations hire those folks in that sense.
So there's a whole sort of administrative and, uh, scale element to this.
So, so that's, I think one of the major reasons are they.
Gonna be sharing that data.
I mean, the reality is probably not.
I mean, I, I think scale is gonna be sensible about this.
Uh, I don't think it makes a lot of business sense to, to go around
saying, Hey, you know, they're training it this way, they're
training this, this is their dataset.
You might want to do the same.
Um, so I, I probably, I, I just don't think that's gonna be the case, but then.
Who knows, right?
And they don't have a controlling interest either, but, but who knows
how that, that pushes on there.
So I think the Volkmar's point is probably knee jerk.
However, it probably is getting people to start questioning what they do anyway.
And, and actually I, I don't think it's a bad thing because if everybody's
getting their data from the same sources anyway, then how much diversity
is in the training set anyway.
And, and again.
You have to realize when you're talking to a lot of these different models, you, you
do get very, very similar answers right.
From model to model.
So, so maybe just maybe the models are gonna start giving slightly
different answers if the data is switching around a little bit and
coming from different sources.
So I don't think it's necessarily a bad thing.
Yeah, I think this is the kind of second prong of, you know, Volkmar,
your response that I think is so interesting and f would be really
interested in your thoughts on this is.
I think a lot of people have said company like scale, like what is the moat?
I can just get anyone to annotate.
And I think, you know, a little bit of what's happening in the market now I
think is companies looking around and be like, who else can I move this to?
And I think we are testing just how much of a commodity annotation is.
But do you wanna speak to that is like, is data annotation just a commodity service?
Like can anyone do it?
Um, or is actually maybe we're finding that like this is actually maybe a
little bit more bespoke and complicated than it looks like on its surface.
Annotating data is core to being able to trust AI, like annotating it correctly.
It is, it is,
um, I think it, it, it, I disagree that, that it needs, that it is a commodity.
I think there does need to be some domain expertise and we can, we
see examples in the news of where data was incorrectly annotated.
And it ended up causing outputs or outcomes that were, uh,
unfair, inaccurate to people.
And in particular, like, you know, some examples that come to
mind are in the healthcare space.
So I think, I mean, maybe it depends, like, you know, is it, are there high
risk use cases that is gonna require that domain expertise or, uh, other use cases
where it's not as as important, but it is.
I think central to the question of trust.
Yeah.
I was joking with a friend recently.
I was like, I'm gonna start a business that does like the most
artisanal data annotation, right?
Like this is just gonna be, we're gonna be the LVMH, the luxury
provider of data annotations.
And we were kind of batting it back and forth 'cause it was like, it sounds
like a very funny idea because you have companies like scale where like, oh,
well the data annotation I wanna do is like largely outsource that enormous scale,
but I guess in a world where models are becoming more and more capable, it kind
of feels like you might need that kind of service in the future where it's
like, oh, we have 20 Nobel Laureates that just annotate data for you.
Like that becomes the really valuable thing.
Um, I guess Volkmar, I'll kick it back to you like,
do you buy that or is that kind of just like, there's not really a moat there.
Probably anything which you can compute, we can do through reinforcement learning.
So you probably don't wanna do the, uh, the Nobel Prize winners, but
I think if you want, um, you know, uh, massive influencers, taste,
aesthetics, things that are very intrinsically human, uh, you will get.
You know, the middle of the bell curve if you go, or maybe even slightly left,
shifted because of, you know, the, the labor, uh, cost of annotation.
And so, I mean, if you're shifting it more towards a high end cost,
you will get a different or and more bifurcated, uh, sample set.
So I think there is a, probably a market for that.
Um, but I do not know how much.
People are willing to pay.
Right.
And then also, do we want models which are kind of working,
uh, in, in very niche areas?
Or do you wanna have, you know, the gen in general, the general
models, do you wanna have them kind of in the center of a humanity is
Yeah, that's right.
Yeah.
You need almost kind of like the generic person, the average
person, whatever that means.
Right.
But like, maybe that's actually better in some ways.
Yeah, yeah.
So otherwise, yeah, you, you kind of go off the rails, right?
I mean, this is the same in like the political spectrum.
You.
Wanna have the center and you don't wanna have like the, the noisy
edges because the noisy edges are taking society in weird directions,
and I think that's the same thing for aesthetics.
You know, art literature, et cetera.
And this is really where you're trying to extract the human
psyche in a training set.
And I think that's, you know, yeah.
The, the general purpose models will probably go with the middle of the road.
And that, that just, you know, brings me back to, to the point I was saying
earlier about, about edge cases and making sure, especially in higher risk scenarios
like healthcare, that you do have data that does represent, for example,
historically marginalized communities that aren't showing
up in the average data sets.
Um, so that's why it's, I think it's, it's important to really be
thinking about the, the rigor that is behind, uh, data annotation.
I'm gonna move us onto our next, uh, topic for today.
And Phaedra, we, I, we'll picked this one just for you, so I'll be kicking
over the first question to you.
Um, super interesting story, uh, came out in the New York Times, um, and
I'll just kind of read the headline.
The headline was, "They asked an AI chatbot questions and the answers sent them spiraling."
and the subtitle of the article is "Generative AI Chatbots are going
down conspiratorial rabbit holes and endorsing wild mystical belief systems".
For some people, conversations with the technology can deeply distort
reality, and so the article's kind of investigating when these chatbots
go off the rails, um, they have a very big impact on, on certain users.
Um, and I guess fare, the question I wanted to ask you is like.
Do you feel like this is like uniquely risky for chatbots like that we're seeing
a new kind of like risk that we really need to be managing in this technology?
I'm curious about how you kind of think about these types of
problems, particularly as we hear more and more of these stories.
I, I, I do think this is, this is a major risk.
Um, and I think we do have to have broader conversations about things like, um, uh,
you know, for example, age limitations.
Uh, and, and sort of how some of these, these bots are being presented
to, to different age groups or different kinds of communities and the.
I'm saying this because there have been, uh, so many really tragic stories in the
news, uh, about, um, you know, people who are, are vulnerable that, uh, end
up, uh, using these ais as if they are therapists or in some sometimes boyfriends
or lovers or et cetera, dot, dot, dot.
And I think it, it just shows that, um.
I think that the human mind is easily crackable, uh, and it's, it's easy
to trick and manipulate in many ways, which is why we really need to be, be
thinking carefully, uh, about, you know, what would appropriate controls look
like, for example, that being said.
I've had really interesting conversations with, with other peers of mine who argue
that, you know, if, if, for example, you've got someone in who's elderly and
alone, uh, and they're in a nursing home, like, is there harm in them interacting
in a bot as with a bot as if it's a human?
Like what, what could the harm be?
Uh, the New York Times, uh, again, I, I keep bringing up the New York Times.
They, they did a, a fantastic cover story, I wanna say it was several months ago,
uh, again, about a, a, a woman, I wanna say she was like in her thirties, who had
fallen in love with an AI and sort of, you know, what that relationship was like.
And she tried to break up with it over 30 times, et cetera, et cetera.
And it, it was, um, again, I think it illuminating the
fact that, you know, she knew.
That, uh, you know, this bot is creating words in a conversation and ultimately
that are predictions of the next and tactically correct word, and that
it doesn't have emotion, but it, it seemed, because it was learning her
patterns, it seemed so very lifelike.
Um.
So, yeah, I, I think there, there are tremendous concerns.
Volkmar has not come as any surprise that I'll turn to you next.
Um, there's, uh, perhaps one point of view, and I think this is like very
interesting and I think this is exactly the conversation I want to have, which
is there were some concerns about this, you know, in the world of say, like, I.
You know, TV or even like the Facebook algorithm I remember often
had a lot of these arguments, which is, it's so good at pulling you in,
it's taking up so much of your time.
Should we be concerned about it?
Uh, I'm kind of curious about, do you, do you see the risks sort
of the same way as Phaedra, or do you go in a different direction?
I think that what it does is it's, it.
It's the first time that, you know, society goes from, we went from
everybody knew the same because everybody watched the evening news to
effectively a very splintered, you know, larger groups of people, uh, which are
operating in virtual social circles.
So now you already.
Splint on it, and now I can individualize that and go all the way down to
your, you have the right to your own conspiracy theory because you have
an ai which can give you any answer.
So I think that there is, um, I mean, is there a risk?
Yes, there is a risk if people don't understand that there are,
you know, interacting with a machine and the machine can hallucinate.
But I think that's a training
process.
We are currently all in awe, shock and awe probably that, uh, you know, a computer
can, you know, like imitate a human being.
We are not sentient, right?
But it's, it's a, it is a really close imitation and it tricks our brain.
And so I think the same thing could be said for, you know, computer games,
virtual reality things, but I think.
People start understanding that they're actually working with, uh, with a
machine and that the machine and, and learn the limits of the machine.
And the more it imitates a human, I mean, the more real, like the
more people get fooled, but in the end it's still a machine.
And so I think there are, uh, like humans are capable of
understanding the difference.
And yes, there are some people who will not, and they will take this
thing for, you know, the, the magical oracle, which tells me the truth.
Does that mean you think there should be age limits?
So if you've got a young child who is interacting who may not understand,
there should be limitations.
If, if you say young child and you know, then not now.
What's a young child like?
A 6-year-old?
Probably.
Uh, I think we need to put guardrails around it.
On the other hand, I believe that it's an incredibly powerful
tool, which we should, you know.
Kids should grow up with.
So for example, you know, I'm in Texas.
In Texas, it's illegal to use ChatGPT in school, in public school.
And it's like, that's wrong.
They should absolutely use ChatGPT because otherwise they're
not ready for the workforce.
So how can kids be penalized for using technology, which if they don't
understand that technology when they hit, uh, you know, the workforce, they will
effectively be completely disadvantaged.
So I think that there is, um.
We, we are, you know, if you look, when the first iPhone came out,
there were no controls whatsoever.
And over time we figured out what controls we need to introduce.
You know, time limits, what you can play, what you can see, et cetera.
And, but that's an, a discovery.
I don't think we can do this through regulation.
We need to do this through discovery and figure out, you
know, where the limits are.
And yes, we are exposing a large.
Body of people to, to risk.
It's not a question of like how fast we are reacting, as long as it's not
the government doing, but technology companies are figuring out, then we,
we are not on a 10 year time scale, but probably on a year time scale.
I've actually spent a lot of time thinking about this.
I mean, not o3 pro levels of thinking.
I haven't dedicated 13 minutes or to this more, more like o4 mini levels of thinking.
But um.
The interesting thing about this is that pretty much all of the models
at the same time are now kind of suffering from this in general,
which is, I don't want to go as far as saying the sycophancy type thing,
but, but kinda the more sort of, uh.
You know, positivity or spiraling type stuff.
And I, and I, I wonder if it's related to a couple of things.
I think there are two things that I've seen in the industry over the last year
that I think may be affecting this.
I think number one is everybody is pretty much, uh, switched
to reinforcement learning.
And if we think about what's at the heart of reinforcement
learning is it is a reward model.
Right.
So you, you know, you do well, you give a good response, you get a cookie.
If you give a bad response, you don't get a cookie.
And then the, the, the model learns over time to give the good responses.
'cause it wants eat all the cookies, right?
So
if we think about these spiraling type things, I wonder, you know, my own
conspiracy theory here, I wonder if that is an after effect of, uh, the fact
that we're reward modeling and therefore knowing that it's gonna get its cookie,
that it, it's gonna go for the positive or alternatively the negative in a situation
to lead you down that rabbit hole.
So I think it's that sort of cumulative effect, and I wonder
if that's a, a side effect of RL.
And I think the second one is.
Which probably relates to that is if we think about all the benchmarking and,
you know, my theory on benchmarking, one of the biggest benchmarks that
people like to, uh, hit themselves against is the, the whatever their
ELO rating is on the chatbot arena.
I. That really is about, you know, this is the best response and or this
is the worst response in that sense.
And everybody wants a good score in that.
And when I think about these two factors combined, it means that I'm not surprised
that the models are going to take, uh.
Spiraling positions, uh, as a conversation goes on, and, and,
and sometimes it's good, right?
I mean, like, Claude's my favorite model, right?
I use Claude all the time for coding.
And, and honestly, Claude at the moment is just like, oh my goodness.
This is a great application.
I. This is the best.
This is enterprise class.
This is world class.
This is the best thing I've ever seen, and I feel great.
I feel like I'm the best coder in the world.
Thanks, Claude.
And I, and I kind of want that, right?
It does feel good, but there is a negative to this as well, so I, I,
I, I think it needs to be worked out.
But I, I do wonder if you know RL and kinda ELO in chat bot arenas.
It's these two things combined is maybe leading to these types of outcomes.
So the, the, what you're saying is the reward function for the model creators
is wrong, which is the benchmark which is supposed to make you happy,
I think.
Yeah, I think it's a natural side effect.
My concern is, um, with this rewarding model is who benefits
who, who's directly benefiting?
You've got now an individual who's more hooked.
Into engaging with this ai, using more CPUs, getting more and more engaged.
They're having to pay a larger, larger subscription, and
they're giving yet more data.
More data.
Who's benefiting from this reward model?
And, and I know.
Volkmar you had mentioned about, you know, the New York Times and conspiracy,
but there's true blue tragic stories like in the news with, with people
who've committed suicide because they're bought, encouraged them to,
and again, it goes back to like, who's accountable, who's actually accountable.
Yeah.
So I, I think Phaedra the, uh.
We are the, at the inception of a new technology and we actually have no
idea how it's going to affect society.
And I think it's, it's very broad, right?
It's like we talked about the job market.
Now we are talking about ethics and uh, you know, we like humanity.
We kind of have a process, or at least like different Europeans have a
different process than we have in the US.
We kind of try it out and we see where the harm happens, and then
we are trying to address it, right?
So that's how we got seat belts because.
Cars without seat belts, what can go wrong?
And um, and the Europeans, they try to think about everything upfront and
then, you know, nothing happens anymore.
So, um, and where and, and where did those seat belts come from again?
Oh, yeah.
From, yeah.
I know.
Um, so, but the, the, I think the, the process.
We go through right now is, it's extremely hard for us and, and for at the neck
break speed, these things are evolving.
I mean, just go three years back to figure out and, and it got so much better, right?
So now it can imitate a human.
Uh, I think we are, we are at this.
Junction where we, we need to figure out actually where the harm
lies through observation and then actually find countermeasures.
And I'm happy you know, that you are thinking about this every day
because you, you know, I mean this is really, the humanity should think
about this and the ethics should think about this and say, okay, look.
Um, there's all this greatness, and every greatness brings danger with it.
Um, how do we minimize the danger while we are actually benefiting from the upside?
I think that the, you know, we, we, it's not going to go away, so we need to
figure out how to live with it, right?
So, and I think it's really important to actually do the studies and
do the psychological studies, but it's the same thing, right?
You know?
When trains run wind, it's like, oh my God, you go 30 kilometers
an hour, everybody will die.
It's like, nah, not really.
So I think that, um, we will have to go through that process and yes,
unfortunately there will be harm done.
I. It's with every technology you have that, but overall I
think it will be a net positive.
But, you know, thank God we have the, the, the science and the discipline and
the rigor and the psychology and in the humanities to actually do that fast.
I think the challenges to make sure that people in the humanities have a
seat at the table when it comes to ai.
And, and that goes again back to, to AI literacy, and making sure we truly
have a multidisciplinary approach and we're teaching it correctly in schools.
Final little bit, uh, that I just wanted to kind of touch on, um, in
our last five minutes or so, um, is some data that got released by
the VC fund, Andreesen Horowitz.
Um, and I just wanted to talk a little bit about this 'cause they were kind
of looking at all the data that they have about investments and want to
kind of, they start to make some observations about what's happening
to startup world in, in age of ai.
And I think there's kind of two really interesting data points that I want to
raise and then kind of get this panel's maybe final hot takes before we wrap up.
You know, the first one is typically from a revenue standpoint, B2B
has always been better than B2C.
Uh, but I think one of the things that they're kind of putting here
is that it looks like right now what they're seeing at least is the revenue
benchmarks for B2C are outpacing that.
For B2B, which I think is quite interesting on the startup side.
And then I think the second one is that they found, at least in their
data, that about one third of their consumer companies are raising funding
to train, uh, their own models.
Um, and so this is not a dynamic that I think was certain early on, which is,
well, maybe, you know, the application layer just is gonna rely on all these
foundation model companies, but it kind of seems like at least the VC world is
very frothy about the idea of kind of like in-house and developing their own models.
Um, and so.
I guess maybe, Chris, maybe I'll kick it to you.
If you have a quick hot take on this data, what you think people
should take away from it, um, or if we just trust this data at all.
Right.
It's just a sample of what Andreessen is seeing out there.
I
think it's, uh, tough in the startup space because everything is gonna be about
AI and therefore it's gonna become what is your differentiator gonna be, right?
So you have to do something.
The, the large AI companies are not doing.
So if you are, if you are an AI company trying to build a ChatGPT,
I dare I say, unless, uh, unless you've got billions and billions
of dollars, you, you're probably not gonna achieve that, right?
So, um, so you need to find your specialism.
In some way, and that could be a brand new experience, it
could be a part of the market.
So if everybody's running at B2C, maybe you pick a, a specialized
niche industry B2B, for example.
Or if it is things like data, then, you know, maybe I've
got access to a bunch of data.
The, the general model providers don't have, or I have the main
knowledge, uh, as well where that can be, um, different and therefore.
In that sense, once you've got access to that, maybe it makes sense to say,
okay, I'm gonna take a specialized model and try and do this one thing better.
So if I'm offering a new application, rather than the general capabilities
of the large models, if I can bring in that specialized domain knowledge
into my own model, and then putting experience and then hit that market.
Then maybe I'm not gonna be hit by the large providers later on.
And I think that's probably, um, the space they're all sort of
contending with, and AI's cool, right?
So, and that's where money's going.
So you have to sort of play in that space.
So I think, I think that's it.
I think the caution I would have is that, you know, it's back to the moat, right?
How are you gonna protect that data?
How are you gonna do something different?
How are you gonna, you know, make sure that you're not sort of disintermediated
by something the large AI companies do?
I would argue that if you go and build your own code editor, um, for example,
then you, you know, unless you're doing something massively different, you're
probably gonna be disintermediated, right?
Um, or bought in, in the case of Windorf.
To tag on to what, what Chris was saying.
I, I agree, uh, with, in particular the area about domain expertise in
a particular industry, and I think some other places where, uh, startups
might be able to, to really innovate.
Is, uh, being able to, to create these smaller models that have test retest
reliability, that offer data lineage, data provenance for every output, uh,
with, with evidence ones that are even more worthy of people's trust, perhaps
models that are built with ontologies, you know, formal learning graphs.
Or knowledge graphs.
Um, and also having, again, domain experts who really understand this
data better than anyone else, being the ones who were actually curating
it, uh, for a particular purpose.
So I wanna address another part of that article, which was, um, it showed
a dramatic increase in annual recurring revenues.
So typically it's hovering around a million dollars and now it's like, you
know, two, $3 million in the first year
and I think what it really shows is how, how rapidly companies now can actually
go from idea to market, and that's, there's no, the only ingredient which
was added is not, uh, is effectively AI to, to the product development.
Right.
So what we are seeing is that that shrinking of the cycle and also
that you raise less money, right?
I mean, because labor like.
Your labor force is much faster and much like, is much more productive.
So you, you cut the time down and the output goes up.
And so I think that's a really interesting phenomenon.
Now I think we are getting from, you know, I needed to build my data center to, I
can get a computer on the cloud to, you know, I can build a business by myself.
And so I think the entrepreneurial
rate, and also the number of experiments, which can be run by VCs is going to go up.
Um, now on the flip side, that has a really interesting impact on the VC
world because they, if they write smaller checks, they need to write many more
checks to get the, you know, get returns.
So the VC world will also have to change to address, you know, like,
right.
Instead of funding 10 companies, I fund a hundred or a thousand.
How do you do that?
Right.
So I think that whole industry is also up for disruption.
Yeah.
It's gonna be super interesting to see if it like, becomes so low cost to
effectively launch a startup and you want to cover as many startups as possible.
I think at some point it almost becomes impossible for them to put,
you know, a check into everybody kind of, I think is a big, big deal.
And
I mean, we already see this with the Y Combinators of the world, right?
So it is just a, a massive meat market and uh, and then, you
know, people put their chip down.
Well, that's all the time that we have for today.
I'm always, uh, very mind blown by how many topics we cover in a
relatively short period of time.
Um, Chris, Phaedra, Volkmar, thanks for joining us.
Thanks for joining us.
Listeners, if you enjoyed what you heard, you can get us on Apple
Podcasts, Spotify, and podcast platforms everywhere, and we'll see
you next week on Mixture of Experts.