Navigating the Unseen: AI Latent Space
Key Points
- The speaker likens today’s AI experience to early internet hyperlink discovery, emphasizing a nostalgic sense of uncovering knowledge beyond simple search.
- He argues that the core challenge with large language models is our failure to understand or visualize their “latent space,” which underpins how they generate outputs.
- Current prompting tricks, development workflows, and tool-specific guides are essentially workarounds aimed at nudging LLMs through this poorly understood latent space.
- Start‑up companies are capitalizing on this gap by packaging unwieldy models into more stable, user‑friendly products, but this merely masks the deeper knowledge deficit.
- Visual attempts—like a digital map of a chain‑of‑thought that looks like a tangled “rat’s nest”—illustrate how little we actually grasp about the structure of latent space.
Full Transcript
# Navigating the Unseen: AI Latent Space **Source:** [https://www.youtube.com/watch?v=bcitUtdi2qM](https://www.youtube.com/watch?v=bcitUtdi2qM) **Duration:** 00:11:25 ## Summary - The speaker likens today’s AI experience to early internet hyperlink discovery, emphasizing a nostalgic sense of uncovering knowledge beyond simple search. - He argues that the core challenge with large language models is our failure to understand or visualize their “latent space,” which underpins how they generate outputs. - Current prompting tricks, development workflows, and tool-specific guides are essentially workarounds aimed at nudging LLMs through this poorly understood latent space. - Start‑up companies are capitalizing on this gap by packaging unwieldy models into more stable, user‑friendly products, but this merely masks the deeper knowledge deficit. - Visual attempts—like a digital map of a chain‑of‑thought that looks like a tangled “rat’s nest”—illustrate how little we actually grasp about the structure of latent space. ## Sections - [00:00:00](https://www.youtube.com/watch?v=bcitUtdi2qM&t=0s) **Navigating AI's Latent Space** - The speaker likens the thrill of early hyperlink browsing to today’s AI experience, arguing that our failure to visualize and understand the latent space of large language models limits effective interaction and prompts a proliferation of ad‑hoc prompting tricks. ## Full Transcript
I'm going to take you back if you are
like gray bearded like me you remember
before Google when we were using the
internet and we depended on hyperlinks
and you remember that sense of Joy when
you like find something and you couldn't
have found it by search because search
didn't exist and there was this little
tip and then you went to the hyperlink
and you found a page that's what AI is
like right now that's what using AI is
like and look I don't want another
Google of AI although we might have one
with open AI to be honest but we do need
something that enables us to more
effectively understand what we have with
artificial intelligence and if you look
at it as a problem of
understanding if you look at it that way
you have so many different dimensions of
this problem that crystallize into a
single core issue and that's what I want
to talk about I would argue that
effectively we don't know how to
navigate latent space we don't
understand what latent space in large
language models is like we haven't
visualized it well haven't understood it
properly and everything stems from that
as an example all of these annoying
prompting tips and prompt this and
prompt this way and like tell it you
want it to be amazing at its job and
tell it you're on vacation in France and
tell it this and tell it that I kid you
not every time I turn around there's
another one of these things they're all
about trying to nudge the llm through
late and space but we don't really
understand it another example that's the
same core issue all of these building
tips that basically are like this is how
you tell cursor or Bol or lovable or
wind surf or pick your tool of choice
how to build something tell it like this
give it a full Dev plan break it into
chunks do this do that it's all about
basically telling a large language model
how to navigate Laten space to produce
tokens of
code and if you go farther a field if
you're talking about how to make
marketing posts about how to produce
content for customer success emails
about how to write sales emails or sales
scripts again you run into the same
issue where effectively companies are
monetizing the inability to navigate
late in space a lot of the solutions
being shared or built or monetized or
built through YC or whatever you have
are basically ways to take these large
unwieldy models and productize them into
something that is a lot more stable and
a lot more consistent and that's not bad
I don't object to companies doing that
being able to basically Bally take an
intelligence and package it is a totally
legitimate service but I do think it
raises the fundamental question it
highlights the fundamental question if
you're watching properly because it
underlines the fact that nobody really
has a good grasp on latent space and we
certainly do not have a good grasp on
how to talk about latent space I saw a
digital representation of a single Chain
of Thought running through an l M and it
looks like a Rat's Nest like it's like
running all through this like visualized
uh latent space and of course that's not
how Laton space actually looks so it's
like this colored string running through
and I was like wow this looks
complicated and that's how I walked away
and at best that's where people are at
like I'm the fluent one the people who
don't even know what late and space are
are scratching their head saying how did
the llm come up with a
sentence
and we are so lost on communicating how
this works that we don't have good
answers to people who are at that level
we certainly don't have good answers to
how to use the technology for people
we've given people a chat screen and
said here's a chat but people are used
to talking with other human beings
they're not used to talking with a
hyperdimensional intelligence that
navigates through Laten space to answer
their queries and a lot of the issues
come from the fact that they treat the
chatbot as if it was a person in some
cases they treat it as if it was a
person with the expectations of a
computer and we've talked about that
where you sort of expect the computer to
be perfect and so you expect the AI to
never make a mistake but by and large
they treat it like a
person make sure you answer me in this
way or write this email to Bonnie in
this style or hey I'm having a bad day
today those companion apps definitely
make money um and
so I think that we can get farther if we
are more honest about about how weird
these things are llms are really weird
it's weird that they work it's not
necessarily
intuitive and we kind of got farther on
the Internet by just acknowledging it
was a new thing like hey here's the
Internet it's not like the newspaper you
can click on links now and you can go
new places and oh there's a search
engine so you can search for anything
it's not like a newspaper or a book
imagine a card catalog but you can
search the entire world those were the
kinds of things we talked about we need
that kind of language for large language
models we need it to be like imagine a
world where you
have an intern that has read every book
ever written but it's still kind of dumb
or imagine a world where you have a very
specialized Professor who knows
everything there is to know about
biochemistry but you'd never trust him
out at dinner because he doesn't know a
whole lot of
or imagine a world
where you need to get an answer to
something and you're going to get six
answers and none of them are right but
all of them are interesting and they
will help you get to the right answer
we're not doing enough of that kind of
communicating and we're not demystifying
it when we portray it as a secret when
we portray it as here's a tip from an
expert it's not helping it makes people
think it's hard and I don't think it
does any of us a service who know AI
well if we keep portraying it as
something that is difficult to practice
something that is difficult to try
something that it is difficult to
execute on at a high
level because honestly it's not let me
give you a specific example here we'll
talk about building for AI which is one
of those
things that people to like if you're an
engineer you sort of know how to build
but then you have to learn how to build
with AI and if you've never built or
never coded you just sort of scratch
your head and like stare at the wall and
you don't know what to do let me try
something on with you like we're
actually going to try and solve this one
as an example of how to solve this stuff
better I have eight steps that I think
you can walk through with anyone even
someone who hasn't built an application
before and say look this is kind of how
you work with an AI on this number one
figure out what you want to make you can
use an AI to brainstorm and that's as
hard as it needs to get right brainstorm
come up with some solutions brainstorm
some ideas for
features um and then you kind of want to
think about what you don't want to have
in right and now in a product management
sense that's scoping I don't need to use
the word scoping for that I can just say
what you don't want to have in now I
will say as I go through these eight
this doesn't mean that if I can explain
this well everyone in the world is going
to be come an AI Builder just like I can
explain cooking but not everyone's going
to become a chef but we can still
communicate clearly so let's go to
number two
architecture you want to outline how the
AI is going to work because as you can
imagine the AI has to use information
where does that information live does it
live on the web page do you want to
change it much does it live in the
library behind the web page are you
going to have payment you want to
understand those pieces and you probably
want to work with an AI to figure out
what all those pieces are and then start
to
brainstorm what you need to build it
might have some fancy words like API it
might have a a word like a database in
it which is really a fancy word for a
data library but at the end of the day
you're going to come out of a
architecture and Technical planning
conversation and you're going to say to
yourself I understand where the data
goes because that's really all it is and
AI can help with that once you
understand where the data goes you have
to understand what the data looks like
we would call that data schema how you
actually structure the data AI can help
a ton with that because if you know what
the data is AI is pretty good with
arranging
it number four setting up your building
world now if you're building something
simple it comes preset up you can go to
lovable you can go to bolt or you can go
to cursor and wind Surf and you can just
set it up yourself with some simple
rules either way it's like clearing the
table and getting set up to build a
model you want to make sure that you're
set up correctly you see how I'm using
these analogies all the way through I'm
not just doing this because I think you
don't know how to build I'm doing this
because I want to practice communicating
well and show how important it is to
communicate well with a real example
that I run into all the time backend in
database implementation is what you
start with after you've decided to start
to build and you could say that really
simply by saying you know what if you're
building you want to start with a
foundation the foundation is the library
of information you want to make sure the
library of information is solid so you
can pull information in and out of it if
you build the front of the website first
and just build the page you'll have a
nice looking website with no library of
information behind it you'll be in
trouble after you build the library of
information you're going to want to
build the web page you look at that's
the part you're probably really excited
about but you see it's 6 we've had to be
really patient to get here just like
building a great model airplane you may
want to put the wings on and make it
look really pretty at the end but you
have to
wait and by the way if if this is all
feeling child level I have in the back
of my head my own kid who I have to
explain this stuff to and so if I can
explain it to her I can explain it to
anybody okay number seven you're going
to have to test to see if it works so
make sure when you're building you
include testing test if the data runs
into the library of data test if it runs
back that's really important finally you
want to put it somewhere where other
people can get it we call that
deployment there are apps for that you
can see where I'm going that's the eight
steps to build now I ran through them
really quickly I'm sure you can find
better ways to communicate that it
doesn't have to be at the level of a
9-year-old which is basically what I did
but we do need to find simpler ways to
explain how llms work and what they do
for us and that's the heart of the point
that I want you to take away it's really
important to be clear about that so give
me your best takes how can we get better
at explaining these weird large language
model
intelligences so that it's easy to
understand and other people can
understand what we're trying to talk
about and share and why it's so cool