AI Note‑Taking: Promise vs Reality
Key Points
- The current hype around AI‑powered note‑taking apps mirrors earlier VC bubbles, but the speaker remains skeptical and wants to assess their real value.
- Studies show workers waste roughly 10 hours a week (about 25% of their time) searching for information across Slack, Docs, and other sources.
- Corporate note‑taking suffers because the underlying data is “dirty” and LLMs struggle to interpret temporal cues and revision histories that humans rely on when evaluating relevance.
- Most personal and corporate note‑taking systems are abandoned not out of laziness but because the maintenance costs outweigh the benefits, and the promise of AI is to make those systems worth keeping.
Sections
- AI Note‑Taking: Hype vs Reality - The speaker critiques venture‑capital hype around AI note‑taking apps, emphasizing the massive time wasted searching for information due to dirty data, and argues for a realistic, grounded use of LLMs to improve knowledge retrieval.
- LLM Limits in Enterprise Knowledge Management - The speaker critiques AI‑driven tools such as Glean, noting high costs, integration hurdles, and hallucination risks that undermine reliable extraction of corporate decisions and information.
- AI‑Assisted Judgment and Auto‑Organization - The speaker contends that, while AI lacks human taste and judgment, using AI tools like the Sparkle auto‑filing system provides indispensable wisdom and seamless organization, compensating for our memory flaws and chaotic file management.
- Optimizing LLM Semantic Search - The speaker emphasizes feeding large language models clean, well‑structured data and using them for semantic retrieval while recognizing their limitations and building consistent note‑taking habits.
- Building an AI‑Powered Second Brain - The speaker stresses organizing the LLM layer over your data so AI can act as a digital librarian, enhancing note‑taking habits and providing sustained cognitive lift that makes a long‑term “second brain” possible.
Full Transcript
# AI Note‑Taking: Promise vs Reality **Source:** [https://www.youtube.com/watch?v=JdTgxpfCa3E](https://www.youtube.com/watch?v=JdTgxpfCa3E) **Duration:** 00:14:51 ## Summary - The current hype around AI‑powered note‑taking apps mirrors earlier VC bubbles, but the speaker remains skeptical and wants to assess their real value. - Studies show workers waste roughly 10 hours a week (about 25% of their time) searching for information across Slack, Docs, and other sources. - Corporate note‑taking suffers because the underlying data is “dirty” and LLMs struggle to interpret temporal cues and revision histories that humans rely on when evaluating relevance. - Most personal and corporate note‑taking systems are abandoned not out of laziness but because the maintenance costs outweigh the benefits, and the promise of AI is to make those systems worth keeping. ## Sections - [00:00:00](https://www.youtube.com/watch?v=JdTgxpfCa3E&t=0s) **AI Note‑Taking: Hype vs Reality** - The speaker critiques venture‑capital hype around AI note‑taking apps, emphasizing the massive time wasted searching for information due to dirty data, and argues for a realistic, grounded use of LLMs to improve knowledge retrieval. - [00:03:09](https://www.youtube.com/watch?v=JdTgxpfCa3E&t=189s) **LLM Limits in Enterprise Knowledge Management** - The speaker critiques AI‑driven tools such as Glean, noting high costs, integration hurdles, and hallucination risks that undermine reliable extraction of corporate decisions and information. - [00:06:26](https://www.youtube.com/watch?v=JdTgxpfCa3E&t=386s) **AI‑Assisted Judgment and Auto‑Organization** - The speaker contends that, while AI lacks human taste and judgment, using AI tools like the Sparkle auto‑filing system provides indispensable wisdom and seamless organization, compensating for our memory flaws and chaotic file management. - [00:09:30](https://www.youtube.com/watch?v=JdTgxpfCa3E&t=570s) **Optimizing LLM Semantic Search** - The speaker emphasizes feeding large language models clean, well‑structured data and using them for semantic retrieval while recognizing their limitations and building consistent note‑taking habits. - [00:12:53](https://www.youtube.com/watch?v=JdTgxpfCa3E&t=773s) **Building an AI‑Powered Second Brain** - The speaker stresses organizing the LLM layer over your data so AI can act as a digital librarian, enhancing note‑taking habits and providing sustained cognitive lift that makes a long‑term “second brain” possible. ## Full Transcript
You know, the joke is the peak of
venture capital is when you get excited
about note-taking apps. And we have now,
by that measure, hit the peak of the AI
cycle because people are talking about
AI powered note-taking apps. So, let me
make instead an honest case for LLMs and
note-taking. And I'm telling you, I'm
coming from a somewhat skeptical
position. And I want to start by
explaining how bad it has been for how
long with note-taking and how important
it is before I set up kind of where I
want to go. We waste about 10 hours a
week searching for information. That's
not me. That's actually studies done on
workers. Like roughly a quarter of our
working time is spent looking for
something, looking through Slacks,
looking through Docs. Now, I know that
there are tools that claim to do this.
There have been tools that have claimed
to help us do this since before Windows
introduced the folder system to most
people as the PC rolled out. And almost
always the data is dirty. In fact, one
of the things I talk about in other
videos is how the dirty data inside
businesses isn't valuable as much as
people think because it is so dirty and
because of the way LLMs process
information. As a very trivial example,
you as a human look at a wiki page and
you look at the updated date and you
look at what is new at the top and you
say, "Aha, I now know what I need to pay
attention to." or oh my gosh, this was
updated, you know, six years ago by
someone who is no longer with the
company. I'm not going to do anything
with this page at all and I'm going to
go ask an actual human, which is what we
do like 80% of the time. But if you do
see something useful, you know how to
observe it. LLMs, even if they can read
the wiki, don't always know. They don't
because they process information as an
entire semantic context. The idea of
linear time affecting updates is not
intuitive to LLMs. One of the challenges
with most notetaking systems in
corporate contexts or even at home is
that we have this implicit idea of the
timeline. It is today therefore I'm
going to make a diary entry at its
simplest or you know it is the 23rd of
June and I'm making a entry into my
project folder to talk about what I've
worked on in the weekly status review
with the engineering team. And then we
abandon it eventually. Hopefully we keep
up with our diaries. You never know. But
but we abandon most of our note-taking
efforts eventually because they seem to
add nothing. We write things down. We
don't know whether the program manager
is paying attention. We're tired at home
and it's 10:00 at night and we don't
really feel like taking notes because
who's going to read our diary of the
day? We're just going to skip today. The
abandoned notetaking setup, it's not
laziness, it's rational behavior. The
cost of maintaining these systems
exceeds their benefit. And the promise
of AI is that that is going to change.
And I want to talk about how much of
that promise has come true and how much
of that promise we still have to make
come true cuz it's not all guaranteed.
Fundamentally though, things should
change with LLMs. LLMs don't just make
search better. Ideally, they eliminate
the need for organization entirely.
Think about it. Why do we organize?
Because computers are dumb. They need
exact matches. They need proper filing.
They need consistent naming. But what if
your computer could understand contacts
like a colleague would? Look, if you can
dump a message transcript in and ask,
"What did we actually decide and watch
the LLM extract the decisions?" That's
not just an incremental improvement.
It's a paradigm shift in the way we
organize information. This is what has
made Glean a valuable company for the
enterprise. Now, no one's recommending
Glean for your personal note-taking
system because it's like 50 or 60 grand
to start. And I've used Gle. It's okay.
It reads a lot like a chat GPT4 model
that suddenly got access to corporate
data. And even that is somewhat
questionable because Salesforce is
apparently cutting off access to Slack,
which is the living, breathing backbone
of information for a lot of companies
unless you're using Teams. Maybe Glean
will be more of a Microsoft angled
company going forward. We will have to
see. That's that's speculation. But
let's be honest about what doesn't work.
It's not just the Mark Beni offs of this
world saying you can't get access to our
data because we value data in the age of
AI. It's that LLMs hallucinate. I've
watched, we just did a case study on
this. We talked about the LLM Claudius
that ran the vending machine. Claudius
made up a colleague named Sarah who did
not exist. That happens. I've watched
them quote policies that do not exist.
There was an entire lawsuit about that
with Air Canada and a bereavement policy
that I've talked about. Stanford has
suggested that in the workplace in
actual use cases, it's a 15 to 20%
fabrication rate. That seems really
terrifying. Why on earth would I be
advocating for that if if this is in a
business context and we have to get this
right? Well, I'll tell you why. Because
at the end of the day, any incremental
forward progress, if it is correct, is
better than nothing. And so what that
suggests to me is that if in the
previous age of computing, our problem
was file organization and we had to bend
our brains to make them work like
computers do today. In this world now
where AI sits, our fundamental problem
is good judgment. We have to have the
judgment to say, "Hey, Sarah's not a
colleague. Sarah doesn't exist. Try that
again." Or, "I'm going to go look at the
sources on this one." And that is the
trick that we have to trade in order to
use these AI note-taking tools the way
we need to. And I don't want to sit here
and pretend that there's something
magical that's going to take that
hallucination rate to zero. There are
absolutely tricks you can do that reduce
it. You can ask more precise questions.
You can install systems that will give
the LLM the option to say, "I don't
know." You can install systems that give
the LL system prompts that give the LLM
the encouragement to ask questions when
it's confused. There are things you can
do that materially reduce hallucination
rates. Clean data is a good help, too.
But you're not going to get it to zero,
which means that your most valuable
skill has moved from can I organize like
a machine if I want to collect
information to can I name and label
appropriately and then can I go and get
it and have the taste to see when it's
wrong if the LLM comes back badly. It's
like you have a magical fishing net with
an LLM and sometimes it brings something
up that is fool's gold and it's not real
and you have to tell the difference.
That taste and judgment is what we're
missing. And it's ironic that it shows
up in so many places because it's almost
like we have some universal truths
coming with this computing revolution.
We always talked about the value of
wisdom as humans. Now we have to show
wisdom and judgment to use our computers
because the computers take care of a lot
of the other things. The computers will
remember for us and sometimes they will
invent memories which by the way is very
human. Humans invent memories too and we
have to tell the difference between an
invented memory and the real thing. So
what I want to suggest to you is that
despite all these drawbacks having an AI
and a note-taking system is eminently
worth it. You want to be in a position
where you can just heap things and it
will just magically work.
I I have been a devoted fan of a product
from every called Sparkle for a long
time. Sparkle is very simple. All it
does is it gets rid of the filing
problem, which is a huge deal for me. I
am not a good filer. I'm not a good
organizer. My local hard drive, every
computer until now has been a complete
mess. Sparkle makes that go away. All
Sparkle does is it automatically runs on
my downloads folder. It automatically
characterizes it into a neat series of
folders by type of data. That's it. Very
simple. Not necessarily the
organizational scheme I would have
chosen if I'm being very honest with
you. But I don't have to care because I
know where stuff is now because the
organization system is rigorously
followed and because I can easily
search. And so even if something is not
fully AI enabled, having automations
like that is a huge cognitive load
lifter. And having optimizations like
that combined with AI, that's where the
value is. Look, there are all kinds of
options for note-taking. I run through a
few. There's Obsidian, there's MEM,
there is notion. I like notion. I put a
lot in Notion. I find Notion's search is
very helpful. Notion allows me to kind
of do my little like throw stuff in the
junk heap habits and I can still find
stuff pretty reliably. Notion also
understands the idea of recency and the
introduction of AI has made it very easy
to add and create and hybridize notes
together the way my brain works. But
everybody's different. I'm not saying
use notion, use Obsidian, use me, use
something else. The point is find a way
for the AI to take some of the cognitive
load off so that you can throw things in
a heap and you can go after what you
want with easy search and focusing on
your good taste and your good judgment.
Now, if you are someone who finds deep
relaxation in organization, that's also
fine. You can still find AI systems that
will allow you to define the
organizational hierarchy and then search
across that. The larger value is still
there. The larger value is that semantic
meaning is not something you have to
remember anymore. Semantic meaning is
something that the AI can help you
remember. Now, there's weaknesses to
that, but guess what? As a human, you
already have those weaknesses. You are
also a semantic meaning maker. And so if
you're searching for something, you're
like, "No, no, no. It's not, you know,
it's it's not like the project manager
and the product manager are similar.
It's like the project manager is
actually connected to this project." I
do that in my head all the time. We are
meaning makers and semantic makers in
the way our neurons make memories. So do
LLMs. They do something similar when
they encode things in vector space. And
so our job is just to set up systems
that enable those LLMs to search
semantic memory appropriately. Clean
data. Maybe don't keep the six-year-old
wiki in there. Make sure that you have
clean markdown. Make sure that you're
comfortable with the file structure. For
me, I don't need to define it. Other
people do. And make sure that you are
using the AI for what it's good at right
now, which is very much semantic,
meaning search, and not for what it's
not good at. It is not good at reliably
getting everything correct. If you got a
keyword search in Windows and the
keyword hit, you know, 100% of the time
that keyword hits. That is not true. And
that is a big difference in search. It's
very fundamental to how AI works and we
have to get used to the idea that we
need to challenge these systems, but
they still add tremendous value because
of the cognitive load they lift the
other 80 90% of the time. Net net,
they're worth it, but you have to be
aware of what you're doing and give them
as clean a data as you can. So, pick a
tool, commit to it, recognize that the
incremental value is the habit you're
building. It is not any individual
retrieval. It is not any individual note
you take. and then lower the barrier to
note-taking. One of the beautiful things
about AI is it also has simplified
note-taking. If you use Granola, I use
Granola. You know what I mean? It's
super easy. You get the transcript right
there. You get the notes right there.
It's not hard. Other people use other
things. People use Otter. Some people
are using Chat GPT's native
transcription. I don't like that as much
because it just sort of hides from you
and then it gives you very generic
notes. I tend to have big surprise
opinions about my notes and I like to be
able to write up custom prompts against
the transcript for the notes. do what
you want. The beautiful thing is you can
actually use AI and its ability to
organize semantic meaning to quickly
organize and reduce the cognitive labor
to put the notes in to your note-taking
app in the first place. And then you can
use AI to search across that. I can use
AI to tag my notes, which I would never
have the discipline to do otherwise. But
by tagging the notes, it makes it more
easy for another AI to find it. And this
is actually not creating synthetic data
in a way that is likely to accelerate
information decay because
the individual steps can be easily kept
an eye on by a human me in this case
just going to let say oh look you know
you applied the wrong label or oh look
the label's right which it almost always
is and one of the things about AI is
those really dramatic hallucinations
that are unhelpful tend to arise in
large multi-step complex situations like
when Claudius went off the rails and had
what I can only describe as the LLM
version of a psychotic break on March
3rd during the vendor uh experiment and
then recovered on April Fool's Day
spontaneously in a way none of us
understand. It was engaged in a month'sl
long complex effort to run a vending
machine with minimal tooling and no
access to the vending machine
physically. I would describe that from a
human perspective as being under a fair
bit of stress. When an LLM is simply
asked to summarize 30 minutes of notes,
I actually rarely see issues. And so
it's important to understand the task
sizing and the retrieval scope when you
are doing this note takingaking a note
architecture exercise. Big surprise.
This is what I say a lot on this
channel. If you put thought into how you
structure the LLM layer on top of your
data layer, you're going to be in better
shape. So netnet, what I want to leave
you with is this. Your brain evolved to
think. It did not really evolve to file.
We've been doing filing for a while
because our computers have been stupid.
But now AI can help help by playing
librarian. It may not be a perfect
librarian, but having a librarian at all
for our data and our memories is really
helpful. LLM can enable you to focus on
what matters and the thinking is what
you need to have good judgment when they
are not great. The question is whether
them helping you is better than you
going it alone. And in particular
whether the cognitive lift you get from
a note-taking system with AI enablement
and support is enough to keep you in the
habit of keeping notes long term so that
over time as you have a good note
takingaking body of work the second
brain can really start to come into
focus. The value of a second brain is in
all of the effort together. It is not in
any individual effort. You have to stick
with it for a period of time in order to
make it work. And that's why I think
it's such an important subject right
now. We have to spend our time thinking
better. And so a good second brain is a
huge step in the right direction for us.
And LLMs can be a big help. And I wanted
to take a minute to just unpack what
makes them difficult to work with, what
makes them easy to work with, why I
think they're a breakthrough in this
whole effort around note-taking.
Everyone I know who I have studied who
is considered a genius or someone who's
an inventor has had some kind of
note-taking system or some kind of
notebook. I don't think that's an
accident. Having a second brain was
actually a skill that was taught in
Scottish universities. It was called
common placing. We have been doing this
for a long time. Now we can do it with
the help of AI. It may not be perfect,
but I would sure rather be here than I
would be trying to make the fountain pen
write the ink right in a Scottish
university in the 18th century.