Bridging SIEM Gaps with Federated Search
Key Points
- Attackers typically remain undetected for roughly 300 days because organizations lack full visibility into all their security data.
- SIEMs aggregate logs from various security devices to provide near‑real‑time alerts, but many sources—such as endpoint detection tools, legacy systems, or newly acquired SIEMs—often remain unconnected, creating “SIEM gaps.”
- Expanding SIEM ingestion to cover every data source quickly becomes cost‑prohibitive, forcing teams to prioritize only the most critical streams for real‑time monitoring.
- A federated search approach can complement SIEMs by leaving data where it resides and retrieving relevant information on‑demand, reducing storage costs while still supporting alerting, forensics, and threat hunting.
Sections
- Visibility Gaps and SIEM Limitations - The speaker explains how attackers remain undetected for months due to blind spots in traditional security monitoring, describes how SIEMs aggregate logs to provide near‑real‑time detection, and highlights common integration gaps—such as missing endpoints or legacy SIEMs—that limit their effectiveness.
- Leveraging Federated Search for SIEM - The speaker describes how integrating a SIEM with federated search allows data to stay in place, reduces costs, and enhances visibility by combining bottom‑up and top‑down investigative approaches.
Full Transcript
# Bridging SIEM Gaps with Federated Search **Source:** [https://www.youtube.com/watch?v=3MsxAwEshfk](https://www.youtube.com/watch?v=3MsxAwEshfk) **Duration:** 00:04:50 ## Summary - Attackers typically remain undetected for roughly 300 days because organizations lack full visibility into all their security data. - SIEMs aggregate logs from various security devices to provide near‑real‑time alerts, but many sources—such as endpoint detection tools, legacy systems, or newly acquired SIEMs—often remain unconnected, creating “SIEM gaps.” - Expanding SIEM ingestion to cover every data source quickly becomes cost‑prohibitive, forcing teams to prioritize only the most critical streams for real‑time monitoring. - A federated search approach can complement SIEMs by leaving data where it resides and retrieving relevant information on‑demand, reducing storage costs while still supporting alerting, forensics, and threat hunting. ## Sections - [00:00:00](https://www.youtube.com/watch?v=3MsxAwEshfk&t=0s) **Visibility Gaps and SIEM Limitations** - The speaker explains how attackers remain undetected for months due to blind spots in traditional security monitoring, describes how SIEMs aggregate logs to provide near‑real‑time detection, and highlights common integration gaps—such as missing endpoints or legacy SIEMs—that limit their effectiveness. - [00:03:11](https://www.youtube.com/watch?v=3MsxAwEshfk&t=191s) **Leveraging Federated Search for SIEM** - The speaker describes how integrating a SIEM with federated search allows data to stay in place, reduces costs, and enhances visibility by combining bottom‑up and top‑down investigative approaches. ## Full Transcript
You can't secure what you can't see.
In fact, surveys tell us that on average,
the bad guys are in your system for almost 300 days before it's realized.
Now, why is that?
It's because the systems that we use to discover this
oftentimes have gaps in their understanding,
and you can't secure what you can't see.
So what do we use to look for these kinds of incidents when someone's breaking in?
Well, we use a technology called a security information and event management system,
or I'll call it a SIEM for short.
And what we do with a SIEM is we take different devices that we have in our environment.
Let's say maybe a firewall, maybe a network intrusion prevention system,
maybe a system where I'm going to take all the logs from that system.
I take all of that information and forward it up into the SIEM
where I have a database where that information is stored,
and then I have a security analyst up at the SIEM that takes the information
once it's been collected, aggregated, correlated, and we apply a security policy to it
in order to find out, in near real time, when someone has broken into our systems.
So that ought to work really well.
And generally it does.
But there are some limitations.
Why do we have limitations?
Well, it turns out that a lot of organizations have other systems
that are not feeding into their SIEM.
Maybe an endpoint detection and response system.
Could be that through an acquisition or merger
you have another SIEM that you acquired from the other organization and it's not feeding in.
And then a lot of organizations are creating things they call data lakes,
where they're collecting security information in order to do some of their own analysis
using their own data scientists that are in-house.
And all of these things collectively amount to a "SIEM gap".
These are the gaps that hurt our visibility.
What I need is a more integrated view that allows me to see all of these things at once.
Well, what's preventing that?
In many cases, it's cost.
It's the fact that the more information I put up into my SIEM,
the more expensive the SIEM gets.
That's the way most of these are priced.
So there's some information I need for real time alerting
and other information I might need for forensic analysis,
for threat hunting, for after-the-fact kind of analysis.
So what I really would like to have
is something that complements the capability that I've just described with a SIEM.
And I'll suggest to you that could be something like a federated search capability.
In federated search, instead of bringing all the information up,
as we did here with the SIEM,
you notice this is a very bottoms-up approach,
in fact, what I want to do with Federated Search is leave the data in place
and then go out and retrieve it just in time.
So you notice I don't have the massive database where everything is collected in advance.
I can have just the information that needs to be into the SIEM there
and have it then serve as a way to let me know
when I need to do an investigation.
Now my analyst sits over here
and they're able to see the alarms that come in.
And, in some cases, I'll tie the SIEM back into the federated search capability.
So now I have the ability to search all of these other areas that had been dark before,
as well as the SIEM, in an integrated system.
And what this does for me now with Federated Search, I leave the data in place.
Why is that useful?
Leaving the data in place helps reduce cost.
So we lower the cost because I'm not sending all the information into the SIEM.
By leaving the data in place, I have a more efficient operation.
A more efficient operation means I don't have to process all of that information in advance.
I can go get the parts that I need in advance
and the rest of it I go get just when I need it.
So I suggest to you that this bottom up, complemented with a top down approach,
give you the best of both worlds.
As I said at the beginning, you can't secure what you can't see.
So we need to fill the gaps in the visibility that we have.
We need to get rid of the SIEM gaps.
And this is a way to do it.
Thanks for watching.
Please remember to like this video
and subscribe to this channel
so we can continue to bring you content that matters to you.