Terms

• EDR: Endpoint Detection and Response
• IDS: Intrusion Detection System
• IoT: Internet of Things
• DLP: Data Loss Prevention
• SIEM: Security Information and Event Management
• XDR: eXtended Detection and Response
• GDPR: General Data Protection Regulation
• ISMS: Information Security Management System
• MDR: Managed Detection and Response

Introduction

Traditional IDS technologies such as Snort and Suricata are detection systems that follow a mechanism that requires them being deployed in a choke point of the network, usually on a firewall device that protects a critical node of a system. They also require maintenance by continuous update of the rule set as soon as new attack patterns emerge.

However, modern networks are implementing policies that bring these technologies to their limits, as the number of devices as well as systems to protect have increased and the security events they generate has become unmanageable.

It is also more limited in terms of the kinds of policies a system administration team can enforce on a fleet of corporate IT assets such as the employee laptops, identity management and remote access administration.

Once an incident occurs, traditional solutions have limited logging capabilities that usually fall short except for the most mature and stable of networks, which makes the forensic process extremely challenging.

It is difficult to identify the point of entry of an intruder, the set of steps they have taken to laterally move across the organization and what resources have been compromised.

The solution was to deploy a special type of host IDS, which was to be know as EDR or endpoint detection and response systems in the industry. An EDR is a solution applied to continually monitor an endpoint, which generally involves some kind of client device such as a smartphone, laptop, servers and IoT Devices to mitigate cybersecurity threats.

Until this point, the main paradigm to mitigate threats on the endpoint devices was to install an antivirus software. But as soon as metamorphic engines became the norm in common malware implants, the signature-based engines of these antivirus systems drastically plumbed in detection capability. Even if detected, the events generated from these systems were insufficient to assess if it was an isolated incident or activities by a persistent advanced threat against the organization.

In this article we will enumerate the technical requirements of such a system, how it is deployed, and what are the main closed and open source options in the market.

History

Using Google Trends we can establish a timeline for this term on the Internet and we can observe that it became a popular search term after 2015 and interest has been increasing linearly ever since.

Some sources establish Anton Chuvakin, one of the main authors in the topic of logging and monitoring, as the coiner originator of this idea from an article at Garner in 2013. Since then, Garner has updated their website and the article is now unreachable, so we are unable to establish him as the author of this term.

However, there is another article from Anton Chuvakin, related to the same area from 2017 in which he shows that cybersecurity is a data problem, due to the large volume of security events being generated on a large organization, so it needs big data solutions.

The solution up to that moment is to build a custom SIEM for the target organization, which was and still is a challenging endeavor due to the knowledge and talent required to build an maintain such a system.

The increase of data generated in organizations and security events also pushed existing centralized systems up to their limits, requiring the distribution of those analytic processes into the service edge of the network, on the nodes where they occurred, and aggregating that data using cloud computing resources.

From an observability and network monitoring perspective, existing tools were also limited in the sense that they were only able to establish a succession of events occurring in certain parts of the network, missing the whole picture of an incident and thus making it impossible to derive what had happened and how to make sure it never happens again.

This showed the need for a new kind of service that could be used to map events over the complete set of assets of a corporate network, aggregate them, and finally execute actions over the whole IT fleet.

However, it is hard to pinpoint exactly which was the first product to be marketed as an EDR, because it is more of an evolution on the ways to detect incidents and threats from the previous paradigm of SIEM engineering and DLP policies.

This is precisely the thesis of Palo Alto Networks, which sees its current EDR solution Cortex XDR, as an evolution of its previous products.

The XDR or eXtended Detection and Response solution, would be an extension of a existing EDR infrastructure to include other systems such as email or messaging in order to include additional data flows in the organization as well as executing actions upon known attack signatures.

The EDR market took steam in 2020 when the whole world went into lock-down and remote work became the norm in most enterprises. Since then, it has been increasing at an annual rate of 22.3%.

Common features

An EDR has similar requirements to antivirus technologies in the sense that it is a device agent with very high privileges on the installed system, allowing administrators to setup policies and control access to corporate data on employee-owned devices.

Some features include:

• Real-time behavioral event analysis of endpoint nodes.
• Historical visibility of memory discs and devices in endpoint nodes.
• Regulatory compliance for the handling of data in endpoint nodes, such as GDPR personal data.
• Organization-wide password requirements on the user-facing endpoint nodes.
• Data encryption policies on endpoint nodes.
• Remote wipe capabilities to protect against data breaches and theft in endpoint nodes.
• Managing a fleet of devices from a centralized platform.

These features are delivered via a Dashboard User Interface in which all this information is aggregated.

Most EDR products are not useful out of the box, but provide a set of building blocks that be used to assemble a ISMS of the organization according to the information system security plan of the company.

Departments, identity management systems and company specific security policies or data regulation are configured into these tools by specialists or consultants and automatically maintained, so that a baseline of rules, alerts and processes for the organization is established.

The information of events, assets or artifacts found on endpoints is also enriched by third party services such as Virus Total or dedicated antivirus solutions, usually directly managed by the EDR commercial provider.

This allows the deployment of an anomaly-based IDS to automatically set alerts and notifications from trusted data sources by creating a comprehensive timeline of system behavior via a process of continuous logging and data collection.

How to use an EDR

Deploying an EDR by itself will not protect an organization, it requires a team of professionals to build, maintain and extend it, as well as considering additional tools.

The security strategy of any organization is given by a specific ISMS of the company, which includes a risk analysis of the cyber threats it is expected to face. For an EDR to be effective, it needs to fit properly within the ISMS and integrate itself with existing technology choices of the company.

The threat hunting process

The main professionals that are users of these systems are what Anton Chuvakin mentions on an article at Garner as threat hunters, which execute a more engaged, creative and proactive approach than a typical SOC operator passively waiting for screen alerts.

If the organization does not have the capability to hire its own threat hunters, it usually externalizes these services to a MDR agency.

Threat hunting involves following a proactive and analyst-driven process that iteratively looks for indicators of compromise, tactics, techniques and procedures used by attackers and anomalous behaviors that evade traditional detection mechanisms by following a structured life cycle involving these steps:

1. Hypothesis generation rooted in existing threat intelligence.
2. Data Scoping and identification of the data sources to test the hypothesis.
3. Investigation and pivoting using structured queries, pattern matching, or behavioral filters, analysis search for signals aligned with the hypothesis.
4. Validate if the collected evidence supports or disproves the initial hypothesis.
5. Action and escalation to the incident response team.
6. Detection and telemetry feedback to generate SIEM correlation rules for the future.
7. Documentation and retrospective analysis.
8. Continuous improvement

Available options

Any business has two budgets, one for time and another for capital. Generally closed source or proprietary solutions optimize the time budget while requiring capital to keep them going. The open source solutions optimize the capital budget if the time invested into engineering a well adapted solution is not taken into account.

Other factor might be geopolitical, we might not trust some closed source vendors due to their alignment towards certain governments, which means that the organization will need to focus on deploying an open source solution instead or develop their own.

From a business continuity perspective, having a well maintained open source solution running in-house can have a very good long term return on investment, especially if we are talking about a MDR team that will have his own solution to provide services towards a set of clients.

However, a small team for a large corporation will benefit much more from their time investment if they use a plug-and-play proprietary solution that allow them to focus into the details of implementing the particular ISMS for the company.

For a student or security researcher trying to understand how these solutions operate from an engineering perspective, open source is the way to go, as it will provide the most transparency and flexibility to maximize learning and experimentation.

We think that the evaluation of each solution is outside the scope of this assignment, it is leaved as an exercise to the reader.

Open Source Solutions

A complete and updated list can be found on Awesome Endpoint Detection and Response tools, which will be a more useful and up to date list on this topic than whatever the author of this article can provide.

Proprietary Solutions

To assemble this list we used a study from Grand View Research on the EDR Market and scores by Gartner, from which the top 4 are the following:

1. CrowdStrike
2. Sophos
3. SentinelOne
4. Fortinet

Other big EDR systems on the market.

• Bitdefender
• Broadcom, Inc.
• Cisco Systems
• ESET
• FireEye
• Kaspersky
• McAfee
• Microsoft Corporation
• Palo Alto Networks
• Trend Micro
• VMware Carbon Black
• Greynoise Intelligence

We however propose the reader to be exhaustive in their research and see which one adapts better to the needs of their organization and the security policies they wish to implement. As can be observed, most of these names are branded with old antivirus companies, giving weight to the theory that EDRs are an evolution of this concept.

Conclusions

On this report we have extended upon the topic of how modern intrusion detection and response is achieved in companies by focusing on the nature of the tools as well as the work processes involved to make them useful.

Finally, a list of closed and open source solutions has been provided for the reader to keep researching towards the particular problem they want to solve.

The next point of action for a student trying to increase their knowledge of cybersecurity would be to either get certified in for a proprietary EDR solution or to tinker with one of the open source ones.