Endpoint Security Part 3: How to correlate threat intelligence
Overview
In this third article of a 5 part series on 'Endpoint Security' I'm going to be showing you how you can incorporate 'Threat Intelligence' into your security operations strategy.
There has been a lot written about threat intelligence over the last decade and it has different meanings to people depending on what industry you are in, what you are trying to achieve and how you intend to use it.
For the purposes of this article and in the context of cyber security, threat intelligence is about collecting information that can immediately enrich the security event data you are already collecting and be acted on by your security team to better defend your organisation from attack.
STOP: If you are not collecting the right events from your systems, application, network devices and cloud systems then you are not mature enough and threat intelligence data sources will not help you. Check out the previous Endpoint Security article 1 and article 2 to get yourself started.
Threat Intelligence Data Sources
There are a large number of data sources of where you can collect 'Threat Intelligence' data. Some of these data sources will be external to your organisation and some will come from within your own organisation. Threat Intelligence is dynamic in nature and overtime the goal posts will keep shifting and this will ebb and flow as the criminal entities up their attack campaigns or conversely sit and wait. Some data sources you may have to pay for but there are a lot of Open Source Intelligence (OSINT) sources to get you started on your journey. Some examples of these are:
- Virus Total
- Shodan
- Alien Vault OTX
- Maltego
- Spiderfoot
- HaveIbeenPwned
- MISP
- GreyNoise
There are of course commercial paid sources of threat intelligence but they may not be for you and not necessary when first starting out. Some examples of these include:
- Crowdstrike
- Mandiant
- Palo Alto Unit42
Threat Intelligence data types
In the most simplest form threat intelligence data could be simply a list of:
- Known bad IP addresses in use by cyber criminal infrastructure
- known bad domain names / urls
- File hashes
It could also be in the form of amore comprehensive report detailing the operations and Tactics, Techniques and Procedures (TTPs) in use by the particular criminal organisation. Mandiant and CrowdStrike produce some fantastic detailed reports on criminal adversaries and their particular modus operandi and TTPs. Examples of these include the likes of APT1, APT 28, Scattered Spider and more.
How to collect threat intelligence data
I'm going to be showing you how to collect threat intelligence data from Alien Vault OTX and send it to Elastic for correlation with your existing event data.
Step 1 - Create yourself an Alien Vault OTX account and generate an API Token.
- Navigate to the Alien Vault OTX website here.
- Once signed up and logged in you will need to generate an API token
- To do this select your profile name in the top right corner and then select settings.
Then in setting section either generate a new token if it is the first time you have done so or regenerate a new token. Then copy this token as you will need it in the next steps.
Step 2 - Install Elastic Filebeat
I'm not going to cover all of the steps on how to install Filebeat you can get that information here.
You also don't have to install Filebeat you could install the Elastic Security Agent and apply the Threat Intelligence integration to it with a one click policy. I'll be showing how to do that in another session.
Step 3 - Configure Filebeat and enable the Threat Intelligence module
- You will need to firstly edit your filebeat.yml file and configure the required credentials to connect it to your elastic cluster.
- Next you will need to enable the Threat Intelligence module. You can do this from the command line using the following command.
filebeat modules enable threatintel
- Once enabled then edit the 'ThreatIntel.yml' file which is in the modules.d sub directory.
Step 4 - Configure the Threat Intel Module
There are a number of sections in this file for different threat intel data sources including:
Abuseurl, Abusemalware, Malwarebazaar, Misp, OTX, Anomali and ThreatQ
To enable these change the enabled setting from 'false' to 'true'.
So for Alienvault OTX it will look like this:
otx:
enabled: true# Input used for ingesting threat intel data
var.input: httpjson# The URL used for OTX Threat Intel API calls.
var.url: https://otx.alienvault.com/api/v1/indicators/export# The authentication token used to contact the OTX API, can be found on the OTX UI.
var.api_token: < ENTER YOUR API TOKEN HERE># Optional filters that can be applied to retrieve only specific indicators.
var.types: "domain,IPv4,hostname,url,FileHash-SHA256"# The timeout of the HTTP client connecting to the OTX API
var.http_client_timeout: 120s# How many hours to look back for each request, should be close to the configured interval. Deduplication of events is handled by the module.
var.lookback_range: 1h# How far back to look once the beat starts up for the first time, the value has to be in hours.
var.first_interval: 400h# The interval to poll the API for updates
var.interval: 5m
Step 5 - Restart the Filebeat Agent
To do this on linux run the following command:
service filebeat restart
To check the status run the command:
service filebeat status
If it successfully started you should see something similar to the output below:
/etc/filebeat/modules.d# service filebeat status
filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2023-12-20 02:39:46 UTC; 35min ago
Docs: https://www.elastic.co/beats/filebeat
Main PID: 119079 (filebeat)
Tasks: 7 (limit: 1121)
Memory: 76.9M
CPU: 17.786s
CGroup: /system.slice/filebeat.service
└─119079 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path>
Now we can check the Kibana dashboard to see that we are ingesting threat intelligence data from our OTX source.
Step 6 - Configure a correlation rule to detect a match against your existing data and the threat intel data
- In your Kibana instance navigate to Security - Rules - Detection Rules (SIEM)
- Then search for threat intel and enable the rules as shown below
Step 7 - Create an Alert to be notified whenever a match is detected
For each rule that you want to receive an elert for, edit the rule and add an alert action. In this example we will add an email action for every time the alert triggers and send this to a specific mail box. There are other alert action such as sending to a ticketing system, pager duty for an sms notification and more.
- For the particular rule select 'edit rule settings'
- Then select the connector type in this case email.
- Configure the mailbox, subject line and body of the email.
- Then save your changes.
That's it for now. You will need to fine tune the intelligence data that you ingest over time as well as tuning your detection matching and corresponding alert rules. The key is to start simple and match your intelligence sources to the datasets you are collecting accordingly.
Want to know more?
If you would like to know more or need assistance with your security operations and detection engineering then please get in touch via the website or you can book a 30 minute no obligation meeting with me below.