Considerations for Leveraging Cyber Threat Feeds Effectively
There are a few different approaches that I have seen organizations take when it comes to consuming threat intelligence. Teams that have been in the threat intel space for a long time started with informal sharing distros (We don’t talk about fight club!) with unorganized TXT files serving as their community standard. These teams probably evolved their sharing efforts into using CSVs, maybe even with some basic structure, before moving on to helping build out projects like STIX, openIOC, etc.. These analysts then started looking at open-source and commercial feeds with automation and integrations — or even starting their own Threat Intelligence Platform (TIP) companies.
On the other side of the spectrum, you have organizations building threat intelligence capabilities from scratch and attempting to go from maturity level 0 to 5 in a single project. These organizations usually get a Threat Intelligence Platform (TIP) with a flashy demo- maybe even a #pewpew map!- And they turn on every threat feed available. In a few short months, they blame their TIP for feeding them too many false positives and their threat feed providers get tagged with “too little; too late.” Many of the problems these organizations face are that they have not gone through the crawl and walk phase of capability building and attempted to purchase their way straight to the run phase of operations.
The good news is that many of us have experienced the hard-fought battles of building an intelligence capability and we are often more than willing to share our lessons learned. Usually, this exchange is over a beer at a post-event happy hour, but I’ve decided to share my experiences a little bit broader than my personal circle - just offer to grab a round if you find this helpful in your organization. This post will cover my thoughts on managing threat feeds within an organization. I plan to follow up later with my thoughts on Threat Intelligence Platforms (TIPs) and cyber threat information sharing standards.
Without further ado, here are my recommendations for implementing a basic cyber threat intelligence capability.
Threat Feed Evaluations
As with all projects, organizations must have a serious conversation about threat feed requirements, processes, and standards before turning on commercial and open source feeds. Once your team has decided to ingest feeds automatically, it is recommended that organizations only turn on 3–5 feeds at a time and run them for at least 60–90 days before turning on additional feeds. During this period, analysts and managers should regularly be talking about the number of false positives in the evaluated feeds and the quality of the context in them.
When do you turn on feeds? If you have a security appliance that receives indicator and signature feeds directly into it, that is a separate discussion that must occur between threat intelligence leadership, detection and monitoring leaders, and security engineering. I generally recommend turning these feeds on and working with the vendor to tune out issues. However, I have heard some mature organizations argue that they only use their own custom signatures. There are pros and cons in this approach that must be discussed internally at each organization.
The absolute worst thing you can do to your organization is trying to play the "catch 'em all" game — this isn’t Pokemon, and your analysts may quit when they end up with an 85% false positive rate with thousands of unreviewed alerts pending in their queue. (That 85% FP rate isn’t made up either; it is rounded from real analysis on a previous project.) Remember, one of the goals of threat intelligence is to prevent alert fatigue- not cause it.
One method to evaluate how your feeds are impacting your organization is to assess the actions that they lead to for your team in a short 60–90-day project. For example, have an analyst review all of the hits (i.e., IOC in feed matches traffic in the environment) and tag them as False Positive, True Positive Escalated, True Positive Mitigated. The definition of Escalated here is that the existing security stack did not block the event, so action was required to scope and mitigate the activity. The meaning of Mitigated here is that the current security controls detected and mitigated the event. These metrics will help you better understand how much good work vs. busy work your feeds are causing within your environment.
A second method is to give your analysts a rating questionnaire and have them run an evaluation on the feeds on a 1–5 scale with criteria like the quick list below. The output is that your team ranks each individual feed, and then someone runs the averages to see how the feeds compare to each other.
Context
Is there enough context around the IOCs to understand how they are used in an event/attack?
Timeliness
Is there a big gap between Date Sighted and the day you receive them?
Ease of Use
Does the feed provide all malware hashes and your organization has to run these manually?
Does the feed provide mostly IPs and resulting in your analysts have to run additional analysis against them before safely deploying?
Is the feed auto-fed into your TIP or does your team have to download them from a site manually?
Threat Feed Considerations
I recommend staying away from threat feed vendors where the primary business line is IOC feeds. Traditionally, these companies focus too much on quantity over quality, and you’ll end up with a giant dump of IPs with minimal context. The other challenge here is that they often deduplicate and anonymize their feeds before pushing them to you- you’ll see this same challenge with TIP vendor’s curated feeds. This is a huge hindrance to your analysts since they will not be able to tell if IOCs are being widely observed across multiple sectors or if other elements of context are being lost in the deduplicate process. Worse still, if you cannot do fundamental supply chain analysis of your IOCs, you may duplicate feeds on your end that your vendor is also pushing you. This costs you money in paying for feeds needlessly, and it creates an echo effect for those IOCs. In a world where algorithms are now telling us what’s essential, this could mislead your system to calculate higher confidence scores.
As an analyst, I have to say the most egregious crime in anonymized threat feeds is not being able to see the real source of the intel. How can analysts effectively assess their sources for confidence and accuracy if the data is coming from mixed sources that are also anonymized? Trust grows in the light.
Sourced from: http://www.littlebobbycomic.com/projects/week-104/
When it comes to cyber threat indicators, context is king. If a vendor has a feed of indicators and it doesn’t include context for each indicator in a Description field, you are not buying indicators- you are buying Observables (more on that in a later article on cyber threat information sharing standards).
Each organization will set their own standards for their threat intel providers. Here are a few key aspects to consider:
Is the feed provider the source of the intelligence or are they repacking IOCs from other sources?
The original source is always preferred since the company will likely stand behind their analysis or be available for an RFI around their info
Bundled feeds can make it harder to evaluate the quality of IOCs because of deduplication processes and anonymization (more on this later)
Do the indicators include enough context to be quickly actionable?
A Description like “This email delivered this URL which led to…”
What type of malware is this malicious hash?
What were the domains that were observed with these malicious IPs?
Is the feed causing unacceptable false-positive rates for your organization?
Each org has to determine “acceptable” rates for their teams
Food for thought: one team I worked with in private sector once told me that they would turn off my organization’s threat feed after three strikes of wasted time- even though our feed was 100% free, it still could cost them money in wasted resources
Is the feed providing too many IOCs that are mitigated by other security appliances?
Example: the majority of IOC matched in your logs are all in blocked events because IPS detected it already
Note: this is okay to have the Intel still enter the TIP, but you may consider tuning them out from entering your SIEM.
Does your feed include properly marked benign indicators?
Example: malware calls out to Google DNS to see if it can reach the internet
While 8.8.8.8 is a horrible IOC for detection, this benign IOC may be a critical piece of intelligence to have in the TIP
Remember- IOCs!=malicious code (more on that in a later post)
Commercial, Community, and Open-source
What about paid feeds vs. open-source? I believe that most organizations are best suited with a blended approach for their threat feeds. Commercial feeds can benefit an organization because they are often backed by intelligence teams that are available to discuss their analysis with your team when support is needed. Generally, these feeds are more mature and will follow established intelligence standards in their analysis. Community feeds, such as Information Sharing & Analysis Centers (ISACs) can provide your organization with intelligence and IOCs directly relevant to your business sectors. They also offer a venue to discuss best practices for the security of sector-specific issues. Open-source can be an invaluable resource for timely intelligence on emerging threats. Major campaigns often first appear publicly on Twitter long before a blog or threat report is written. Organizations should invest the time to evaluate which open-source feeds, blogs, and personalities they should follow for the most up-to-date information available.
Commercial Vendors
I won’t pump one vendor over the other, but I will provide some basic advice here. As budgets allow, I recommend purchasing at least one premium intelligence service from vendors like CrowdStrike or FireEye- both if the budget allows. In my opinion, the quality of the context from these companies is unmatched, and these companies do a fantastic job enforcing traditional intelligence principles in their analysis and production cycles.
Other commercial vendors worth noting are companies like Flashpoint that conduct business, risk, and intelligence analysis for companies. These types of services often include active monitoring of closed forums and the dark web that can help companies identify data breaches.
Community-Based Feeds
Since your team is looking to maximize their own value and the value of your feeds, the best bet is to focus efforts on collecting high context IOCs from established resources. Suitable examples include sector appropriate Information Sharing and Analysis Centers (ISACs), and DHS’s Cyber Information Sharing & Collaboration Program (CISCP). Both offer automated indicator sharing, as well as in person analyst exchanges where teams can learn from each other’s experiences.
Open-source feeds
More is not always best… So now you have an awesome threat feed or two coming in from a commercial vendor, you’re connected to a community channel or two, and you would like to bring in open-source feeds to round out your threat feeds. AlienVault’s Open Threat Exchange, Cymon.io, and Abuse.ch are all honorable mentions in the open-source feed discussion. There are so many good options to consider that it is best to point at collections like the SANS’ Threat Feed Map and Herman Slatman’s Awesome Threat Intelligence page for full lists of amazing resources. (full links below for all references)
“PRO TIP - if a new report comes out and it’s not in your threat feeds, AlienVault’s OTX will usually have a pulse of the IOCs within a few hours of publication. This is an awesome resource to use for large reports where the IOCS aren’t provided in a consumable format. OTX lets you choose the download format and I believe they also have a STIX field available for auto-ingest to your TIP.
”
There are numerous analysts in the field that Tweet out robust threat indicators and campaign analysis regularly. Some worth noting: @ItsReallyNick @DrunkBinary @cyb3rops @QW5kcmV3 (and probably a solid dozen more you’ll find from following these folks). You can run TweetDeck with keywords of interest so your analysts can monitor for breaking events too.
“PRO TIP - TweetDeck also looks good on a SOC wall during those infamous tours, so bonus points for having something actually useful to show off on the tour.”
Screenshot of TweetDeck
Enrichment
Separate from your threat feeds, you should also invest in enrichment sources like paid VirusTotal accounts and DomainTools accounts for whois and pDNS information. These aren’t feeds but they will add more value to your threat feeds and TIP than any single feed ever will. For example, a good TIP has the ability for analysts to click a button to have the system call out to VirusTotal and pull in the SHAs, detection scores, and even detection names. The same can be done for domains and URLs to collect the whois and pDNS information. The amount of time saved for your analysts will likely cover the costs of the accounts.
In Closing
There is a lot I can say about this topic, and I know that most of us have at least 20 Chrome tabs ready to launch at any moment when research is calling. My intent with this post was to pass on my thoughts for evaluating threat feeds and resources rather than merely pass on a list of resources.
If anyone has any questions, I am more than happy to discuss and can be found on Twitter @klrgrz, on LinkedIn here through the Center For Cyber Intelligence at andy.piazz@centerforcyberintelligence.org.
Threat Intelligence Resources
And of course… “Threat Intelligence and Me: A Book for Children and Analysts” (I literally have a copy on my desk to spark conversations with non-threat intel team coworkers)