Considerations for Leveraging Cyber Threat Feeds Effectively

There are a few different approaches that I have seen organizations take when it comes to consuming threat intelligence. Teams that have been in the threat intel space for a long time started with informal sharing distros (We don’t talk about fight club!) with unorganized TXT files serving as their community standard. These teams probably evolved their sharing efforts into using CSVs, maybe even with some basic structure, before moving on to helping build out projects like STIX, openIOC, etc.. These analysts then started looking at open-source and commercial feeds with automation and integrations — or even starting their own Threat Intelligence Platform (TIP) companies.

On the other side of the spectrum, you have organizations building threat intelligence capabilities from scratch and attempting to go from maturity level 0 to 5 in a single project. These organizations usually get a Threat Intelligence Platform (TIP) with a flashy demo- maybe even a #pewpew map!- And they turn on every threat feed available. In a few short months, they blame their TIP for feeding them too many false positives and their threat feed providers get tagged with “too little; too late.” Many of the problems these organizations face are that they have not gone through the crawl and walk phase of capability building and attempted to purchase their way straight to the run phase of operations.

The good news is that many of us have experienced the hard-fought battles of building an intelligence capability and we are often more than willing to share our lessons learned. Usually, this exchange is over a beer at a post-event happy hour, but I’ve decided to share my experiences a little bit broader than my personal circle - just offer to grab a round if you find this helpful in your organization. This post will cover my thoughts on managing threat feeds within an organization. I plan to follow up later with my thoughts on Threat Intelligence Platforms (TIPs) and cyber threat information sharing standards.

Without further ado, here are my recommendations for implementing a basic cyber threat intelligence capability.

Threat Feed Evaluations

As with all projects, organizations must have a serious conversation about threat feed requirements, processes, and standards before turning on commercial and open source feeds. Once your team has decided to ingest feeds automatically, it is recommended that organizations only turn on 3–5 feeds at a time and run them for at least 60–90 days before turning on additional feeds. During this period, analysts and managers should regularly be talking about the number of false positives in the evaluated feeds and the quality of the context in them.

When do you turn on feeds? If you have a security appliance that receives indicator and signature feeds directly into it, that is a separate discussion that must occur between threat intelligence leadership, detection and monitoring leaders, and security engineering. I generally recommend turning these feeds on and working with the vendor to tune out issues. However, I have heard some mature organizations argue that they only use their own custom signatures. There are pros and cons in this approach that must be discussed internally at each organization.

The absolute worst thing you can do to your organization is trying to play the "catch 'em all" game — this isn’t Pokemon, and your analysts may quit when they end up with an 85% false positive rate with thousands of unreviewed alerts pending in their queue. (That 85% FP rate isn’t made up either; it is rounded from real analysis on a previous project.) Remember, one of the goals of threat intelligence is to prevent alert fatigue- not cause it.

One method to evaluate how your feeds are impacting your organization is to assess the actions that they lead to for your team in a short 60–90-day project. For example, have an analyst review all of the hits (i.e., IOC in feed matches traffic in the environment) and tag them as False Positive, True Positive Escalated, True Positive Mitigated. The definition of Escalated here is that the existing security stack did not block the event, so action was required to scope and mitigate the activity. The meaning of Mitigated here is that the current security controls detected and mitigated the event. These metrics will help you better understand how much good work vs. busy work your feeds are causing within your environment.

A second method is to give your analysts a rating questionnaire and have them run an evaluation on the feeds on a 1–5 scale with criteria like the quick list below. The output is that your team ranks each individual feed, and then someone runs the averages to see how the feeds compare to each other.

• Context

  • Is there enough context around the IOCs to understand how they are used in an event/attack?

• Timeliness

  • Is there a big gap between Date Sighted and the day you receive them?

• Ease of Use

  • Does the feed provide all malware hashes and your organization has to run these manually?

  • Does the feed provide mostly IPs and resulting in your analysts have to run additional analysis against them before safely deploying?

  • Is the feed auto-fed into your TIP or does your team have to download them from a site manually?

Threat Feed Considerations

I recommend staying away from threat feed vendors where the primary business line is IOC feeds. Traditionally, these companies focus too much on quantity over quality, and you’ll end up with a giant dump of IPs with minimal context. The other challenge here is that they often deduplicate and anonymize their feeds before pushing them to you- you’ll see this same challenge with TIP vendor’s curated feeds. This is a huge hindrance to your analysts since they will not be able to tell if IOCs are being widely observed across multiple sectors or if other elements of context are being lost in the deduplicate process. Worse still, if you cannot do fundamental supply chain analysis of your IOCs, you may duplicate feeds on your end that your vendor is also pushing you. This costs you money in paying for feeds needlessly, and it creates an echo effect for those IOCs. In a world where algorithms are now telling us what’s essential, this could mislead your system to calculate higher confidence scores.

As an analyst, I have to say the most egregious crime in anonymized threat feeds is not being able to see the real source of the intel. How can analysts effectively assess their sources for confidence and accuracy if the data is coming from mixed sources that are also anonymized? Trust grows in the light.

Sourced from: http://www.littlebobbycomic.com/projects/week-104/

When it comes to cyber threat indicators, context is king. If a vendor has a feed of indicators and it doesn’t include context for each indicator in a Description field, you are not buying indicators- you are buying Observables (more on that in a later article on cyber threat information sharing standards).

Each organization will set their own standards for their threat intel providers. Here are a few key aspects to consider:

1. Is the feed provider the source of the intelligence or are they repacking IOCs from other sources?

   1. The original source is always preferred since the company will likely stand behind their analysis or be available for an RFI around their info

   2. Bundled feeds can make it harder to evaluate the quality of IOCs because of deduplication processes and anonymization (more on this later)

2. Do the indicators include enough context to be quickly actionable?

   1. A Description like “This email delivered this URL which led to…”

   2. What type of malware is this malicious hash?

   3. What were the domains that were observed with these malicious IPs?

3. Is the feed causing unacceptable false-positive rates for your organization?

   1. Each org has to determine “acceptable” rates for their teams

   2. Food for thought: one team I worked with in private sector once told me that they would turn off my organization’s threat feed after three strikes of wasted time- even though our feed was 100% free, it still could cost them money in wasted resources

4. Is the feed providing too many IOCs that are mitigated by other security appliances?

   1. Example: the majority of IOC matched in your logs are all in blocked events because IPS detected it already

   2. Note: this is okay to have the Intel still enter the TIP, but you may consider tuning them out from entering your SIEM.

5. Does your feed include properly marked benign indicators?

   1. Example: malware calls out to Google DNS to see if it can reach the internet

   2. While 8.8.8.8 is a horrible IOC for detection, this benign IOC may be a critical piece of intelligence to have in the TIP

   3. Remember- IOCs!=malicious code (more on that in a later post)

Commercial, Community, and Open-source

What about paid feeds vs. open-source? I believe that most organizations are best suited with a blended approach for their threat feeds. Commercial feeds can benefit an organization because they are often backed by intelligence teams that are available to discuss their analysis with your team when support is needed. Generally, these feeds are more mature and will follow established intelligence standards in their analysis. Community feeds, such as Information Sharing & Analysis Centers (ISACs) can provide your organization with intelligence and IOCs directly relevant to your business sectors. They also offer a venue to discuss best practices for the security of sector-specific issues. Open-source can be an invaluable resource for timely intelligence on emerging threats. Major campaigns often first appear publicly on Twitter long before a blog or threat report is written. Organizations should invest the time to evaluate which open-source feeds, blogs, and personalities they should follow for the most up-to-date information available.

Commercial Vendors 

I won’t pump one vendor over the other, but I will provide some basic advice here. As budgets allow, I recommend purchasing at least one premium intelligence service from vendors like CrowdStrike or FireEye- both if the budget allows. In my opinion, the quality of the context from these companies is unmatched, and these companies do a fantastic job enforcing traditional intelligence principles in their analysis and production cycles.

Other commercial vendors worth noting are companies like Flashpoint that conduct business, risk, and intelligence analysis for companies. These types of services often include active monitoring of closed forums and the dark web that can help companies identify data breaches.

Community-Based Feeds 

Since your team is looking to maximize their own value and the value of your feeds, the best bet is to focus efforts on collecting high context IOCs from established resources. Suitable examples include sector appropriate Information Sharing and Analysis Centers (ISACs), and DHS’s Cyber Information Sharing & Collaboration Program (CISCP). Both offer automated indicator sharing, as well as in person analyst exchanges where teams can learn from each other’s experiences.

Open-source feeds 

More is not always best… So now you have an awesome threat feed or two coming in from a commercial vendor, you’re connected to a community channel or two, and you would like to bring in open-source feeds to round out your threat feeds. AlienVault’s Open Threat Exchange, Cymon.io, and Abuse.ch are all honorable mentions in the open-source feed discussion. There are so many good options to consider that it is best to point at collections like the SANS’ Threat Feed Map and Herman Slatman’s Awesome Threat Intelligence page for full lists of amazing resources. (full links below for all references)

“PRO TIP - if a new report comes out and it’s not in your threat feeds, AlienVault’s OTX will usually have a pulse of the IOCs within a few hours of publication. This is an awesome resource to use for large reports where the IOCS aren’t provided in a consumable format. OTX lets you choose the download format and I believe they also have a STIX field available for auto-ingest to your TIP.
”

— Andy Piazza

There are numerous analysts in the field that Tweet out robust threat indicators and campaign analysis regularly. Some worth noting: @ItsReallyNick @DrunkBinary @cyb3rops @QW5kcmV3 (and probably a solid dozen more you’ll find from following these folks). You can run TweetDeck with keywords of interest so your analysts can monitor for breaking events too.

“PRO TIP - TweetDeck also looks good on a SOC wall during those infamous tours, so bonus points for having something actually useful to show off on the tour.”

— Andy Piazza

Screenshot of TweetDeck

Enrichment  

Separate from your threat feeds, you should also invest in enrichment sources like paid VirusTotal accounts and DomainTools accounts for whois and pDNS information. These aren’t feeds but they will add more value to your threat feeds and TIP than any single feed ever will. For example, a good TIP has the ability for analysts to click a button to have the system call out to VirusTotal and pull in the SHAs, detection scores, and even detection names. The same can be done for domains and URLs to collect the whois and pDNS information. The amount of time saved for your analysts will likely cover the costs of the accounts.

In Closing

There is a lot I can say about this topic, and I know that most of us have at least 20 Chrome tabs ready to launch at any moment when research is calling. My intent with this post was to pass on my thoughts for evaluating threat feeds and resources rather than merely pass on a list of resources.

If anyone has any questions, I am more than happy to discuss and can be found on Twitter @klrgrz, on LinkedIn here through the Center For Cyber Intelligence at andy.piazz@centerforcyberintelligence.org.

Threat Intelligence Resources

• “Awesome Threat Intelligence”

• ThreatCrowd - Open Source Threat Intelligence

• ThreatMiner.org

• CISCO Talos Intelligence Group

• RiskIQ Community | Passive Total

• SANS Internet Storm Center Threat Map

• AlienVault Open Threat Exchange

• Department of Homeland Security CISCP

And of course… “Threat Intelligence and Me: A Book for Children and Analysts” (I literally have a copy on my desk to spark conversations with non-threat intel team coworkers) 

The Need for Industry Accepted Cyber Intelligence Standards

Today if you spend any time working in the Cyber Security industry you will inevitably hear the term “threat intelligence” tossed around like candy. Most people have a general understanding of what threat intelligence means but in case you forgot, or are not aware of the definition — Generally speaking (depending on who you are talking to), the accepted industry definition of Threat Intelligence is:

“Threat intelligence is evidence-based knowledge, including context, mechanisms, indicators, implications and actionable advice, about an existing or emerging menace or hazard to assets that can be used to inform decisions regarding the subject’s response to that menace or hazard.”

— Gartner, 2013

Then Gartner charges $195 to read the rest of their paper…

Unfortunately, good old fashioned commercialism has completely saturated the term “threat intelligence” and up until pretty recently, has viciously marketed it as something along the lines of selling an indicator of compromise (IOC) feed.

“We have the best indicators; Our indicators are the best; No one has better indicators than we do; Buy our threat intelligence feed; Use our feed to block “badness” at your firewall; Use our IOCs, and you will be safe from all of the cyber threats.”

— Every threat intelligence vendor ever

Let’s be real, if we stick with Gartner’s definition above, companies that follow this model are really selling “threat information,” not threat intelligence. Further, many of these same companies work to turn a profit by teaching organizations how to “do” threat intelligence and yet, for those organizations who can afford to pay these firms to come in and help, they still struggle to meet their goals. Thankfully, many of us know today that the answer to all of our security woes is not a multi-thousand dollar per year indicator sharing feed.

Acquiring the indicator sharing feed is where many organizations tend to stop when it comes to implementing threat intelligence. This is mainly because businesses are not sure how to implement or apply a threat intelligence program that creates value for their organization resulting in existing threat intelligence analysts become “indicator sharing” analysts instead of highly skilled cyber intelligence analysts. Organizations begin focusing on how many IOCs the “threat intelligence team” input into a glorified indicator sharing platform (also known as a threat intelligence platform), or how they can assist the security operations center in building context around individual events and maybe even an incident here and there. Granted, while this stuff is precious to any organization, this is not all that threat intelligence is. A moment of clarity — Yes, we need our indicators of compromise (IOCs) sharing feeds — Today we live and work in such a fast-paced environment that we need the newest IOCs to put on our endpoints as soon as they are discovered; IOCs are typically useless after a short period. And, many organizations are willing to pay top dollar to ensure they have the most timely “threat information” as they can get their hands on. There is nothing wrong with this — keep doing it.

However, Threat Intelligence is more than ensuring endpoints have the most up to date IOCs — thankfully today, most people can agree with this. Despite the agreement, I have seen multiple organizations in the private sector as well as government, in both large and small teams, still struggle with how to break out of the tactical mindset and start working on developing operational and strategic level intelligence, as well as tending to the tactical.

The Center for Cyber Intelligence (CCI) was established to provide a solution to this issue. CCI serves to develop and improve cyber intelligence standards to mobilize and encourage the cybersecurity community to apply a threat intelligence capability. Further, we exist to motivate the cyber intelligence community to share knowledge, experience, and expertise to identify and validate cyber intelligence best practices.

Probably most important, CCI is unique in that it was created by and for the cybersecurity community and operates as a volunteer-based organization. We do not seek any profit in this venture, we only seek to establish threat intelligence standards.

---

The Cyber Intelligence Framework

By The Center for Cyber Intelligence

I’ve been working in the intelligence field for about 12 years now, and if I’ve learned anything, it’s that intelligence as a practice works when we follow a good process. Cyber Intelligence, in practice, is no different than military intelligence. The only difference is the landscape in which you fight. The United States Department of Defense is really good at “doing” intelligence for one fundamental reason — They have standards, policies, and methodologies that are used across all branches of the Department. Further, when working outside of the organization, the U.S. Government recognizes the same processes and methods to conduct intelligence operations. Regardless of the fight, air, sea, or land, the same techniques are used to ensure the quick and timely delivery of critical information to the right people at the right time.

So why do we struggle to do this in the cyber intelligence world?

A quick Google search turns up countless white papers, articles (this one soon to be amongst the search results), and blog posts on how organizations can implement a threat intelligence capability. In large part, none of these recommendations are wrong, or inaccurate. So why haven’t we yet done it? Why does the average organization struggle to operationalize a threat intelligence capability past the tactical intelligence level? I’ve read numerous papers and books on this topic, and one would think that with this wealth of information and knowledge the cyber intelligence industry would be more mature, but I continue to find this isn’t the case.

In the cyber intelligence world I’ve found there are, in general, two types of people:

• People with a sharp intelligence background but a weaker IT/cybersecurity background

• People with a strong IT/security background but a weaker intelligence background

To be clear, neither of these types of people is a poor fit for a Cyber Intelligence role. However, because of these inherent shortcomings, we are left with threat intelligence programs that are often very tactically focused. Intelligence analysts straight out of the military or government are commonly used to very tactical operations, the “boots on ground” fight if you will. And, your IT personnel are usually really good at security but aren’t quite sure what you mean when you say that his/her analysis suffers from inherent bias let alone what “structured analytical techniques” mean.

To alleviate these and other industry shortcomings, CCI is putting together the Cyber Intelligence framework to serve as a guide for both private sector and government organizations both large and small, enabling them to establish a threat intelligence program that provides value at the tactical, operational, and strategic levels. While there are numerous frameworks in place today that assist with varying degrees of cybersecurity, there is no open source framework defining how to “do” threat intelligence. CCI’s Cyber Intelligence Framework fills this gap by pulling together industry expertise and existing cybersecurity frameworks to establish a community-accepted standard.

CCI’s Threat Intelligence Framework is a collection of industry knowledge that is intended to provide organizations of any size with a guide to establishing a functioning Cyber Threat Intelligence capability within a traditional “Cyber Security Stack.” The Threat Intelligence Framework provides organizations with a baseline defining how to structure and build a Threat Intelligence program that is capable of informing decision-makers and critical stakeholders of current and potential future cyber threats to their organization at tactical, operational, and strategic levels.

Today, the CCI Cyber Intelligence Framework is intended to be a starting point for the cyber intelligence community to come together and establish a commonly understood model of how we should be doing business. The Cyber Intelligence Framework will encompass everything from business requirements down to what kinds of reports should be produced by a threat intelligence program ensuring the cyber intelligence community has a “one-stop shop” detailing how to build and implement a capable Cyber Threat Intelligence function.

CCI is currently requesting input and feedback on Version 1.0 of the Cyber Intelligence Framework from security and intelligence experts from across the industry. If you are interested in contributing or learning more, please feel free to email us at info@centerforcyberintelligence.org. You can check out our work on the Cyber Intelligence Framework here.

---

Cyber Intelligence Framework by Center for Cyber Intelligence is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License