The Continuous Recon Mindset – Implementing Automated Proactive EASM [Part 1]
By Itamar Weller
The road towards automated attack surface discovery and management is paved with good intentions.
Nowadays, an enterprise’s digital footprint tends to be enormous and complex. It is comprised of web applications, APIs, IoT, and the never-ending “shadow-IT,” and it’s our goal to secure it all.
As a wise person once said, “evil only needs to win once.”
Can we level up by adding a hacker mindset, a learn & automate feedback loop, and a generous dose of community cooperation to our quiver?
While understanding the attack surface of an enterprise is only half of the solution, it is still a crucial and exhaustive first step for us to tackle. So, how do we start?
The need arose after trying several commercial EASM solutions that resulted in several issues:
- Some tools require you to manually “feed” them your assets.
- Smarter and more cutting-edge tools do cover the discovery part by themselves but will never be able to provide you with the full picture.
- The tools often focus on low-severity, informative findings (e.g., missing security headers), while producing many false positives (e.g., CSRF in search forms).
- We saved the biggest gap for the end – the tools don’t update as fast as the bad guys can.
As InfoSec nerds, we enjoy hacking in our free time (legally, of course). Combining this mindset with our internal knowledge and innovative tools, we’ve created what we like to call “Continuous Recon.”
The Continuous Recon mindset involves having a set of open-source and internally developed tools that continuously map and attack assets and remediate vulnerabilities, with minimal need for a human touch.
For such a move to succeed, bi-directional community cooperation is essential. Indeed, during our journey to the “Promised Land,” (i.e. achieving Continuous Recon), we have used, and more importantly, contributed to, numerous open-source tools and projects commonly used by the hacker community, such as Nuclei and dnsprobe (both by @projectdiscovery).
The last major pillar that enables the Continuous Recon mindset is constant improvement. This can be achieved by learning from and implementing the outcomes of every incoming bug report. These reports can come from our internal and external bug bounty programs, or from our penetration testing activities. The lessons learned from them are then implemented in our tools and contributed back to the community.
In this blog post, we aim to share two tools that were developed by our Product Security team and subsequently contributed to the open-source community.
D-Collector
TLDR; D-Collector is a tool that fetches DNS records from various DNS management and cloud providers, normalizing all records into a unified format.
Nowadays, most enterprises use multiple different cloud providers and DNS registrars. This creates a fertile ground for shadow IT to grow and makes it extremely hard for information security teams to map, manage, and scan their attack surface.
The above led us to create D-Collector. Now, we can continuously scan our infrastructure for security vulnerabilities and drastically reduce the risk of shadow IT.
Who is this tool for?
This tool was created to enable IT and information security practitioners to easily automate their domains and attack surface management activities.
How does it work?
- You provide D-Collector with your cloud providers’ API tokens/keys.
- D-Collector fetches all DNS records from the given cloud providers.
- D-Collector creates a unified structured “domains.json” file with all the DNS records.
- You harness the magic for your enterprise’s needs.
The tool can be found on Playtika’s Github repository: https://github.com/Playtika/D-Collector.
DDFR – Dangling Domains Finder
DDFR is a lightweight Python utility that detects DNS records that are suspected to be dangling.
This is particularly critical, as subdomain takeovers have become a popular technique used by attackers and bug bounty hunters.
One particular use case of this attack that is pretty hard to catch (for both blue and red teams) is DNS records that point to unclaimed IP addresses.
Who is this tool for?
As a red-teamer, these kinds of takeovers are hard to find, as many resources are needed to perform a successful attack (bruteforcing VMs IP addresses attached by the cloud providers) and reverse DNS lookup.
As a blue-teamer, companies currently have thousands of DNS records, making it extremely hard for you to find them manually.
By being able to pull your company’s DNS records from your DNS management system – be it via AWS Route53 or any other system – you can use this tool to proactively look for dangling records and mitigate such takeovers.
How does it work?
- You provide DDFR with a list of your DNS records (see Usage section).
- You provide DDFR with a list of all your registered cloud IPs in one of the following ways:
- Allow DDFR to automatically collect all of your registered IPs from Palo Alto’s Prisma Cloud product.
- Manually provide a file with your registered cloud IPs.
- DDFR compares the lists to see if there are domains pointing to IPs that you do not own and, therefore, are suspected to be dangling.
NOTE: DDFR also provides you with a mechanism to reduce false positives, by providing it a list of company-known SSL certificates’ common names. - DDFR generates a report of the suspected DNS records.
The tool can be found on Playtika’s Github repository https://github.com/Playtika/ddfr.
