Sherlock
Sherlock is an open-source command-line tool written in Python that is used to locate a specific username across a wide range of social media platforms and online communities. It is a staple utility in the field of Open Source Intelligence (OSINT), allowing security professionals, investigators, and penetration testers to rapidly construct a digital footprint of a target by identifying accounts registered with the same handle.
The project is hosted on GitHub and is widely recognized for its speed, simplicity, and ability to query over 400 websites simultaneously, including major platforms like GitHub, Instagram, and Reddit, as well as niche forums.
How the Sherlock Tool Works
Sherlock operates by automating the process of "username enumeration." When a user inputs a specific handle (e.g., user123), the tool performs the following actions:
• URL Construction: It systematically inserts the target username into the profile URL structure of hundreds of supported websites (e.g., facebook.com/user123).
• HTTP Request Analysis: It sends an HTTP request to each constructed URL.
• Status Code Interpretation: It analyzes the server's response. A successful response (typically a 200 OK status) usually indicates the profile exists, while a "404 Not Found" error suggests the username is available or invalid on that site.
• Result Aggregation: It compiles a list of all positive matches and presents them to the user, often including direct links to the discovered profiles.
Key Features for Security Professionals
Sherlock is designed to support the workflow of cybersecurity experts. Its primary features include:
• Massive Scope: It checks for account existence on over 400 different platforms, ranging from social networks to coding repositories and creative portfolios.
• Tor Support: The tool can route traffic through the Tor network, allowing investigators to maintain anonymity and prevent their IP address from being flagged by target servers.
• Export Options: Users can save results in various formats, such as plain text (.txt) or CSV, making it easy to import data into reporting tools or spreadsheets for further analysis.
• Docker Support: It can be run inside a Docker container, ensuring a consistent environment without the need to manage complex Python dependencies on the host machine.
Common Cybersecurity Use Cases
Security teams and researchers use Sherlock for several distinct purposes:
• Red Teaming and Reconnaissance: during authorized penetration tests, ethical hackers use Sherlock to find employee accounts that may leak sensitive company information or serve as entry points for social engineering attacks.
• OSINT Investigations: Investigators use the tool to connect disparate online identities. Finding a target's username on a coding forum might reveal their email address, which can then be cross-referenced with a social media profile found by Sherlock.
• Identity Theft Monitoring: Individuals and corporations use the tool to check if their brand names or personal handles are being impersonated or squatted on by malicious actors.
Limitations and Technical Constraints
While powerful, Sherlock has specific limitations that users must understand to interpret results accurately:
• False Positives: Some websites return a "200 OK" status code even for non-existent profiles (often due to "soft 404" errors or catch-all pages). This can lead Sherlock to incorrectly report that an account exists.
• Exact Match Only: The tool generally searches for the exact character string provided. It does not automatically search for variations (e.g., searching for john_doe will not find john.doe or johndoe).
• Rate Limiting: Aggressive scanning can trigger anti-bot defenses on some platforms, causing them to block the investigator's IP address or return false negatives.
Frequently Asked Questions
Is Sherlock illegal to use?
No, Sherlock is a legal tool that accesses public information. However, using it to harass individuals or as part of unauthorized cyberattacks is illegal. It should only be used for ethical research, authorized testing, or protecting one's own data.
Does Sherlock hack into accounts?
No. Sherlock does not bypass passwords or hack into accounts. It strictly checks if a public profile page exists for a given username. It cannot see private posts, messages, or hidden data.
What operating systems run Sherlock?
Sherlock is cross-platform. It runs on Windows, macOS, and Linux. It is also pre-installed or easily available on security-focused distributions like Kali Linux and Parrot OS.
What are the prerequisites to install Sherlock?
To run Sherlock directly, you need a computer with Python 3.6 or higher installed. You will also need pip to install the required dependencies listed in the project's requirements.txt file.
Can Sherlock find deleted accounts?
No. Sherlock detects active accounts. If an account has been permanently deleted and the profile URL no longer resolves, Sherlock will report it as "Not Found."
How ThreatNG Enhances Open Source Intelligence
ThreatNG serves as a comprehensive External Attack Surface Management (EASM), Digital Risk Protection (DRP), and security ratings solution. When used in conjunction with username enumeration tools, ThreatNG elevates simple data points into actionable intelligence by correlating isolated findings with broader infrastructure risks, regulatory compliance gaps, and threat actor behaviors.
External Discovery Capabilities
ThreatNG performs purely external, unauthenticated discovery without requiring connectors or internal agents. This capability is critical when analyzing usernames or handles identified by complementary solutions. While a username search might identify that a profile exists, ThreatNG expands the scope by discovering the entire digital infrastructure associated with that identity.
• Asset Discovery: It identifies subdomains, cloud environments, and exposed assets linked to the entity.
• Cloud Hosting Identification: The solution uncovers subdomains hosted on major platforms like AWS, Microsoft Azure, and Google Cloud Platform, as well as specific SaaS applications.
• Shadow IT Detection: By operating from an outside-in perspective, it reveals "unknown unknowns," such as forgotten development servers or marketing landing pages that may be linked to a discovered username.
Deep Dive into External Assessment
ThreatNG performs extensive external assessments to assign security ratings and identify specific vulnerabilities. These assessments provide the context needed to understand the risk level of a discovered account or digital footprint.
Web Application Hijack Susceptibility
ThreatNG assigns a rating (A-F) based on the presence or absence of key security headers on subdomains. It specifically analyzes targets for missing Content-Security-Policy (CSP), HTTP Strict-Transport-Security (HSTS), and X-Frame-Options headers. For example, if a username search leads to a personal blog or developer portal, ThreatNG assesses if that site is vulnerable to clickjacking or cross-site scripting (XSS) due to these missing controls.
Subdomain Takeover Susceptibility
This assessment involves cross-referencing subdomains against a comprehensive vendor list to find "dangling DNS" records.
• Process: ThreatNG performs DNS enumeration to find CNAME records pointing to third-party services like GitHub, Heroku, or Shopify.
• Validation: It performs a specific validation check to confirm if the resource is inactive or unclaimed on the vendor's platform.
• Relevance: If an investigation reveals a username associated with an old project page, ThreatNG confirms if that page’s subdomain is abandoned and liable to be claimed by attackers for phishing campaigns.
BEC & Phishing Susceptibility
ThreatNG evaluates susceptibility to Business Email Compromise (BEC) and phishing by analyzing domain name permutations, compromised credentials, and email security records like DMARC and SPF.
• Example: If a username is associated with a corporate executive, ThreatNG identifies lookalike domains (e.g., typosquatting) that could be used to impersonate that executive in phishing attacks.
Non-Human Identity (NHI) Exposure
This rating assesses vulnerability to threats from high-privilege machine identities, such as leaked API keys or service accounts. ThreatNG continuously assesses exposure vectors like Sensitive Code Exposure to identify if a username (e.g., a bot account on GitHub) has leaked secrets that could grant adversaries access to internal systems.
Investigation Modules with Detailed Examples
ThreatNG includes specialized investigation modules that allow analysts to pivot from a simple username to a complete threat profile.
Username Exposure Module
ThreatNG features a native Username Exposure module that conducts passive reconnaissance to determine if a username is taken across a wide range of platforms. This acts as a powerful verification layer for findings from complementary solutions.
• Social & Messaging: Scans platforms like Facebook, Twitter, Tumblr, and obscure sites like 999.md.
• Development & Tech: Checks for accounts on GitHub, BitBucket, Docker Hub, and StackOverflow.
• Adult & Dating: Investigates high-risk categories including Tinder, AdultFriendFinder, and Cam sites to identify potential blackmail or reputational risks.
• Financial & Business: Looks for presence on platforms like Patreon, TradingView, and angel.co.
Social Media and Reddit Discovery
These modules manage "Narrative Risk" by transforming unmonitored chatter into intelligence.
• Function: They turn the "Conversational Attack Surface" (public discussions) into a defensive shield.
• Application: If a username is identified in a forum, ThreatNG's Reddit Discovery module can analyze associated chatter to detect early warning signs of targeted attacks or data leaks.
Domain Intelligence & Permutations
ThreatNG detects domain name permutations (typosquatting, homoglyphs, hyphenations) and checks for their availability.
• Web3 Discovery: It specifically checks for the availability of Web3 domains (like .eth and .crypto) to detect brand impersonation risks in decentralized environments.
• Keyword Targeting: It searches for targeted keywords combined with brand names, such as "login," "secure," or "payment," to identify malicious infrastructure set up to harvest credentials from users of the target brand.
Intelligence Repositories (DarCache)
ThreatNG enhances investigations by cross-referencing findings with its proprietary data repositories, branded as DarCache.
• Compromised Credentials (DarCache Rupture): Checks if the identified username or associated emails have appeared in known breaches.
• Ransomware Groups (DarCache Ransomware): Tracks over 100 ransomware gangs (e.g., LockBit, Clop, BlackCat) and their activities. If a username is linked to a sector targeted by specific groups, ThreatNG provides context on the likely threat actors.
• Dark Web (DarCache Dark Web): Monitors dark web sources for mentions of the organization or identity.
Reporting and Continuous Monitoring
Continuous Monitoring
ThreatNG provides continuous monitoring of the external attack surface, digital risk, and security ratings. Unlike a one-time username search, ThreatNG constantly watches for changes, such as a new repository being created by a target user or a new typosquatted domain being registered.
Reporting
The solution generates Executive and Technical reports that prioritize risks (High, Medium, Low) and map findings to GRC frameworks.
• Knowledgebase Integration: Reports include embedded reasoning, recommendations, and reference links to help teams understand why a specific finding (like a leaked username or exposed port) matters and how to remediate it.
Cooperation with Complementary Solutions
ThreatNG functions as a force multiplier when used alongside complementary username enumeration tools. While a standalone tool might answer "Does this user exist?", ThreatNG answers "What is the risk associated with this user?"
• From Identity to Infrastructure: When a complementary solution identifies a valid handle on a coding forum, ThreatNG can be used to scan the associated repositories for "Sensitive Code Exposure," looking for API keys or hardcoded credentials.
• From Profile to Phishing Risk: If a complementary tool finds a user on LinkedIn, ThreatNG's "LinkedIn Discovery" module identifies if that employee is susceptible to social engineering. ThreatNG then correlates this with "Domain Name Permutations" to see if attackers have already registered lookalike domains to target that specific individual.
• From Username to Dark Web Exposure: A username found on a public site can be fed into ThreatNG's "DarCache" to check if that same handle appears in dark web marketplaces or compromised credential dumps, instantly elevating the finding from "informational" to "critical risk".
• Contextual Risk Intelligence: ThreatNG’s Context Engine™ ingests data points (like a username) and fuses them with legal, financial, and operational context to provide "Legal-Grade Attribution," differentiating between a benign user and a verified threat actor.
