OSINT Reconnaissance: Tools and Techniques

Open Source Intelligence is how every serious offensive engagement begins. Before you write a single payload, before you send a single request to a target system, you spend time learning everything that's publicly accessible. What servers are exposed. What software versions are running. What employees work there and what tools they use. What subdomains exist that the security team may have forgotten about. What credentials have appeared in previous breaches.

The intelligence gathered in this phase determines the quality of everything that follows. Engagements where recon is rushed produce generic findings that miss the high-value targets. Engagements where recon is thorough produce the account-takeover chains and critical-infrastructure findings that appear in breach postmortems.

This guide covers professional-level OSINT methodology: domain intelligence, infrastructure mapping, people reconnaissance, credential exposure analysis, and the legal framework that keeps you operating on the right side of the law.

The Intelligence Cycle Applied to OSINT

OSINT isn't random Googling. It follows a structured process:

1. Direction — define precisely what you need to know. "Investigate example.com" is too vague. "Find all external infrastructure, employee email addresses, and evidence of credential exposure for example.com and its subsidiaries" is a target.

2. Collection — systematic data gathering from primary and secondary sources. Primary sources are directly accessible data (DNS records, certificate logs, WHOIS). Secondary sources are aggregated data (Shodan, Censys, breach databases).

3. Processing — normalize, deduplicate, and structure raw data. A list of 50,000 subdomains is noise. A filtered, resolved, probed list of 200 live hosts is intelligence.

4. Analysis — draw conclusions and identify attack paths. An exposed Kubernetes dashboard on a non-production subdomain is interesting. Credentials from a 2022 breach that match the company's email format are a lead worth pursuing.

5. Dissemination — document findings with timestamps. OSINT data has a shelf life — open ports get closed, credentials get rotated, subdomains change. Capture everything with the date and time it was observed.

Domain Reconnaissance

WHOIS and Registration Data

WHOIS records expose registrant details, registration history, name servers, and registrar information. Privacy protection services hide most modern registrations, but corporate domains, legacy registrations, and careless small businesses often still expose useful data.

# Basic WHOIS lookup
whois example.com
whois -h whois.arin.net 93.184.216.34  # IP block owner
 
# Historical WHOIS — see registration history before privacy services
# whoxy.com, DomainTools, SecurityTrails all maintain historical data
# These often expose the registrant email before they switched to privacy protection

What to extract from WHOIS:

• Registrant email (pivot to breach databases, other registrations, LinkedIn)
• Name servers (reveal hosting infrastructure, CDN usage, secondary DNS providers)
• Registration date (older domains often have weaker security posture)
• Registrar abuse contact (useful for legitimate disclosure)

Cross-reference registrant emails against other domain registrations using DomainTools reverse WHOIS:

registrant_email:"[email protected]" — shows all domains registered with that email

This frequently reveals related companies, subsidiary brands, development environments registered under personal emails, and testing domains with weaker security.

DNS Enumeration

DNS is an intelligence goldmine. The record types themselves reveal infrastructure choices:

# Retrieve all available record types
dig example.com ANY +noall +answer
 
# MX records — mail server infrastructure
dig example.com MX +short
# If you see: mail.example.com, this is self-hosted
# If you see: *.protection.outlook.com, they use Microsoft 365
# If you see: *.google.com, they use Google Workspace
# The email provider is now known — focus phishing appropriately
 
# SPF record — reveals all services authorized to send email
dig example.com TXT | grep spf
# "v=spf1 include:mailchimp.com include:sendgrid.net include:_spf.google.com ~all"
# Now you know: Mailchimp (email marketing), Sendgrid (transactional email), Google Workspace
 
# DMARC policy — tells you how aggressively they enforce email authentication
dig _dmarc.example.com TXT
# "v=DMARC1; p=reject; rua=mailto:[email protected]"
# p=reject: strict enforcement
# p=none: no enforcement — email spoofing may work for phishing
 
# Zone transfer attempt (rarely succeeds but always worth trying)
# A successful zone transfer reveals the entire DNS namespace at once
dig axfr @ns1.example.com example.com
dig axfr @ns2.example.com example.com
 
# DNSSEC validation
dig example.com DNSKEY
# Lack of DNSSEC means DNS cache poisoning is theoretically possible
 
# Check for wildcard DNS (affects subdomain enumeration validity)
dig nonexistentsubdomain123.example.com A
# If this resolves, wildcard DNS is configured — all discovered subdomains may be false positives

Certificate Transparency Logs

Every TLS certificate issued by a trusted CA is logged in public Certificate Transparency logs. This means every subdomain that has ever had an HTTPS certificate can be discovered without touching the target — zero network traffic to the target, permanent historical record.

# crt.sh — query certificate transparency logs
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sed 's/\*\.//g' | \
  sort -u > ct_subdomains.txt
 
# Look for:
# - subdomain patterns revealing internal naming conventions (dev-, stage-, internal-, admin-)
# - Recently issued certs on previously unknown subdomains (new attack surface)
# - Wildcard certificates (*.example.com) — confirms wildcard DNS
 
# Facebook's CT log search (different database, different results)
# https://developers.facebook.com/tools/ct/
 
# crtfinder — automates CT log discovery with additional filtering
crtfinder -d example.com -o ct_results.txt
 
# Check for SAN (Subject Alternative Name) domains in cert
openssl s_client -connect example.com:443 2>/dev/null | \
  openssl x509 -noout -text | grep -A1 "Subject Alternative Name"
# Reveals all domains on the same certificate — often exposes subsidiaries and internal hostnames

Subdomain Enumeration at Scale

# Phase 1: Passive collection (no traffic to target)
subfinder -d example.com -all -recursive -o subfinder_results.txt
# -all: uses all configured sources (requires API keys for best results)
# -recursive: also enumerate subdomains of discovered subdomains
 
amass enum --passive -d example.com -o amass_passive.txt
 
theHarvester -d example.com -b all -f harvester_results
 
# Phase 2: Combine all passive results
cat subfinder_results.txt amass_passive.txt ct_subdomains.txt | sort -u > all_passive.txt
wc -l all_passive.txt  # Typical range: 100-10,000 depending on target size
 
# Phase 3: DNS resolution — filter to actually-resolving subdomains
# puredns with a large public resolver list
puredns resolve all_passive.txt \
  -r ~/resolvers.txt \  # list of 10,000+ public resolvers
  --resolvers-trusted ~/resolvers-trusted.txt \
  -w resolved_subs.txt
 
# Phase 4: Active brute-force (only with explicit authorization)
puredns bruteforce /usr/share/seclists/Discovery/DNS/subdomains-top1million-20000.txt \
  example.com \
  -r ~/resolvers.txt \
  -w brute_results.txt
 
# Phase 5: Permutation generation
# Common patterns: api-v2, dev-api, internal-app, staging-api
# gotator generates these automatically from known subdomains
gotator -sub resolved_subs.txt \
  -perm permutations_list.txt \
  -depth 2 \
  -numbers 5 \
  -md \
  -prefixes | \
  puredns resolve -r ~/resolvers.txt > permuted_results.txt
 
# Merge all discovered subdomains
cat resolved_subs.txt brute_results.txt permuted_results.txt | sort -u > all_subdomains.txt

Live Host Probing and Fingerprinting

# httpx — HTTP probe with rich metadata
httpx -l all_subdomains.txt \
  -ports 80,443,8080,8443,8888,9000,3000,5000,4000,8000,9090,9443 \
  -status-code \
  -title \
  -tech-detect \
  -content-length \
  -follow-redirects \
  -threads 50 \
  -o live_hosts_full.txt
 
# Extract interesting targets from live host data
cat live_hosts_full.txt | grep -i "admin"    # Admin panels
cat live_hosts_full.txt | grep -i "jenkins"  # CI/CD systems
cat live_hosts_full.txt | grep -i "grafana"  # Monitoring dashboards
cat live_hosts_full.txt | grep -i "kibana"   # Log management
cat live_hosts_full.txt | grep "401\|403"    # Possibly restricted but accessible
cat live_hosts_full.txt | grep "200" | grep -v "Login\|Sign In"  # Accessible without auth

Infrastructure OSINT: Shodan, Censys, and FOFA

Shodan

Shodan continuously scans the entire IPv4 address space and indexes banners, certificates, and service metadata. It answers the question: "What internet-connected services does this organization expose, and what versions are they running?"

# Install Shodan CLI
pip install shodan
shodan init YOUR_API_KEY
 
# Search by organization name
shodan search "org:\"Example Corporation\""
 
# Search by ASN
shodan search "asn:AS12345"
 
# Search for specific software in an organization's IP space
shodan search "org:\"Example Corporation\" product:\"Apache httpd\""
shodan search "org:\"Example Corporation\" http.title:\"Dashboard\""
shodan search "org:\"Example Corporation\" Jenkins"
 
# Find hosts with known CVEs (requires enterprise Shodan plan)
shodan search "vuln:CVE-2021-44228 org:\"Example Corporation\""
shodan search "vuln:CVE-2023-44487 org:\"Example Corporation\""  # HTTP/2 Rapid Reset
 
# Domain-based search (finds all IPs serving certs for that domain)
shodan search "ssl.cert.subject.cn:example.com"
 
# Full host information
shodan host 93.184.216.34  # returns all open ports and banner data
 
# Download and parse results
shodan download --limit 1000 example_org "org:\"Example Corporation\""
shodan parse --fields ip_str,port,transport,product,version example_org.json.gz
 
# Search for exposed admin interfaces
shodan search "org:\"Example Corporation\" title:\"Admin\""
shodan search "org:\"Example Corporation\" title:\"Kubernetes Dashboard\""
shodan search "org:\"Example Corporation\" title:\"Grafana\""
shodan search "org:\"Example Corporation\" title:\"Kibana\""
shodan search "org:\"Example Corporation\" title:\"phpMyAdmin\""

High-value Shodan filters:

Censys

Censys offers complementary coverage to Shodan with stronger certificate enumeration and a more structured query language. Run both — their crawling schedules differ and they often find different hosts.

# Censys CLI
pip install censys
export CENSYS_API_ID=your_api_id
export CENSYS_API_SECRET=your_secret
 
# Search for hosts serving certificates for a domain
censys search "services.tls.certificate.parsed.subject.common_name: example.com" \
  --index-type hosts \
  --fields ip,services.port,services.service_name
 
# Find hosting infrastructure behind a CDN
# CDNs serve their own certs on edge nodes, but the origin server often has its own cert
# Search for certs issued before the CDN deployment date
censys search "services.tls.certificate.parsed.subject.common_name: example.com AND \
  services.tls.certificate.parsed.validity.start: [2020-01-01 TO 2022-01-01]"
 
# Search Censys in-browser at search.censys.io
# Query: parsed.subject.common_name: example.com AND parsed.issuer.organization: "Let's Encrypt"
# This finds Let's Encrypt certs for a domain — origin servers, APIs, internal services

Finding Origin IPs Behind CDN

A common CDN misconfiguration: the origin server's IP is discoverable through Shodan/Censys despite being "hidden" behind Cloudflare, Fastly, or AWS CloudFront.

# Method 1: Certificate history search
# The origin server may have had a cert before the CDN was deployed
# Search Censys or SecurityTrails for historical DNS and cert data
 
# Method 2: Find subdomains not behind CDN
# Many targets protect their main domain with CDN but leave API/admin subdomains exposed
# api.example.com, direct.example.com, origin.example.com, mail.example.com
 
# Method 3: SPF record reveals origin IP
dig example.com TXT | grep spf
# "v=spf1 ip4:203.0.113.1 include:mailchimp.com -all"
# 203.0.113.1 is likely the origin mail server, possibly also the web origin
 
# Method 4: Previous DNS history
# SecurityTrails, WhoisXML API, and Passive Total maintain historical DNS records
# The IP before Cloudflare was configured is often still the origin IP
# Test: curl -H "Host: example.com" http://203.0.113.1/

People OSINT

LinkedIn Intelligence

LinkedIn is the most data-rich corporate intelligence source available. For a target organization, you can enumerate:

Manual intelligence gathering from LinkedIn:
- Total employee count and growth rate (headcount indicates company size)
- Organizational hierarchy (who reports to whom)
- Technical stack from engineer profiles ("Working with Kubernetes, Terraform, Golang")
- Technologies in job postings ("2+ years experience with HashiCorp Vault preferred")
- Recent hires (new CISO = security priorities changing; new DevOps hires = cloud migration)
- Former employees (may retain VPN credentials, know internal systems)
- Contractors and consultants (often have broader access with weaker security controls)

Google dorks for LinkedIn intelligence:

site:linkedin.com/in/ "example.com" "software engineer"
site:linkedin.com/in/ "example.com" "security" "CISO OR director"
site:linkedin.com/jobs/ "example.com" "aws" "kubernetes" "terraform"

Email Format Identification

Once you have employee names from LinkedIn, you need the email format to generate valid addresses.

# Hunter.io — finds email format and lists known addresses
# CLI version:
curl "https://api.hunter.io/v2/domain-search?domain=example.com&api_key=YOUR_KEY" | \
  jq '.data.pattern'
# Returns the pattern, e.g., "first.last" or "first_initial+last"
 
# Use the pattern to generate email addresses for all employees found on LinkedIn:
# John Smith → [email protected] (first.last pattern)
# John Smith → [email protected] (first_initial+last pattern)
 
# Verify emails without sending anything:
# MX record lookup confirms domain has mail servers
# SMTP VRFY command (most servers disable this)
# haveibeenpwned.com API (checks if email appeared in a breach — confirms existence)

Username Enumeration

A single username bridges platforms. Researchers correlate usernames across social media, forums, code repositories, and breach data.

# Sherlock — checks 400+ platforms simultaneously
git clone https://github.com/sherlock-project/sherlock
python3 sherlock/sherlock.py john.doe
# Returns: GitHub, Twitter, Reddit, HackerNews, etc. where the username exists
 
# WhatsMyName — additional platform coverage
# https://whatsmyname.app
 
# Manual investigation platforms:
# GitHub: github.com/johndoe — check public repos, gists, starred repos
# HackerNews: https://hn.algolia.com/?q=johndoe (search comments and posts)
# Reddit: old.reddit.com/user/johndoe (profile, comment history)
# Keybase: keybase.io/johndoe (may have PGP key, linked accounts)

GitHub and Code Repository Intelligence

Code repositories are consistently the richest source of exposed secrets, internal architecture documentation, and misconfigured access.

# Google dorks for GitHub exposure
# site:github.com "example.com" AND (password OR secret OR key OR token)
# site:github.com "example.com" AND ".env"
# site:github.com "example.com" AND "BEGIN RSA PRIVATE KEY"
 
# GitHub search (use web interface or API)
# In GitHub search:
# org:example-company language:Python db_password
# org:example-company filename:.env
# org:example-company "amazonaws.com/s3" bucket_name
 
# GitDorker — automated GitHub dork search
python3 gitdorker.py -tf EXAMPLE_COMPANY_TOKEN \
  -q "example.com" \
  -d dorks.txt \
  -o github_results.txt
 
# truffleHog — scan GitHub repos for secrets
trufflehog github --org=example-company --concurrency=20 --json | \
  jq 'select(.verified == true)' > verified_secrets.json
 
# gitrob — discover repos and scan for sensitive files
gitrob analyze --access-token GITHUB_TOKEN example-company
 
# Specific file types to search:
# *.env, .env.local, .env.production
# docker-compose.yml (often contains credentials)
# terraform.tfvars (cloud credentials)
# kubernetes/secrets.yaml
# config/database.yml
# application.properties / application.yml
# settings.py (Django — DEBUG mode, SECRET_KEY)

Credential Exposure Analysis

Checking whether employee credentials have appeared in previous breaches is a core OSINT technique for social engineering assessments, phishing simulations, and attack path discovery.

# haveibeenpwned.com API — check if emails appeared in breaches
curl "https://haveibeenpwned.com/api/v3/breachedaccount/[email protected]" \
  -H "hibp-api-key: YOUR_API_KEY" | jq '.[].Name'
 
# Bulk email check for corporate accounts
# Get employee email list from Hunter.io / LinkedIn
# Check each against HIBP
for email in $(cat employee_emails.txt); do
  result=$(curl -s "https://haveibeenpwned.com/api/v3/breachedaccount/$email" \
    -H "hibp-api-key: YOUR_KEY" -H "User-Agent: OrgRecon/1.0")
  if [ "$result" != "[]" ]; then
    echo "$email: $result"
  fi
  sleep 1.5  # Respect HIBP rate limits
done
 
# Domain-level breach check
curl "https://haveibeenpwned.com/api/v3/breacheddomain/example.com" \
  -H "hibp-api-key: YOUR_API_KEY"
# Returns a list of all breaches where @example.com addresses appeared
 
# DeHashed — commercial breach data search (more comprehensive, requires subscription)
# Returns actual plaintext/hashed passwords from breach databases
# Useful for: demonstrating that specific employee passwords are exposed

What to do with breach data:

On an authorized red team engagement, breach data becomes:

• Password spray input (test the most common passwords from the breach against corporate SSO)
• Credential stuffing basis (test recovered plaintext credentials against VPN, Outlook Web Access)
• Social engineering context (reference the breach in a phishing pretext to build credibility)

Infrastructure Scanning and Service Fingerprinting

Network Mapping with Shodan vs. Active Nmap

Use Shodan/Censys first (passive, no target traffic) to understand the attack surface, then verify with Nmap only on systems you're authorized to actively scan.

# Nmap — comprehensive service fingerprinting (authorized targets only)
# Full TCP port scan with version detection
nmap -sS -sV -sC -p- -T4 --min-rate 1000 -oA full_scan TARGET_IP
 
# Aggressive fingerprinting on specific ports
nmap -sV -sC -p 80,443,8080,8443 --script "http-*" -oA http_scan TARGET_IP
 
# UDP services (often missed)
nmap -sU -sV -p 53,123,161,500,1900,5353 TARGET_IP
 
# SMB enumeration
nmap -p 445 --script smb-enum-shares,smb-enum-users,smb-security-mode TARGET_IP
 
# Service-specific scripts
nmap -p 5432 --script pgsql-brute TARGET_IP   # PostgreSQL
nmap -p 6379 --script redis-info TARGET_IP     # Redis
nmap -p 9200 --script http-elasticsearch-info TARGET_IP  # Elasticsearch
nmap -p 27017 --script mongodb-info TARGET_IP  # MongoDB

Exposed Internal Services

The most common high-value findings from infrastructure OSINT:

# Redis — often no authentication, full read/write access
redis-cli -h TARGET_IP ping
redis-cli -h TARGET_IP KEYS "*"   # list all keys
redis-cli -h TARGET_IP GET "session:user:1234"  # read session data
 
# Elasticsearch — frequently no authentication
curl http://TARGET_IP:9200/_cat/indices?v   # list all indices
curl http://TARGET_IP:9200/users/_search    # query user data
 
# MongoDB — no auth common in development environments
mongosh --host TARGET_IP --port 27017
> show dbs
> use production
> db.users.findOne()
 
# Kubernetes API — exposed dashboard or API
curl https://TARGET_IP:6443/api/v1/pods     # list pods
curl https://TARGET_IP:6443/api/v1/secrets  # list secrets (if unauthenticated access)
# Kubernetes dashboard exposed on :8001 or :30000+ with no auth = critical finding
 
# Jupyter Notebook — often exposed with code execution
curl http://TARGET_IP:8888/api/kernels  # if returns kernel list, execution is possible

OSINT Frameworks and Tools

Maltego

Maltego is the standard tool for visualizing and pivoting through OSINT data. It uses "transforms" to automatically convert one data type to another — domain to IP, IP to ASN, email to social profiles — and displays relationships as a graph.

The community (free) edition is limited but functional for learning. Maltego CE allows transforms against public sources including WHOIS, DNS, Shodan, and Pipl.

Useful Maltego transform sequences:

• Domain → DNS Name → IP Address → Netblock → Organization
• Email Address → Person → Social Network Profile → Related Emails
• Organization → Domain → All Subdomains → Live Web Servers → Technologies

SpiderFoot

SpiderFoot automates collection across 200+ data sources and is excellent for comprehensive, unattended reconnaissance:

# SpiderFoot command line
spiderfoot -s example.com \
  -t INTERNET_NAME \
  -m sfp_whois,sfp_dns,sfp_cert,sfp_shodan,sfp_hackertarget \
  -f json \
  -o results.json
 
# SpiderFoot web interface (more feature-rich)
python3 sf.py -l 127.0.0.1:5001
# Browse to http://127.0.0.1:5001
# New Scan → Enter domain → Select modules → Start
 
# Useful SpiderFoot modules:
# sfp_shodan: Shodan search for target IP space
# sfp_hunter: Hunter.io email enumeration
# sfp_dnscommonsrv: common service subdomain bruteforce
# sfp_cert: certificate transparency logs
# sfp_leakedcredentials: breach database checks
# sfp_linkedinmatch: LinkedIn profile discovery
# sfp_gitreposearcher: GitHub code search

OSINT Framework

OSINT Framework (osintframework.com) is a categorized tree of OSINT tools and techniques. It's not a tool itself — it's a structured directory. Useful for:

• Finding tools in specific categories you're less familiar with
• Identifying sources you haven't checked for a particular data type
• Teaching OSINT methodology to new analysts

Legal and Ethical Boundaries

OSINT operates on publicly available data, but the line between passive intelligence gathering and active reconnaissance — and between legal and illegal — is important to understand.

Generally permissible without authorization:

• Querying DNS records, WHOIS, and certificate transparency logs
• Searching indexed web content (Google, Bing, DuckDuckGo)
• Using Shodan and Censys to query their existing indexes
• Reviewing public social media profiles and posts
• Searching GitHub for publicly committed code
• Checking breach databases for your own organization's exposure

Requires explicit authorization:

• Active DNS brute-forcing against a target's name servers
• Subdomain enumeration via HTTP probing that generates target traffic
• Automated scraping of platforms that prohibit it in their ToS (LinkedIn explicitly prohibits automated scraping)
• Network scanning with Nmap or similar tools

Clearly illegal (regardless of intent):

• Accessing systems or accounts using credentials found during OSINT
• Exploiting discovered vulnerabilities without written authorization
• Bypassing authentication to view data (even if the account appears abandoned)
• Social engineering individuals without explicit scope coverage in an authorized engagement

GDPR/CCPA considerations: If your OSINT surfaces personal data (home addresses, personal phone numbers, medical information), handle it appropriately. Don't aggregate, store, or republish personal data beyond what the engagement requires. Document what was found, inform the client, and don't dig further into personal information that's outside your engagement scope.

Building a Repeatable Recon Workflow

The difference between researchers who find consistent results and those who don't is a repeatable, documented methodology. Every step should be reproducible.

#!/bin/bash
# Professional OSINT workflow for authorized engagements
 
TARGET_DOMAIN=$1
CLIENT=$2
DATE=$(date +%Y%m%d)
OUTPUT="recon/${CLIENT}-${DATE}"
 
mkdir -p "$OUTPUT"/{dns,subs,infra,people,web}
 
echo "=== PHASE 1: DNS INTELLIGENCE ==="
whois "$TARGET_DOMAIN" > "$OUTPUT/dns/whois.txt"
dig "$TARGET_DOMAIN" ANY +noall +answer > "$OUTPUT/dns/dns_records.txt"
dig "$TARGET_DOMAIN" MX +short >> "$OUTPUT/dns/dns_records.txt"
dig "_dmarc.$TARGET_DOMAIN" TXT +short > "$OUTPUT/dns/dmarc.txt"
 
echo "=== PHASE 2: CERTIFICATE TRANSPARENCY ==="
curl -s "https://crt.sh/?q=%.${TARGET_DOMAIN}&output=json" | \
  jq -r '.[].name_value' | sed 's/\*\.//g' | sort -u > "$OUTPUT/subs/ct_subs.txt"
 
echo "=== PHASE 3: PASSIVE SUBDOMAIN ENUMERATION ==="
subfinder -d "$TARGET_DOMAIN" -all -silent -o "$OUTPUT/subs/subfinder.txt"
amass enum --passive -d "$TARGET_DOMAIN" -o "$OUTPUT/subs/amass.txt"
 
cat "$OUTPUT/subs/"*.txt | sort -u > "$OUTPUT/subs/all_passive.txt"
echo "Passive subdomains found: $(wc -l < $OUTPUT/subs/all_passive.txt)"
 
echo "=== PHASE 4: INFRASTRUCTURE MAPPING ==="
# Shodan search (requires API key)
shodan search "org:\"$CLIENT\"" --fields ip_str,port,transport,product \
  > "$OUTPUT/infra/shodan_results.txt"
 
echo "=== PHASE 5: LIVE HOST PROBING ==="
puredns resolve "$OUTPUT/subs/all_passive.txt" -r ~/resolvers.txt \
  -w "$OUTPUT/subs/resolved.txt" --quiet
 
httpx -l "$OUTPUT/subs/resolved.txt" \
  -ports 80,443,8080,8443,3000,5000 \
  -status-code -title -tech-detect \
  -o "$OUTPUT/web/live_hosts.txt" --silent
 
echo "=== PHASE 6: EMAIL ENUMERATION ==="
theHarvester -d "$TARGET_DOMAIN" -b all -f "$OUTPUT/people/harvester"
 
echo "=== COMPLETE ==="
echo "Results in: $OUTPUT/"
echo "Live hosts: $(wc -l < $OUTPUT/web/live_hosts.txt)"
echo "Subdomains resolved: $(wc -l < $OUTPUT/subs/resolved.txt)"

Documentation Standard

Document every finding with:

• Timestamp — OSINT data ages; what was true today may not be tomorrow
• Source — which tool or platform produced this finding
• Raw data — a screenshot or copy of the original data, not a summary
• Interpretation — what this finding implies and what leads it suggests

The goal of OSINT is not to collect data — it's to develop actionable intelligence. Every piece of data you gather should either answer a specific question or generate a new, more specific question to pursue. Work the intelligence cycle, not the tool list.

Detecting reconnaissance against your own organization

The methodology above works both directions. If you defend an organization, understanding how recon is performed tells you where to watch for it and what an early attack looks like before any exploitation begins. Most passive collection is invisible to you by design, since it queries third-party indexes rather than your own systems, and that shapes the detection strategy.

Certificate transparency is the clearest example. When an attacker enumerates your subdomains through crt.sh, no request touches your infrastructure, so there is nothing in your logs to find. The defensive answer is to monitor the same logs yourself. Subscribe to certificate transparency monitoring for your domains so that every newly issued certificate generates an alert. A certificate for a hostname you did not expect is both an attack-surface discovery for you and a signal that the same subdomain is now visible to anyone enumerating you. This turns a source of attacker intelligence into a defender early-warning system.

Infrastructure indexes like Shodan and Censys are similar. You cannot see an attacker querying them, and you can query them for your own address space on a schedule and treat any newly exposed service as an incident. A database that appears in the index without authentication, an admin panel that becomes reachable, or an end-of-life operating system banner is something you want to find before anyone else does. Running the passive tools against yourself, continuously, is one of the highest-value defensive habits available, because it sees your estate exactly as an external attacker sees it.

Active recon does leave traces, and this is where log analysis earns its place. DNS brute-forcing generates a flood of lookups for hostnames that do not exist, so a spike in queries returning the non-existent-domain response against your authoritative name servers is a strong signal of subdomain brute-forcing. HTTP probing across a wide range of hostnames and ports shows up as many short-lived connections from a small set of source addresses. Password-reset and login endpoints hit with a list of employee addresses harvested from LinkedIn look like a slow, distributed enumeration. None of these is subtle once you know the shape, and correlating them across DNS, web, and authentication logs surfaces the transition from passive study to active engagement.

Passive versus active, and why the boundary matters

The single most important distinction in this discipline is between passive collection and active engagement, because it governs both legality and stealth. Passive collection reads data that already exists in third-party systems: certificate logs, DNS records, WHOIS, search indexes, Shodan, breach databases, and public social profiles. None of it sends traffic to the target, so none of it is visible to the target and, for your own organization or with the appropriate basis, it is generally permissible to gather.

Active engagement sends traffic to systems the target owns: resolving candidate subdomains against their name servers, probing hosts over HTTP, scanning ports with Nmap. This generates entries in the target's logs, and it crosses from studying public data into interacting with someone's infrastructure, which is where authorization becomes mandatory. The methodology is deliberately ordered to exhaust passive sources first, both because they are stealthy and because they often reveal enough that active steps can be narrow and targeted rather than broad and noisy.

The practical consequence is that a well-run engagement front-loads everything that leaves no trace. By the time any packet reaches the target, you already know the likely subdomains, the hosting infrastructure, the email provider, the employee roster, and the credential exposure. Active steps then confirm specifics rather than discover them, which keeps the footprint small and the results precise. Rushing to active scanning skips the intelligence that would have made the scanning focused, which is the difference the introduction describes between generic findings and the high-value chains that appear in breach postmortems.

Common mistakes in OSINT

Several habits separate analysts who produce intelligence from those who produce noise. Each one is easy to correct once named.

Collecting without direction. Gathering fifty thousand subdomains feels productive and produces a pile of data nobody can act on. Every collection step should trace back to a specific question from the direction phase. If a piece of data does not answer a question or generate a sharper one, it is noise, and more of it is worse, not better.

Skipping processing. Raw output from automated tools is full of duplicates, dead hosts, and false positives, especially where wildcard DNS makes every candidate subdomain resolve. Handing an unfiltered list to the analysis phase wastes effort on hosts that do not exist. The filtered, resolved, probed shortlist is the intelligence, and the raw list is only its input.

Trusting stale data. OSINT ages quickly. An open port from a scan taken weeks ago may be closed, a credential from an old breach may be rotated, and a subdomain may have moved. Findings acted on without a fresh check lead to conclusions that were true once and are not now. This is why every finding carries a timestamp and why verification precedes action.

Crossing the legal line by reflex. The most damaging mistake is treating a discovered credential or an exposed panel as an invitation. Finding an abandoned account or an unauthenticated database during passive OSINT does not authorize logging into it. Access requires authorization regardless of how easy the target makes it, and the moment an analyst uses a found credential the activity stops being intelligence gathering and becomes intrusion.

Ignoring the shape of the data. A list of hosts without interpretation is inventory, not intelligence. The value comes from noticing that a naming convention reveals internal environments, that an SPF record enumerates every service authorized to send mail, or that a breach password matches the corporate format. Analysis is the step where data becomes a lead, and it is the step most often skipped in favor of collecting more.

How the discipline evolved

OSINT reconnaissance grew alongside the infrastructure it studies. Early recon leaned on WHOIS and DNS, because those were the public records that existed, and much of the craft was manual cross-referencing of registrant details and name servers. As the web grew, search-engine dorking became a core skill, since indexed content exposed configuration files, directory listings, and documents that were never meant to be public.

Two developments changed the scale of what a single analyst could see. Certificate transparency, introduced to make the certificate authority ecosystem auditable, had a side effect that reshaped recon: because every issued certificate is logged publicly, the complete historical subdomain inventory of most organizations became queryable with no traffic to the target. Around the same time, internet-wide scanning services began continuously indexing the entire address space, so questions that once required scanning a target yourself could be answered from a pre-built index instead. Together these moved a large share of reconnaissance from active and detectable to passive and invisible.

The tooling matured to match. Where an analyst once ran individual lookups, frameworks now orchestrate collection across hundreds of sources, correlate usernames and emails across platforms, and visualize relationships as graphs to reveal pivots a human would miss. The constant through all of it is the intelligence cycle. Tools change constantly, and the discipline of direction, collection, processing, analysis, and dissemination is what turns any generation of tooling into intelligence rather than a heap of output.

Turning findings into attack paths

Collection and analysis matter because they combine into paths, and the pivots between data types are where recon becomes dangerous. A single WHOIS registrant email is a starting point rather than an endpoint. Run it through reverse WHOIS and it reveals other domains the same person registered, which surface subsidiary brands and forgotten development environments with weaker security. Run it through breach databases and it may return a password the person reused. Run it through social platforms and it links to a LinkedIn profile that names the technologies they work with. One field becomes a map.

Subdomains pivot the same way. A hostname discovered in certificate transparency resolves to an IP, the IP belongs to a netblock, the netblock reveals the hosting provider and often the origin server hidden behind a content delivery network. A naming convention seen across several subdomains, such as a consistent prefix for internal or staging environments, predicts the names of hosts you have not found yet, which feeds the permutation step. Each finding narrows the next query, which is why a structured process compounds while random searching plateaus.

Credential exposure is the pivot that most often ends in access on an authorized engagement. An employee roster from LinkedIn, combined with an email pattern from a format-discovery service, produces a list of valid addresses. Checking those against breach data returns the subset with exposed passwords. On an engagement whose rules of engagement permit it, those become password-spray and credential-stuffing input against corporate single sign-on, mail access, or the VPN. The chain from a public profile to a working login is short, and every link in it was assembled from data the organization exposed without meaning to. Seeing that chain in advance is exactly what recon exists to do, and it is why thorough collection at the start determines the quality of everything that follows.

Related guides

• OWASP Top 10 Explained
• Command Injection Explained
• API Security Guide
• Secure Coding Practices