This guide provides detailed instructions for installing, configuring, and using five essential OSINT (Open Source Intelligence) tools for cybersecurity researchers, digital investigators, and security professionals.
OSINT Framework is a web-based collection of OSINT resources, organized by category. It provides links to hundreds of tools for various investigation needs.
OSINT Framework doesn't require installation as it's a web-based resource.
- Visit https://osintframework.com/
- Navigate the interactive mind map by clicking on categories and subcategories
- Each node in the framework represents either a subcategory or a direct link to a tool
- Click on tool links (indicated by a globe icon) to access the respective resource
- Use the search function at the top to find specific tools or categories
- Bookmark the OSINT Framework for quick access during investigations
- Explore all categories to understand the breadth of available resources
- The framework is regularly updated, so check back for new tools
- Consider contributing to the framework via its GitHub repository if you discover useful tools not listed
Maltego is a data mining and visualization tool that maps relationships between information entities such as people, companies, domains, and more.
- Visit https://www.maltego.com/downloads/
- Create a Maltego account if you don't have one
- Download the appropriate version for your operating system:
- Windows (64-bit)
- macOS
- Linux
- Run the installer and follow the prompts
- Launch Maltego after installation
- Upon first launch, you'll be guided through the setup wizard
- Log in with your Maltego account credentials
- Select your edition (Community, Classic, XL, etc.)
- Configure the transforms (data sources) you want to use:
- Community transforms (free)
- Transform Hub options (some require payment)
- Custom transforms (if available)
- Configure API keys for transforms that require them under "Manage" → "Manage Transforms"
- Create a new graph through File → New
- Start by adding an entity from the palette on the left (e.g., Person, Domain, IP Address)
- Right-click on an entity and select "Run Transform" to discover related information
- Common transforms include:
- For domains: DNS information, WHOIS data, linked IP addresses
- For people: Social media profiles, email addresses, company associations
- As transforms run, new entities will appear, creating a visualization of connections
- Organize your graph using the layout tools in the top toolbar
- Save your investigation as a Maltego graph file (.mtgx)
- Use the "Machine" feature to run pre-configured sequences of transforms
- Create custom entity types for specialized investigations
- Export results in various formats (CSV, JSON, etc.)
- Use bookmarks to organize complex investigations
- Create custom palettes for frequently used entities
- Start with the free Community transforms before investing in commercial options
- Take advantage of the "Detail View" to see all properties of selected entities
- Use the filtering options to manage large graphs
- Group related entities to keep your visualizations clean
Shodan is a search engine for Internet-connected devices, from servers and routers to IoT devices and industrial control systems.
- Visit https://www.shodan.io/
- Create an account (free or paid)
- No installation required
¶ Command Line Interface
- Ensure Python and pip are installed on your system
- Install the Shodan CLI:
pip install shodan
- Initialize with your API key:
shodan init YOUR_API_KEY
- Visit your browser's extension store:
- Add the extension to your browser
- Configure with your Shodan API key in the extension settings
- Obtain your API key from your Shodan account dashboard
- For CLI: Configure using
shodan init YOUR_API_KEY
- For browser extension: Enter your API key in the extension settings
- For API usage: Store your API key securely for use in scripts or applications
- Basic search: Enter keywords or filters in the search bar
- Examples:
apache, webcam, country:US port:22
- Filter results using the left sidebar
- Click on a result to view detailed information about the device
- Use "Explore" to browse popular searches and tags
- Check "Maps" to visualize search results geographically
¶ Command Line Interface
# Basic search
shodan search apache
# Get information about a specific IP
shodan host 8.8.8.8
# Download search results
shodan download results apache
# Parse downloaded results
shodan parse results.json.gz
# Get your account information
shodan info
- Network:
net:192.168.0.0/16
- Organization:
org:"Microsoft"
- Country:
country:DE
- Operating System:
os:"Windows Server 2012"
- Port:
port:3389
- Product:
product:nginx
- Vulnerability:
vuln:ms17-010
- Use specific filters to narrow down results and avoid overwhelming data
- Create a paid account for more comprehensive access and higher query limits
- Save important searches for later reference
- Combine multiple filters for precise targeting
- Use the "Alert" feature to monitor for new devices matching your criteria
TheHarvester is a tool designed to gather emails, subdomains, hosts, employee names, open ports, and banners from different public sources.
TheHarvester comes pre-installed on Kali Linux. To update:
sudo apt update
sudo apt install theharvester
# Clone the repository
git clone https://github.com/laramies/theHarvester.git
# Navigate to the directory
cd theHarvester
# Install requirements
pip3 install -r requirements.txt
-
Create an API keys file:
cp api-keys.yaml.sample api-keys.yaml
-
Edit the file to add your API keys:
nano api-keys.yaml
-
Add keys for services you use (optional but recommended):
- Shodan
- Hunter.io
- BinaryEdge
- Censys
- SecurityTrails
- Others as available
theHarvester -d [domain] -b [sources] -l [limit]
-d: Target domain to investigate-b: Data sources to use (comma-separated)-l: Limit the number of results-f: Output file name to save results-S: Start result number (for pagination)-p: Use proxy server-n: Perform DNS reverse lookup
# Basic search against a domain using Google and LinkedIn
theHarvester -d company.com -b google,linkedin
# Search using all available sources and save results
theHarvester -d company.com -b all -f output_file
# Search with limited results
theHarvester -d company.com -b bing -l 200
# DNS brute force to find subdomains
theHarvester -d company.com -b dnsdumpster,virustotal -c
- search engines: google, googleCSE, bing, bingapi, yandex
- social media: linkedin, twitter, instagram
- specialized: shodan, censys, virustotal, dnsdumpster
- Use
-b all to use all available sources
- Console output (default)
- HTML report (
-f output_file -o output.html)
- XML report (
-f output_file -o output.xml)
- JSON format (
-f output_file -o output.json)
- Combine data from multiple sources for the most comprehensive results
- Use targeted searches before trying "all" sources to avoid rate limiting
- Rotate IP addresses or use proxies for large-scale reconnaissance
- Verify findings with other tools to reduce false positives
- Be mindful of API rate limits with services like Shodan or Censys
Mitaka is a browser extension that provides quick lookups for various indicators of compromise (IoCs) across multiple search engines and OSINT services.
- Visit the Chrome Web Store
- Click "Add to Chrome"
- Confirm the installation when prompted
- Visit the Firefox Add-ons page
- Click "Add to Firefox"
- Confirm the installation when prompted
- Click on the Mitaka icon in your browser toolbar
- Select "Options" or "Preferences" (varies by browser)
- Configure which search engines to use for each indicator type:
- IP address
- Domain
- URL
- Hash (MD5, SHA1, SHA256)
- Email
- Cryptocurrency address
- Set default search engines for each indicator type
- Enable or disable specific search engines based on your needs
- Select text on any webpage containing an indicator (IP, domain, hash, etc.)
- Right-click and navigate to the Mitaka submenu
- Choose a search engine to query the selected indicator
- Example: Select an IP address → Right-click → Mitaka → IP → VirusTotal
- Alternatively, click the Mitaka icon in your browser toolbar
- Enter an indicator in the search field
- Select the appropriate search engine from the dropdown
- IP addresses
- Domains
- URLs
- Email addresses
- Cryptocurrency addresses (Bitcoin, Ethereum, etc.)
- File hashes (MD5, SHA1, SHA256)
- ASN
- CVE IDs
- VirusTotal
- AlienVault OTX
- Censys
- Shodan
- DomainTools
- SecurityTrails
- RiskIQ
- IBM X-Force Exchange
- And many more
- Learn the keyboard shortcuts for faster lookups
- Use "Search on all" sparingly as it opens multiple tabs
- Customize the right-click menu to include only services you frequently use
- Pin the extension for quick access during investigations
- Use the "Options" page to organize services in order of preference
¶ Planning and Preparation
- Define clear objectives for your investigation
- Create a structured methodology before starting
- Prepare a secure environment:
- Use a dedicated browser profile or VM
- Consider using a VPN or proxy service
- Keep tools updated to the latest versions
¶ Data Collection and Management
- Document all findings methodically
- Maintain a clear chain of evidence
- Record search parameters used for reproducibility
- Save raw data before processing or analysis
- Use multiple tools to verify information
- Create a consistent file naming convention
- Be aware that some OSINT activities may be traceable
- Use anonymization techniques when appropriate
- Understand legal boundaries of information gathering
- Avoid actions that could be interpreted as hacking or intrusion
- Consider the ethical implications of your investigation
- Develop a systematic approach to move between tools
- Export data in compatible formats when possible
- Use findings from one tool to inform searches in others
- Consider automation for repetitive tasks
- Create templates for common investigation types
¶ Analysis and Reporting
- Distinguish between facts, assumptions, and inferences
- Identify information gaps and limitations
- Create visual representations of complex data relationships
- Structure reports based on audience needs:
- Executive summaries for management
- Technical details for security teams
- Legal-focused reporting for compliance
- Review and update your toolkit regularly
- Stay informed about new OSINT techniques and sources
- Participate in OSINT communities to share knowledge
- Practice with capture-the-flag (CTF) exercises
- Document lessons learned after each investigation
¶ Resources and Further Learning
¶ Communities and Forums
Disclaimer: Always use these tools ethically and legally. Different jurisdictions have different laws regarding information gathering, privacy, and computer access. Ensure your OSINT activities comply with applicable laws and regulations.