What is OSINT?
May 05, 2023
The meaning of this abbreviation uncovers the whole task of that phenomenon. The intelligence conducted based on the open-source data works exceptionally well when you need to keep the fact of surveillance secret. All breaches or attempts of ones may easily be detected by system administrators or other qualified personnel. Meanwhile, obtaining the info about the co or the person on Instagram or the website with local companies’ information is much more stealthy and easier.
It is essential to uncover how that works. It is hard to describe how much information you leave while acting on the Internet. And it is not your fault if you are not paranoid about your privacy and don’t think twice about each message. You mentioned your location here and there and said when you usually finish your workday – and that is already enough to say a lot about you.
Comprehension of shattered facts about a person or a company is not an easy task, especially when in OSINT you rely on databases or unstructured info rather than on messages in social networks. At that moment, OSINT tools come into view.
OSINT Tools
Primary task of OSINT toolkits is to make the data analysis more accessible by structuring it. But most modern OSINT tools offer the ability to sort your info in a preferred way and ready databases to analyze. Yes, that may look like an attempt to be more clever than the user is, but it is just the way to make these semi-legal tools less fragmented.
Majority of the OSINT tools are available even in the surface Web - on GitHub, in particular. However, some of them - especially ones for deep analysis - could be found only in the Darknet. Some special tools designed for actions that can easily be classified as spying are placed for sale on the Darknet forums or sales platforms.
| Name | Source | Description | Language | Price |
|---|---|---|---|---|
| Amass | Git | DNS enumeration and network mapping tool suite: scraping, recursive brute forcing, crawling web archives, reverse DNS sweeping | Go | 🆓 |
| Asnlookup | Git | Leverage ASN to look up IP addresses (IPv4 & IPv6) owned by a specific organization for reconnaissance purposes, then run port scanning on it | Python | 🆓 |
| AttackSurfaceMapper | Git | Subdomain enumerator | Python | 🆓 |
| AutoRecon | Git | Multi-threaded network reconnaissance tool which performs automated enumeration of services | Python | 🆓 |
| badKarma | Git | Advanced network reconnaissance tool | Python | 🆓 |
| Belati | Git | OSINT tool, collect data and document actively or passively | Python | 🆓 |
| Certstream | Git | Intelligence feed that gives real-time updates from the Certificate Transparency Log network | Elixir | 🆓 |
| Darkshot | Git | Lightshot scraper with multi-threaded OCR and auto categorizing screenshots | Python | 🆓 |
| dataleaks | Git | Self-hosted data breach search engine | PHP | 🆓 |
| datasploit | Git | OSINT framework, find, aggregate and export data | Python | 🆓 |
| DeadTrap | Git | Track down footprints of a phone number | Python | 🆓 |
| DNSDumpster | Domain research tool that can discover hosts related to a domain | - | 🆓 | |
| dnsenum | Git | DNS reconnaissance tool: AXFR, DNS records enumeration, subdomain bruteforce, range reverse lookup | Perl | 🆓 |
| dnsenum2 | Git | Continuation of dnsenum project | Perl | 🆓 |
| DNSRecon | Git | DNS reconnaissance tool: AXFR, DNS records enumeration, TLD expansion, wildcard resolution, subdomain bruteforce, PTR record lookup, check for cached records | Python | 🆓 |
| dnsx | Git | Multi-purpose DNS toolkit allow to run multiple DNS queries | Go | 🆓 |
| EagleEye | Git | Image recognition on instagram, facebook and twitter | Python | 🆓 |
| Espionage | Git | Domain information gathering: whois, history, dns records, web technologies, records | Python | 🆓 |
| eTools.ch | Metasearch engine, query 16 search engines in parallel | - | 🆓 | |
| Facebook_OSINT_Dump | Git | OSINT tool, facebook profile dumper, windows and chrome only | Shell | 🆓 |
| FinalRecon | Git | Web reconnaissance script | Python | 🆓 |
| Findomain | Git | Fast subdomain enumerator | Rust | 🆓 |
| FOCA | Git | OSINT framework and metadata analyser | Csharp | 🆓 |
| GHunt | Git | Investigate Google accounts with emails and find name, usernames, Youtube Channel, probable location, Maps reviews, etc. | Python | 🆓 |
| gitGraber | Git | Monitor GitHub to search and find sensitive data in real time for different online services such as: Google, Amazon, Paypal, Github, Mailgun, Facebook, Twitter, Heroku, Stripe, etc. | Python | 🆓 |
| Gorecon | Git | Reconnaissance toolkit | Go | 🆓 |
| GoSeek | Git | Username lookup comparable to Maigret/Sherlock, IP Lookup, License Plate & VIN Lookup, Info Cull, and Fake Identity Generator | - | 🆓 |
| gOSINT | Git | OSINT framework; find mails, dumps, retrieve Telegram history and info about hosts | Go | 🆓 |
| h8mail | Git | Email OSINT & Password breach hunting tool; supports chasing down related email | Python | 🆓 |
| Harpoon | Git | CLI tool; collect data and document actively or passively | Python | 🆓 |
| holehe | Git | Check if the mail is used on different sites like Twitter, Instagram and will retrieve information on sites with the forgotten password function | Python | 🆓 |
| IVRE | Git | IVRE (Instrument de veille sur les réseaux extérieurs) or DRUNK (Dynamic Recon of UNKnown networks); network recon framework including tools ofr passive and active recon | Python | 🆓 |
| kitphishr | Git | Hunts for phishing kit source code by traversing URL folders and searching in open directories for zip files; supports list of URLs or PhishTank | Go | 🆓 |
| Kostebek | Git | Tool to find firms domains by searching their trademark information | Python | 🆓 |
| LeakDB | Git | Normalize, deduplicate, index, sort, and search leaked data sets on the multi-terabyte-scale | Go | 🆓 |
| LeakIX | Git | Search engine for devices and services exposed on the Internet | - | 🆓 |
| LeakLooker | Git | Discover, browse and monitor database/source code leaks | Python | 🆓 |
| leakScraper | Git | Set of tools to process and visualize huge text files containing credentials | Python | 🆓 |
| LinEnum | Git | System script for local Linux enumeration and privilege escalation checks | Shell | 🆓 |
| LittleBrother | Git | Information gathering (OSINT) on a person (EU), checks social networks and Pages Jaunes | Python | 🆓 |
| Maigret | Git | Collect a dossier on a person by username from a huge number of sites and extract details from them | Python | 🆓 |
| MassDNS | Git | High-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration) | C | 🆓 |
| Metabigor | Git | OSINT tool that doesn't require any API key | Go | 🆓 |
| Netflip | Git | Scrape sensitive information from paste sites | CSharp | 🆓 |
| NExfil | Git | Finding profiles by username over 350 websites | Python | 🆓 |
| Nmmapper | Cybersecurity tools offered as SaaS: Nmap, subdomain finder (Sublist3r, DNScan, Anubis, Amass, Lepus, Findomain, Censys), theHarvester, etc. | - | 💰 | |
| nqntnqnqmb | Git | Retrieve information on linkedin profiles, companies on linkedin and search on linkedin companies/persons | Python | 🆓 |
| Oblivion | Git | Data leak checker and monitoring | Python | 🆓 |
| ODIN | Git | Observe, Detect, and Investigate Networks, Automated reconnaissance tool | Python | 🆓 |
| Omnibus | Git | OSINT framework; collection of tools | Python | 🆓 |
| OneForAll | Git | Subdomain enumeration tool | Python | 🆓 |
| OnionSearch | Git | Script that scrapes urls on different .onion search engines | Python | 🆓 |
| OSINT Framework | Git | A web-based collection of tools and resources for OSINT | Javascript | 🆓 |
| Osintgram | Git | Interactive shell to perform analysis on the Instagram account of any users by their nickname | Python | 🆓 |
| Osmedeus | Git | Automated framework for reconnaissance and vulnerability scanning | Python | 🆓 |
| Photon | Git | Fast crawler designed for OSINT | Python | 🆓 |
| PITT | Git | Web browser loaded with links and extensions for doing OSINT | - | 🆓 |
| ProjectDiscovery | Git | Monitor, collect and continuously query the assets data via a simple webUI | Go | 🆓 |
| Qualear | Git | Reconnaissance Apparatus; Information gathering, a conglomerate of tools including custom algorithms, API wrappers | Go | 🆓 |
| ReconDog | Git | Multi-purpose reconnaissance tool, CMS detection, reverse IP lookup, port scan, etc. | Python | 🆓 |
| reconFTW | Git | Perform automated recon on a target domain by running a set of tools to perform scanning and finding out vulnerabilities | Shell | 🆓 |
| Recon-ng | Git Repository (inactive) | Web-based reconnaissance tool | Python | 🆓 |
| Reconnoitre | Git | Tool made to automate information gathering and service enumeration while storing results | Python | 🆓 |
| ReconScan | Git | Network reconnaissance and vulnerability assessment tools | Python | 🆓 |
| Recsech | Git | Web reconnaissance and vulnerability scanner tool | PHP | 🆓 |
| Redscan | Git | Mix of a security operations orchestration, vulnerability management and reconnaissance platform | Python | 🆓 |
| Red Team Arsenal | Git | Automated reconnaissance scanner and security checks | Python | 🆓 |
| reNgine | Git | Automated recon framework for web applications; customizable scan engines & pipeline of reconnaissance | Python | 🆓 |
| SearchDNS | Netcraft tool; Search and find information for domains and subdomains | - | 🆓 | |
| Sherlock | Git | Hunt down social media accounts by username across social networks | Python | 🆓 |
| Shodan | Search devices connected to the internet; helps find information about desktops, servers, and IoT devices; including metadata such as the software running | - | 🆓 | |
| shosubgo | Git | Grab subdomains using Shodan api | Go | 🆓 |
| shuffledns | Git | Wrapper around masses that allows you to enumerate valid subdomains using active brute force as well as resolve subdomains with wildcard handling and easy input-output support | Go | 🆓 |
| SiteBroker | Git | Tool for information gathering and penetration test automation | Python | 🆓 |
| Sn1per | Git | Automated reconnaissance scanner | Shell | 💰 |
| spiderfoot | Git | OSINT framework, collect and manage data, scan target | Python | 🆓 |
| Stalker | Git | Automated scanning of social networks and other websites, using a single nickname | Python | 🆓 |
| SubDomainizer | Git | Find subdomains and interesting things hidden inside, external Javascript files of page, folder, and Github | Python | 🆓 |
| Subfinder | Git | Discovers valid subdomains for websites, designed as a passive framework to be useful for bug bounties and safe for penetration testing | Go | 🆓 |
| Sublist3r | Git | Subdomains enumeration tool | Python | 🆓 |
| Sudomy | Git | Subdomain enumeration tool | Python | 🆓 |
| Th3inspector | Git | Multi-purpose information gathering tool | Perl | 🆓 |
| theHarvester | Git | Multi-purpose information gathering tool: emails, names, subdomains, IPs, URLs | Python | 🆓 |
| tinfoleak | Git | Twitter intelligence analysis tool | Python | 🆓 |
| Totem | Git | Retrieve information about ads of a Facebook page, retrieve the number of people targeted, how much the ad cost, and a lot of other information | Python | 🆓 |
| trape | Git | Analysis and research tool, which allows people to track and execute intelligent social engineering attacks in real time | Python | 🆓 |
| TWINT | Git | Twitter Intelligence Tool; Twitter scraping & OSINT tool that doesn't use Twitter's API, allowing one to scrape a user's followers, following, Tweets and more while evading most API limitations | Python | 🆓 |
OSINT as an attack vector
Let’s look at where you can get the information about the suspect you’re interested in. Once again - all data from all OSINT sources can be gathered and analyzed manually, without any additional software. Apps are needed to make this process faster and easier, one may say - available for everyone.
How much can you get by having only your target's email address? Surprisingly, a lot. With the help of Google, you can uncover various facts, including shocking ones. But only by analyzing the exact email address can you get the user’s name, year of birth (or even the full date), or other important dates. The pronunciation of this individual, along with the language and possible country, may be there, too.
Applying Google or other search engines (or even special tools), you can find the social network connections for this particular email. Sometimes, you will also find the related images, domains registered with this email, and other things. If there was a breach with this email in the leaked data pack - you will likely see this pack, too.
Social networks (Twitter, Instagram examples)
Twitter and Instagram are where people are not shy to uncover their details. What is more important for OSINT - these networks have a lot of valuable data exposed by default. Hence, all things are available - just take it and use it. And there are a lot of new users to both of these networks who still don’t know how to hide their private information.
Main points of interest in these social networks are concentrated in profiles and posts. The information lies on the surface, so you don’t need to run any additional tools to complete the jigsaw. Instagram reconnaissance surface is the target’s username, followers and followed accounts, its websites, biography in the profile top, and so on. In the posts, you will find enough information to track this person's favorite places or the friends of your object of interest.
Twitter offers other data to observe, but you will have almost the same full picture. You can see the profile pic and track the last replies of the user, see the join date and birthday. Some users add a location to their profile - this action is not obligatory but may say a lot if the target does not try to counteract the reconnaissance. Twitter also allows everyone to save the media attached to the message. That allows you to make the image analysis - see the next paragraph.
Images analysis
Metadata in the images is a juicy source of information about who, where, and how it was made. Even if you deal with screenshots, you can figure out the screen characteristics, phone model, operating system, and camera app. Going deeper may uncover the geolocation and time when the photo was taken. However, metadata is pretty hard to analyze without the special software - it looks like a row of slightly structured numbers and letters.
Another edge of image analysis touches the exact contents of the image. Buildings, persons, views - all these things will help you to determine location and circumstances. For such an analysis, you will likely need to use Google Street View, Google Maps, and Google Lens. All of them are free and pretty effective. However, people who have a decent knowledge about cybersecurity and privacy will likely wipe the metadata - manually or with the use of special tools.
IP analysis
The old good verbal threat to figure out the location by the IP address is not as silly a joke as it could look. Sure, you will still likely fail to get the precise location, but there are many other things to compensate for that. This analysis is done prevalently using special tools - websites or web applications.
First step into IP analysis is to make sure that you are dealing with a real IP, not a VPN server or Tor Exit Node IP-addresses. After clearing that moment, you may step forward - to check the IP properties. You can use certain sites to determine if that IP is static or dynamic, private or public, and check if Cloudflare bans it. By the way, services like Cloudflare can reveal to you if that IP is associated with threat actions or spamming.
Scanning that IP address with basic network tools will show you the open ports on that address, OS, hostname, and the services run under that IP. Using some specific keywords and functions of the torrent-trackers, you can get what did the user download, which search engines used in both Surface and Dark Web.
YouTube
The most popular video hosting sites also keep secrets poorly. However, that is just the specific model of that social network. You can easily track the comments of the person you need by having their nickname. On the exact YouTube page, you can see which games this user plays, what are his typical activity hours, and possibly get some details on his personal life. Location may be obtained through either checking the activities and in the Community tab.
Another part of the information hides in the Channel Info section. Here you can see the subscriptions of that user, channels they like, email of that account, joined date, and a lot of other interesting stuff. What will you do with all this information - that is your choice. But YouTube offers one of the biggest surfaces for OSINT attacks.
Of course, that is not a full list of possible vectors of open-source intelligence. Moreover, there is no guarantee that you will get even a thing from the object of intelligence. Users who know the key rules of online anonymity will leave no significant tracks. Ones who know the same OSINT vectors will also understand how to avoid a successful survey or even mislead it.
Who and why uses the OSINT?
Open-source intelligence is demanded by a wide range of different users. Don’t think that if it contains the word “intelligence,” thus it is demanded only by the FBI and cybercriminals. They use it, too, but they have access to more serious and sophisticated surveillance tools. Meanwhile, OSINT is widespread in companies and sometimes used by individuals. Companies use it to prepare for negotiations with a company they have never had a deal with. Knowing more about your potential partner or rival before the talks is useful.
Individuals, on the other hand, use open-source intelligence to get some information. OSINT may be extremely useful if you want to know more about the company you’re trying to get a job in or see who that beautiful girl you met in the park is. That may look strange and overkill, but stopping before that goes too far is possible.
Is OSINT legal?
Unless you use illegal software for it - yes. However, illegal programs for OSINT that you can see for sale on the Darknet are rather a spyware than just a program for data analysis. That’s nothing bad in checking the available information without any evil intentions. However, a benevolent search may turn malevolent when you go too far to get the information. Searching online is OK, starting a phishing campaign to get the info out of the employees is not.
However, from a moral point of view, any type of intelligence is bad. If someone doesn't want to say something to you, they likely have some motivation. And searching for some info on that topic in the social networks of this person may be harmful to your relationships. Sure, it cannot be called illegal from the legislative point of view but can make you an ill-famed maniac. Reconnaissance based on open-source data is a potent tool, so use it wisely to keep your reputation clean.
Frequently Asked Questions
Although OSINT sounds quite powerful, it is not without its drawbacks. Many professionals using OSINT experience information overload. It may take too long to filter every detail. Further, with social media intelligence (SOCMINT), it is tough to verify the legitimacy of each part collected. One disadvantage of OSINT tools is that many do not have fact-checking features, making it difficult to distinguish between real and fake news.
The other problem of OSINT is its social acceptance. A lot of people will consider gathering the information on social networks and wherever else as spying, even though no privacy borders are corrupted. It is understandable that folks typically dislike when someone collects facts about them, but as we mentioned earlier, there is nothing illegal in open-source intelligence.