Home > Information Security > Search Engines for OSINT and Recon

Search Engines for OSINT and Recon

January 31st, 2014

Based on the title to this post, you’re thinking, “Awesome, Dave! Welcome to 2006!” Well hang on there. There’s an amazing number of awesome search facilities that can be useful when doing OSINT and recon work for pen testing. I’ll list a lot of different sites that I have discovered and use regularly for both.

Google and Bing: These are the best known sites for these activities, and provide a lot of data. One cool feature many don’t use is the “Search Tools” in Google Search and Image Search. After you submit your search, click the “Search Tools” button in the upper right. You can hone in on date ranges for results, and for images you can choose only faces, clip art, and others, as well. Lots of good query tools out there for these two, but SearchDiggity from the Bishop Fox guys rocks.

GoogleSearchTools

 

 

 

 

 

 

 

The Wayback Machine: I think most folks are aware of this these days, too, but it deserves to be mentioned. If you’re looking for older versions of a site or data set, this is the place to grab it.

Google Groups or Yahoo GroupsSearching for gossip on companies or people posting stupid things that they shouldn’t may lead you to one of these sites. I’d say you’ll be surprised by what you find there, but…we all know that people just feel like they HAVE to share sometimes.

Bing vs. Google: For a comparison of the two search engines’ results, this is a great site, and may save you some time for quick keyword searches.

2LingualTo search using two different languages simultaneously on Google, consider 2Lingual. Great results sometimes, especially if you’re assessing an international organization or person who travels and works abroad.

2lingual

 

 

 

 

 

 

 

Carrot2Carrot2 is one of my new favorites. It provides some cool visualization options for searches, and also has great results.

carrot2

 

 

 

 

 

 

 

 

iSeek: iSeek is another of my favorites currently, primarily due to its categorization column on the left-hand side after searching. You can drill down into phone numbers, locations, and other keywords detected very quickly.

iseek

 

 

 

 

 

 

 

 

 

 

International Search with Yandex (Russian) and Baidu (Chinese): For international targets, these sites turn up some fascinating results, and both have good search delimiters, too.

GlobalFileSearch: A quick way to search for files scanned across FTP services. Similar to the “filetype:” or “ext:” queries with Google, but may prove useful…I’ve found a few excellent results there.

GlobalFileSearch

 

 

 

 

 

 

 

 

 

NerdyData: This site rocks, especially when I am doing recon for Web app tests. You can search for code snippets buried in site pages, JavaScript, etc. IMMENSELY USEFUL. You can look for code, patterns in code, look specifically for <meta> tags, look only for blog keywords, etc. Awesomesauce.

NerdyData

 

 

 

 

 

 

 

Qwant: Qwant sometimes has interesting info, as it aggregates across traditional search, social media, shopping sites, and more traditional news.

If anyone has different engines and sites they use for this, I’d love to hear about them, and I’m sure others would too. Any comments with new sites, I’ll vet and add to the post. Cheers!

  1. February 3rd, 2014 at 16:22 | #1

    EchoSec – https://echosec.net/ – demo: https://app.echosec.net/

    We’ve been working on the EchoSec beta and thought you might enjoy playing with it for OSINT. Our slide deck http://www.slideshare.net/kswannie/echo-sec-slideshare

    This is a great tool to start a search, when you don’t know anything but a location. Please feel free to give it a try.

  2. Sue Young
    February 3rd, 2014 at 18:45 | #2

    I just had a GIAC Gold paper accepted on using OSINT for evaluating business partners. I use Search Diggity, Shodan, FOCA, and recon-ng.

    When you know the general type of information you need it’s great to use recon-ng to automate your searches. FOCA will let you search documents for metadata. It’s useful to get version of internal software but will also search Shodan and look for software versions. Shodan is what I always use for initial screenings to see what ports are open. I check the sites I’m responsible for as well as checking business partners and vendors. I focus on tools that don’t touch the target.

    You would not believe the lack of firewalls on internet facing systems. At work my group was talking about some control systems we have and we started wondering about the likelihood of finding them unprotected on the internet. I checked last night and found over 100,000 of the same type of system visible on Shodan. There is a lot of work to be done.

    My paper is here if anyone’s interested. http://www.sans.org/reading-room/whitepapers/bestprac/open-source-reconnaissance-tools-business-partner-vulnerability-assessment-34490

Comments are closed.