What is Website Finder?
Website Finder is a software tool for searching for websites by your set criteria. By default the homepage of each website is checked for the criteria but the URL that is returned from Google searches can also be checked.
What criteria is available?
- HTML – does the website(s) contain certain HTML
- HTTP codes – does server return a specific type of HTTP code e.g a 404 HTTP error code
- HTTP error code type – does the website(s) return an certain error code type, eg failure, not allowed, redirect
- Websites that have Google analytics installed – does the website(s) have the Google Analytics javascript on its homepage
- Websites where meta data contains specific text – does the homepage title, keywords, and description contain specific text
- Websites where title tag contains specific text
- Websites where title tag is missing
- Websites that are using WordPress
- Websites that are using Joomla
- Websites that are using Magento
- Websites where domain contains specific text
- Websites where URL contains specific text
- Websites that are linking to specific outbound domains
- Websites where page load time is over X amount of seconds
- Websites where the HTTP responce header contains specific text – e.g. Apache 2.1 / PHP 4.3
How does this software work?
Website Finder has three methods of finding websites matching your criteria.
- Crawl a list of websites for matching criteria
If you have a list of websites that you would like to check simply paste them in and Website Finder will check the homepage of each one for your set criteria. - Crawl search results by niche related keywords for matching websites/criteria
Enter a list of niche related keywords and Webiste Finder will search Google for all these niche related keywords and check every result (website) for your set criteria. There are two main options with this search, the criteria check can be applied to the result (URL) Google returns or the homepage of the result that is returned. - Endless crawl for matching websites
Enter your seed list of websites to start crawling from and Website Finder will check that inital seed website(s) for your criteria but then also crawl a specified amount of URLs on that website (30 by default) extracting all outbound links and then crawling these outbound websites. This process is repeated endlessly. Website Finder will just keep crawling from website to website checking to see if they match your set criteria.
Why should I use this tool?
List of possible uses are but not limited to…
- Find websites with outdated plugins to market to
- Find slow websites to sell faster hosting to
- Find websites with security issues to offer security patching services to
- Find websites to sell themes to, e.g use the Google search option to find all dog related websites running WordPress and sell them dog related WordPress themes
Tips for use
- If you are checking websites that are being returned from a Google scrape make sure you have a sensible delay when sending requests to Google to avoid your IP adderess being blocked
- See the most Pinterest pinned pages for a keyword.
- More threads doesn’t always mean faster, there will be a point depending on your internet connection and CPU/RAM where you’ll hit your fastest speed. For me on village broadband with an i7 this is 3 threads per website and 40 concurrent websites.
- When doing an endless crawl make sure you pick a website where many outbound links can be found to crawl next, such as reddit.
Sorry, You need to be logged in to access this page.
GET FULL ACCESS all the contents on this site if you are VIP MEMBERS.
