DomCop Bot is the web crawler behind the DomCop Guru plan. DomCop members ask it to crawl websites of their choice, and it looks for links whose destination domain no longer exists, so that the member can find expired domains worth registering.

To do that it reads the pages of the websites it was asked to crawl, follows the links within those websites up to the depth the member chose, and checks the domain name behind every outgoing link. It records only domain names of links that lead nowhere, never page content, e-mail addresses or any personal data. It also checks whether a website is still available on a domain by reading the domain's robots.txt file, once per domain.

Bot Type: Good (Identifies itself, has an official moniker)
Category: Marketing
Obeys Robots.txt: Yes
Obeys Crawl Delay: Yes
User-Agent String: DomCopBot (https://www.domcop.com/bot)

Request rate: at most two requests to a website at any one time and at least three seconds between two requests to the same host. A longer Crawl-delay in your robots.txt is respected. The bot reads HTML pages only, up to 2 MB each, and does not fetch images, scripts, stylesheets or downloads.

Traffic claiming to be DomCopBot can be checked against this page. Genuine DomCopBot requests come only from the crawl address listed here, which we keep current:

Crawl address: 45.79.173.72 (crawler.domcop.com)

The crawl address supports forward-confirmed reverse DNS, the same verification method the major crawlers document: a reverse lookup of the connecting IP address returns the hostname listed above, and a forward lookup of that hostname returns the same IP address. Requests from any other address using our user-agent string are not ours.

The DomCop Bot strictly follows the robots.txt file on your website.
To prevent the DomCop Bot from visiting your site, put the two following lines into the robots.txt file on your server:

To let it visit but slow it down further, add a crawl delay in seconds:

You can read more about the robots.txt file in a Google support page.
If you have any questions or need more information, please email us at support@domcop.com.