IVskiBot

You see IVskiBot in your logs, but never asked for an audit?

Then a website we audit links to your site. Part of an audit is checking that the links on the audited site still work, including links to other websites. That is a short visit to the one address that is linked, not a crawl of your site:

1. IVskiBot reads your /robots.txt, once per audit. If it disallows the linked address for IVskiBot, the link is not checked and nothing else is requested.
2. It sends one HEAD request for the linked address: only the status code is needed.
3. Only if HEAD is answered with an error (many servers do not support HEAD) does it repeat the request once as GET.

IVskiBot does not follow the links on your pages, and it does not read or store your content. It keeps the status code, the response time and a redirect target, if there is one, in a private report for the owner of the linking site. If that redirect leads to another address, that address is checked the same way.

"GET /robots.txt HTTP/1.1" 200 312 "-" "IVskiBot/0.1 (+https://ivski.com/bot)"
"HEAD /blog/some-article HTTP/1.1" 200 0 "-" "IVskiBot/0.1 (+https://ivski.com/bot)"

IVskiBot sends no Referer, so the log does not show which site links to you.

How to block IVskiBot

Recommended: robots.txt

Add these lines to the robots.txt at the root of your host, for example https://example.com/robots.txt:

User-agent: IVskiBot
Disallow: /

After that, IVskiBot requests nothing from your site except robots.txt itself: no link checks and no audits. Your robots.txt is read again on every visit, so a change counts from the next time IVskiBot comes.

User-agent: IVskiBot
Disallow: /private/
Disallow: /checkout

User-agent: IVskiBot
Crawl-delay: 10

Alternative: block it on the server

You can also refuse the user agent with status 403. A link check accepts that, but it still has to knock to get the 403: one HEAD per linked address and, because HEAD failed, one GET. With robots.txt it does not request your pages at all.

if ($http_user_agent ~* "ivskibot") {
    return 403;
}

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ivskibot [NC]
RewriteRule ^ - [F]

What IVskiBot never does

To websites, IVskiBot sends only GET and HEAD requests. It reads, it never writes. It never:

• sends POST, PUT, PATCH, DELETE or any other method
• fills in or submits forms: contact, search, comments, newsletter or sign-up
• tries to log in, guesses passwords or sends credentials
• keeps cookies between requests
• looks for security holes, admin panels, backups or leaked files such as /.env, /.git/, /wp-login.php
• adds attack strings to addresses or parameters (SQL, ../, <script>)
• invents addresses: apart from the standard ones listed below, it requests only addresses it found in links, sitemaps and redirects
• collects your content for a search engine, for AI training or for sale
• ignores robots.txt: a Disallow for IVskiBot is final

The only POST IVskiBot ever sends goes to Google's Chrome UX Report API at chromeuxreport.googleapis.com, an API that returns real-user speed data for a site we audit. It never goes to your server.

Is it really IVskiBot?

Anyone can write IVskiBot into a user agent. If a request with our name does any of the following, it is not IVskiBot. It is a script that borrows our name to look harmless:

dig +short ivski.com A
dig +short ivski.com AAAA

A fake ignores robots.txt anyway, so block it by IP address or with your firewall. If you like, tell us at abuse@ivski.com with the time, the IP address and the request line from your log.

What an audit requests

This section is about websites we audit: our own, and those whose owners asked for an audit. Linked websites only get the link check described above.

Beyond HTTP

• a TLS handshake on port 443: certificate, protocol version, expiry
• DNS lookups: MX, SPF, DMARC, DKIM, CAA, DNSSEC
• a short conversation with the domain's mail servers on port 25: EHLO, STARTTLS, QUIT. No e-mail is sent.
• public sources that never touch your server: Certificate Transparency logs (crt.sh, Cert Spotter) for the host names of the domain, the RDAP registry for the domain's expiry date, and Google's Chrome UX Report

Pace and limits

• By default at least 250 ms between two requests (at most 4 per second) and at most 4 requests at a time
• a Crawl-delay in robots.txt makes it slower, up to 30 seconds between requests
• on 429 Too Many Requests, or 503 with Retry-After, it waits as long as asked (up to a minute), slows down for the rest of the visit and tries at most twice more
• by default at most 1,000 pages and 2,000 files per site, at most 10 clicks deep from the start page

Browser check

On a site we audit, IVskiBot can also open a few pages in a headless Chrome browser, to see them the way a first-time visitor does: speed (Core Web Vitals), accessibility, keyboard navigation with Tab, screenshots in desktop and phone size, and the cookie banner. It records what loads before consent, then clicks Reject and records what still loads after that.

This visit uses the ordinary Chrome user agent, not IVskiBot, because many cookie banners are not shown to bots. The page's own scripts run as in any browser, so any requests they make come from the page's own code. The browser check never runs on websites that are only linked, and it does not run when robots.txt keeps IVskiBot away from the start page.

User-agent: IVskiBot
Allow: /

Audit reports are private. They stay with IVski and the owner of the audited site and are not published.

Contact

Questions about IVskiBot, a complaint, or an audit of your own website: abuse@ivski.com. IVskiBot is run by IVski, see the Impressum.