| 69 different robots | Hits | Bandwidth | Last visit |
|---|
| WGet tools | 9,596,740 | 30584.39 GB | 31 May 2020 - 23:59 |
| Common *nix tool for automating web document retireval. Most likely a bot. | 147,070 | 443.76 GB | 31 May 2020 - 23:59 |
| SemrushBot | 50,176 | 225.94 GB | 31 May 2020 - 23:59 |
| Unknown robot (identified by 'crawl') | 49,239 | 682.34 GB | 31 May 2020 - 16:04 |
| Unknown robot (identified by empty user agent string) | 35,222 | 559.11 GB | 31 May 2020 - 23:58 |
| Python-urllib | 19,538 | 37.27 GB | 31 May 2020 - 23:50 |
| AhrefsBot | 16,182 | 569.90 GB | 31 May 2020 - 23:57 |
| SEOkicks Webcrawler | 10,660 | 106.83 GB | 31 May 2020 - 23:59 |
| check_http (nagios) | 8,581 | 27.72 MB | 31 May 2020 - 23:49 |
| Yandex Bot | 7,343 | 346.05 GB | 31 May 2020 - 23:45 |
| Bingbot | 5,833 | 78.09 GB | 31 May 2020 - 23:59 |
| DotBot, Open Site Explorer | 5,581 | 85.17 GB | 31 May 2020 - 23:27 |
| BLEXBot, seems to belong to the WebMeUp backlink tool | 4,843 | 4.88 GB | 31 May 2020 - 21:14 |
| Googlebot | 3,844 | 68.16 GB | 31 May 2020 - 23:57 |
| Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 3,390 | 15.79 GB | 31 May 2020 - 23:45 |
| Alexa (IA Archiver) | 3,055 | 99.67 GB | 28 May 2020 - 10:18 |
| Unknown robot (identified by 'spider') | 1,572 | 32.38 GB | 31 May 2020 - 20:15 |
| Java (Often spam bot) | 1,035 | 1.36 GB | 31 May 2020 - 23:54 |
| BaiDuSpider | 1,014 | 67.38 GB | 31 May 2020 - 22:50 |
| MagpieRSS | 734 | 144.18 GB | 31 May 2020 - 23:24 |
| Perl tool | 619 | 6.05 GB | 31 May 2020 - 15:24 |
| FavIconizer | 451 | 2.68 MB | 31 May 2020 - 22:53 |
| Suspected bot masqurading as Mozilla | 398 | 19.21 GB | 30 May 2020 - 10:07 |
| MJ12bot | 383 | 11.66 GB | 31 May 2020 - 14:13 |
| Common Crawl | 323 | 224.05 MB | 31 May 2020 - 18:13 |
| Mail.ru bot | 294 | 487.50 MB | 31 May 2020 - 13:11 |
| ning | 168 | 9.69 GB | 31 May 2020 - 09:08 |
| Suspected bot masqurading as Mozilla | 138 | 14.48 KB | 31 May 2020 - 13:24 |
| 360spider | 99 | 7.44 GB | 26 May 2020 - 23:18 |
| Sogou Spider | 96 | 123.12 MB | 31 May 2020 - 08:14 |
| LinkChecker | 90 | 0 | 30 May 2020 - 20:04 |
| Web Core / Roots | 82 | 2.68 GB | 31 May 2020 - 23:18 |
| SeznamBot | 74 | 1.71 GB | 31 May 2020 - 22:51 |
| Jakarta commons-httpclient | 69 | 171.60 MB | 22 May 2020 - 06:34 |
| Sift | 66 | 1.13 GB | 31 May 2020 - 17:23 |
| Custo | 53 | 6.32 GB | 31 May 2020 - 01:36 |
| FaceBook bot | 43 | 2.13 GB | 28 May 2020 - 12:42 |
| YaCy | 35 | 1023.56 MB | 27 May 2020 - 10:26 |
| Netcraft | 33 | 36.22 KB | 30 May 2020 - 01:47 |
| SMTBot | 31 | 92.92 KB | 12 May 2020 - 20:33 |
| Googlebot-Image | 29 | 27.41 KB | 30 May 2020 - 19:06 |
| MojeekBot | 23 | 21.16 MB | 21 May 2020 - 03:46 |
| Cliqzbot | 20 | 12.00 MB | 29 May 2020 - 14:06 |
| twitterbot | 15 | 749.55 MB | 29 May 2020 - 15:56 |
| Bing Preview bot | 15 | 6.84 GB | 29 May 2020 - 10:16 |
| Phantom | 13 | 28.98 KB | 26 May 2020 - 19:02 |
| CFNetwork | 12 | 15.81 GB | 30 May 2020 - 05:06 |
| Suspected bot masqurading as Mozilla | 10 | 19.84 KB | 25 May 2020 - 02:47 |
| ltx71 | 10 | 48.90 MB | 29 May 2020 - 07:06 |
| WebCopy | 8 | 13.15 KB | 16 May 2020 - 21:03 |
| archive.org bot | 8 | 17.38 KB | 31 May 2020 - 07:37 |
| Fish search | 6 | 15.34 KB | 31 May 2020 - 19:02 |
| WordPress | 5 | 48.61 MB | 25 May 2020 - 05:46 |
| Turn It In | 5 | 21.02 MB | 28 May 2020 - 03:33 |
| A PHP script | 4 | 13.27 KB | 28 May 2020 - 16:25 |
| Exabot | 3 | 9.80 KB | 30 May 2020 - 08:58 |
| InfoBot | 2 | 8.78 KB | 22 May 2020 - 02:29 |
| Unknown robot (identified by 'robot') | 2 | 15.34 KB | 15 May 2020 - 15:47 |
| IstellaBot | 2 | 6.70 KB | 10 May 2020 - 18:47 |
| yeti | 2 | 5.54 KB | 27 May 2020 - 23:47 |
| InternetArchive | 2 | 0 | 09 May 2020 - 17:11 |
| NG 1.x (Exalead) | 2 | 0 | 21 May 2020 - 18:03 |
| MSNBot-media | 1 | 6.13 KB | 16 May 2020 - 02:47 |
| Monster | 1 | 4.36 KB | 03 May 2020 - 18:26 |
| HTTrack off-line browser | 1 | 5.56 KB | 24 May 2020 - 12:59 |
| Ruby script | 1 | 3.22 KB | 04 May 2020 - 19:06 |
| Cfetch | 1 | 772 Bytes | 06 May 2020 - 04:14 |
| yie8 | 1 | 292.09 MB | 14 May 2020 - 15:36 |
| Yahoo Slurp | 1 | 14.55 KB | 02 May 2020 - 16:39 |
|