| 61 different robots | Hits | Bandwidth | Last visit |
|---|
| WGet tools | 3,483,614 | 8224.87 GB | 31 May 2021 - 23:59 |
| Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 166,071 | 633.70 GB | 31 May 2021 - 23:59 |
| Unknown robot (identified by 'crawl') | 158,504 | 722.27 GB | 31 May 2021 - 23:59 |
| Bingbot | 96,262 | 599.85 GB | 31 May 2021 - 23:59 |
| Common *nix tool for automating web document retireval. Most likely a bot. | 25,457 | 128.65 GB | 31 May 2021 - 23:57 |
| SemrushBot | 17,117 | 101.21 GB | 31 May 2021 - 23:57 |
| Unknown robot (identified by empty user agent string) | 14,756 | 78.25 GB | 31 May 2021 - 23:55 |
| AhrefsBot | 9,558 | 170.75 GB | 31 May 2021 - 23:56 |
| check_http (nagios) | 8,579 | 27.73 MB | 31 May 2021 - 23:55 |
| Googlebot | 7,542 | 64.51 GB | 31 May 2021 - 23:52 |
| Python-urllib | 2,869 | 24.17 GB | 31 May 2021 - 16:05 |
| DotBot, Open Site Explorer | 2,320 | 21.64 GB | 31 May 2021 - 23:28 |
| MJ12bot | 1,756 | 41.03 GB | 31 May 2021 - 15:17 |
| BLEXBot, seems to belong to the WebMeUp backlink tool | 1,392 | 207.66 MB | 30 May 2021 - 01:38 |
| Unknown robot (identified by 'spider') | 771 | 4.77 GB | 31 May 2021 - 18:03 |
| Java (Often spam bot) | 682 | 14.91 MB | 31 May 2021 - 22:08 |
| Yandex Bot | 634 | 20.95 GB | 31 May 2021 - 15:14 |
| SEOkicks Webcrawler | 619 | 8.80 GB | 31 May 2021 - 21:12 |
| Perl tool | 562 | 7.64 GB | 31 May 2021 - 07:10 |
| Common Crawl | 405 | 409.45 MB | 19 May 2021 - 00:26 |
| Suspected bot masqurading as Mozilla | 381 | 30.33 KB | 31 May 2021 - 23:50 |
| YaCy | 250 | 4.46 GB | 31 May 2021 - 06:41 |
| Mail.ru bot | 219 | 1.19 GB | 31 May 2021 - 20:42 |
| FavIconizer | 189 | 649.06 KB | 31 May 2021 - 21:13 |
| FaceBook bot | 175 | 17.65 GB | 27 May 2021 - 23:28 |
| Bing Preview bot | 150 | 7.98 GB | 31 May 2021 - 11:39 |
| Yahoo Slurp | 142 | 21.23 GB | 26 May 2021 - 22:58 |
| Web Core / Roots | 134 | 869.58 MB | 27 May 2021 - 13:52 |
| ltx71 | 128 | 2.64 GB | 30 May 2021 - 07:30 |
| LinkChecker | 124 | 327.06 MB | 31 May 2021 - 17:58 |
| SeznamBot | 117 | 566.65 MB | 31 May 2021 - 18:39 |
| Googlebot-Image | 91 | 30.84 MB | 31 May 2021 - 13:14 |
| BaiDuSpider | 62 | 4.26 GB | 31 May 2021 - 13:53 |
| Google Web Preview | 55 | 118.71 KB | 27 May 2021 - 07:01 |
| Custo | 50 | 167.29 KB | 29 May 2021 - 05:41 |
| SMTBot | 44 | 82.24 MB | 14 May 2021 - 05:11 |
| Unknown robot (identified by 'robot') | 41 | 9.79 GB | 29 May 2021 - 19:24 |
| Suspected bot masqurading as Mozilla | 37 | 646.26 MB | 23 May 2021 - 13:25 |
| MojeekBot | 30 | 313.50 KB | 30 May 2021 - 15:57 |
| Sift | 22 | 73.06 KB | 29 May 2021 - 20:41 |
| Netcraft | 22 | 72.68 KB | 30 May 2021 - 03:53 |
| twitterbot | 21 | 2.98 GB | 28 May 2021 - 11:36 |
| WordPress | 15 | 311.67 MB | 16 May 2021 - 18:42 |
| yeti | 13 | 44.51 KB | 29 May 2021 - 20:48 |
| Sogou Spider | 13 | 42.79 KB | 28 May 2021 - 19:35 |
| Anthill | 13 | 5.21 MB | 29 May 2021 - 16:41 |
| ning | 11 | 24.11 KB | 27 May 2021 - 23:33 |
| 360spider | 11 | 17.15 MB | 28 May 2021 - 22:38 |
| Unknown robot (identified by 'discovery') | 9 | 29.77 KB | 17 May 2021 - 07:41 |
| GigablastOpenSource, an Open Source Search Engine(Wiki) | 9 | 1.26 GB | 17 May 2021 - 11:06 |
| CFNetwork | 4 | 421.08 MB | 29 May 2021 - 11:00 |
| Motor | 3 | 1.79 MB | 25 May 2021 - 07:11 |
| Simmany Robot Ver1.0 | 3 | 12.83 KB | 05 May 2021 - 19:39 |
| Powermarks | 2 | 6.45 KB | 04 May 2021 - 21:42 |
| LinkBot | 2 | 2.01 MB | 19 May 2021 - 09:29 |
| Unknown robot (identified by a space or one of the characters _+:,.;/\- followed by 'bot') | 2 | 11.93 KB | 03 May 2021 - 09:51 |
| Nutch | 2 | 6.57 KB | 30 May 2021 - 07:32 |
| HTTrack off-line browser | 1 | 3.47 KB | 29 May 2021 - 12:00 |
| InternetArchive | 1 | 14.60 KB | 10 May 2021 - 01:48 |
| InfoBot | 1 | 5.17 KB | 06 May 2021 - 22:18 |
| GigaBot | 1 | 8.26 KB | 04 May 2021 - 04:04 |
|