| 66 different robots | Hits | Bandwidth | Last visit |
|---|
| WGet tools | 2,841,777 | 14429.51 GB | 31 Jul 2020 - 23:59 |
| SemrushBot | 84,040 | 346.01 GB | 31 Jul 2020 - 23:59 |
| Common *nix tool for automating web document retireval. Most likely a bot. | 55,450 | 169.08 GB | 31 Jul 2020 - 23:58 |
| Unknown robot (identified by 'crawl') | 51,640 | 350.22 GB | 31 Jul 2020 - 09:18 |
| Bingbot | 45,058 | 478.33 GB | 31 Jul 2020 - 23:59 |
| Unknown robot (identified by empty user agent string) | 27,366 | 585.74 GB | 31 Jul 2020 - 23:55 |
| AhrefsBot | 20,753 | 790.06 GB | 31 Jul 2020 - 23:37 |
| DotBot, Open Site Explorer | 13,941 | 147.31 GB | 31 Jul 2020 - 23:56 |
| Python-urllib | 10,187 | 11.79 GB | 31 Jul 2020 - 23:50 |
| check_http (nagios) | 8,876 | 28.67 MB | 31 Jul 2020 - 23:49 |
| BLEXBot, seems to belong to the WebMeUp backlink tool | 6,808 | 8.19 GB | 31 Jul 2020 - 05:45 |
| Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 3,804 | 248.80 GB | 31 Jul 2020 - 23:54 |
| Googlebot | 3,735 | 56.41 GB | 31 Jul 2020 - 23:40 |
| Yandex Bot | 3,123 | 59.40 GB | 31 Jul 2020 - 23:55 |
| Java (Often spam bot) | 2,347 | 41.14 GB | 31 Jul 2020 - 18:32 |
| Unknown robot (identified by 'spider') | 1,411 | 25.80 GB | 31 Jul 2020 - 21:38 |
| Alexa (IA Archiver) | 846 | 35.10 GB | 31 Jul 2020 - 17:33 |
| MagpieRSS | 812 | 156.28 GB | 31 Jul 2020 - 23:07 |
| BaiDuSpider | 494 | 47.65 GB | 31 Jul 2020 - 19:35 |
| Common Crawl | 399 | 27.06 MB | 14 Jul 2020 - 05:08 |
| FavIconizer | 306 | 21.82 MB | 31 Jul 2020 - 16:30 |
| Perl tool | 269 | 6.47 GB | 31 Jul 2020 - 07:10 |
| Suspected bot masqurading as Mozilla | 239 | 10.11 GB | 31 Jul 2020 - 17:16 |
| MJ12bot | 220 | 11.98 GB | 31 Jul 2020 - 18:54 |
| SEOkicks Webcrawler | 179 | 10.27 GB | 26 Jul 2020 - 10:09 |
| SeznamBot | 154 | 3.23 GB | 31 Jul 2020 - 20:09 |
| 360spider | 149 | 489.15 MB | 30 Jul 2020 - 00:00 |
| Web Core / Roots | 131 | 1.76 GB | 30 Jul 2020 - 08:47 |
| Mail.ru bot | 123 | 80.85 MB | 30 Jul 2020 - 22:19 |
| LinkChecker | 81 | 0 | 31 Jul 2020 - 23:02 |
| Sogou Spider | 70 | 187.00 MB | 31 Jul 2020 - 19:18 |
| Suspected bot masqurading as Mozilla | 62 | 19.40 KB | 31 Jul 2020 - 14:58 |
| Googlebot-Image | 57 | 4.12 MB | 31 Jul 2020 - 12:37 |
| Bing Preview bot | 43 | 2.20 GB | 28 Jul 2020 - 14:26 |
| ltx71 | 31 | 200.91 MB | 31 Jul 2020 - 14:23 |
| FaceBook bot | 30 | 1.77 GB | 30 Jul 2020 - 23:04 |
| Sift | 30 | 347.48 MB | 31 Jul 2020 - 12:03 |
| Netcraft | 23 | 23.20 KB | 28 Jul 2020 - 21:31 |
| ning | 22 | 2.87 GB | 20 Jul 2020 - 09:02 |
| archive.org bot | 21 | 6.67 GB | 31 Jul 2020 - 14:40 |
| YaCy | 20 | 897.14 MB | 29 Jul 2020 - 10:29 |
| Nutch | 20 | 1.72 GB | 03 Jul 2020 - 14:12 |
| Exabot | 19 | 15.43 MB | 17 Jul 2020 - 22:25 |
| MojeekBot | 16 | 267.95 KB | 31 Jul 2020 - 06:19 |
| Custo | 14 | 2.26 GB | 18 Jul 2020 - 00:32 |
| CFNetwork | 12 | 294.82 MB | 21 Jul 2020 - 02:15 |
| twitterbot | 11 | 2.23 GB | 31 Jul 2020 - 08:09 |
| Motor | 9 | 19.39 KB | 09 Jul 2020 - 03:30 |
| yeti | 7 | 15.69 KB | 27 Jul 2020 - 23:44 |
| WordPress | 6 | 694.78 MB | 09 Jul 2020 - 04:43 |
| Cliqzbot | 6 | 3.51 MB | 18 Jul 2020 - 15:09 |
| Powermarks | 6 | 858.15 KB | 29 Jul 2020 - 22:34 |
| Cfetch | 4 | 816.94 MB | 29 Jul 2020 - 20:44 |
| LinkBot | 4 | 2.01 MB | 16 Jul 2020 - 20:45 |
| HTTrack off-line browser | 3 | 16.07 KB | 29 Jul 2020 - 00:56 |
| Unknown robot (identified by 'scanner') | 2 | 80.80 KB | 31 Jul 2020 - 19:36 |
| A PHP script | 2 | 6.57 KB | 30 Jul 2020 - 06:47 |
| InfoBot | 2 | 8.78 KB | 22 Jul 2020 - 18:07 |
| CMS Crawler | 2 | 6.45 KB | 29 Jul 2020 - 23:38 |
| TechBOT | 2 | 6.70 KB | 11 Jul 2020 - 11:31 |
| Unknown robot (identified by 'discovery') | 1 | 3.35 KB | 20 Jul 2020 - 14:01 |
| Heritrix | 1 | 5.93 KB | 01 Jul 2020 - 04:08 |
| Ruby script | 1 | 3.22 KB | 23 Jul 2020 - 19:38 |
| VSE | 1 | 3.35 KB | 18 Jul 2020 - 02:16 |
| Jakarta commons-httpclient | 1 | 2.28 GB | 27 Jul 2020 - 19:12 |
| Emacs-w3 Search Engine | 1 | 423.00 MB | 21 Jul 2020 - 16:46 |
|