| 61 different robots | Hits | Bandwidth | Last visit |
|---|
| WGet tools | 5,706,619 | 36919.21 GB | 30 Nov 2020 - 23:59 |
| Common *nix tool for automating web document retireval. Most likely a bot. | 93,160 | 209.62 GB | 30 Nov 2020 - 23:59 |
| SemrushBot | 28,222 | 210.41 GB | 30 Nov 2020 - 23:58 |
| Googlebot | 7,224 | 42.65 GB | 30 Nov 2020 - 23:58 |
| Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 7,928 | 141.78 GB | 30 Nov 2020 - 23:57 |
| Bingbot | 32,354 | 233.10 GB | 30 Nov 2020 - 23:57 |
| AhrefsBot | 9,662 | 172.04 GB | 30 Nov 2020 - 23:54 |
| BaiDuSpider | 169 | 16.11 GB | 30 Nov 2020 - 23:54 |
| check_http (nagios) | 8,406 | 27.16 MB | 30 Nov 2020 - 23:52 |
| Python-urllib | 6,417 | 24.27 GB | 30 Nov 2020 - 23:50 |
| Yandex Bot | 8,958 | 216.62 GB | 30 Nov 2020 - 23:48 |
| Unknown robot (identified by empty user agent string) | 14,106 | 73.15 GB | 30 Nov 2020 - 23:35 |
| DotBot, Open Site Explorer | 2,931 | 29.36 GB | 30 Nov 2020 - 23:27 |
| MJ12bot | 1,051 | 14.61 GB | 30 Nov 2020 - 23:26 |
| Googlebot-Image | 46 | 264.87 KB | 30 Nov 2020 - 22:38 |
| MagpieRSS | 914 | 180.34 GB | 30 Nov 2020 - 22:11 |
| Suspected bot masqurading as Mozilla | 96 | 0 | 30 Nov 2020 - 21:55 |
| Unknown robot (identified by 'spider') | 1,949 | 1.52 GB | 30 Nov 2020 - 19:53 |
| FaceBook bot | 90 | 7.00 GB | 30 Nov 2020 - 18:57 |
| LinkChecker | 122 | 0 | 30 Nov 2020 - 17:49 |
| FavIconizer | 224 | 780.00 KB | 30 Nov 2020 - 17:22 |
| SeznamBot | 143 | 3.26 GB | 30 Nov 2020 - 17:10 |
| Anthill | 11 | 36.22 KB | 30 Nov 2020 - 15:33 |
| MojeekBot | 56 | 331.39 KB | 30 Nov 2020 - 13:25 |
| Sift | 40 | 16.69 MB | 30 Nov 2020 - 12:57 |
| Unknown robot (identified by 'crawl') | 112,876 | 708.20 GB | 30 Nov 2020 - 12:10 |
| Suspected bot masqurading as Mozilla | 34 | 17.48 MB | 30 Nov 2020 - 12:07 |
| twitterbot | 10 | 1.10 GB | 30 Nov 2020 - 08:43 |
| Mail.ru bot | 187 | 1.13 GB | 30 Nov 2020 - 08:29 |
| Perl tool | 618 | 6.22 GB | 30 Nov 2020 - 08:10 |
| HTMLParser | 2 | 6.45 KB | 29 Nov 2020 - 20:43 |
| Java (Often spam bot) | 768 | 5.48 GB | 29 Nov 2020 - 19:00 |
| Cfetch | 16 | 1.95 GB | 29 Nov 2020 - 16:02 |
| 360spider | 30 | 22.46 MB | 29 Nov 2020 - 13:03 |
| Bing Preview bot | 87 | 7.52 GB | 29 Nov 2020 - 12:43 |
| ltx71 | 255 | 49.37 GB | 28 Nov 2020 - 21:46 |
| YaCy | 171 | 6.13 GB | 28 Nov 2020 - 16:15 |
| Motor | 41 | 911.10 MB | 28 Nov 2020 - 13:46 |
| yeti | 11 | 26.50 KB | 28 Nov 2020 - 09:28 |
| Sogou Spider | 15 | 664.11 KB | 28 Nov 2020 - 09:19 |
| Common Crawl | 201 | 3.48 GB | 28 Nov 2020 - 06:08 |
| ning | 47 | 414.16 MB | 27 Nov 2020 - 23:46 |
| Yahoo Slurp | 1 | 8.34 MB | 27 Nov 2020 - 22:05 |
| Steeler | 2 | 8.17 MB | 27 Nov 2020 - 15:59 |
| Unknown robot (identified by a space or one of the characters _+:,.;/\- followed by 'bot') | 27 | 4.21 GB | 27 Nov 2020 - 13:45 |
| Netcraft | 25 | 82.73 KB | 27 Nov 2020 - 07:04 |
| The Tweeted Times | 1 | 8.26 KB | 27 Nov 2020 - 02:37 |
| Alexa (IA Archiver) | 145 | 1016.77 MB | 27 Nov 2020 - 01:23 |
| Custo | 2 | 6.57 KB | 26 Nov 2020 - 13:43 |
| HTTrack off-line browser | 5 | 4.00 KB | 25 Nov 2020 - 17:28 |
| Web Core / Roots | 31 | 224.98 MB | 24 Nov 2020 - 10:41 |
| MSIECrawler | 3 | 10.05 KB | 24 Nov 2020 - 08:11 |
| BLEXBot, seems to belong to the WebMeUp backlink tool | 1,312 | 151.22 MB | 23 Nov 2020 - 06:34 |
| Google AdSense | 2 | 1.54 MB | 22 Nov 2020 - 00:16 |
| SMTBot | 43 | 19.32 MB | 21 Nov 2020 - 11:59 |
| SEOkicks Webcrawler | 708 | 16.94 GB | 19 Nov 2020 - 05:59 |
| Nutch | 1 | 3.48 KB | 14 Nov 2020 - 14:44 |
| Phantom | 13 | 32.72 KB | 13 Nov 2020 - 22:41 |
| Unknown robot (identified by 'robot') | 1 | 3.35 KB | 12 Nov 2020 - 08:14 |
| MSNBot | 2 | 31.67 MB | 10 Nov 2020 - 17:38 |
| CMS Crawler | 1 | 3.35 KB | 07 Nov 2020 - 17:35 |
|