59 different robots | Hits | Bandwidth | Last visit |
WGet tools | 4,046,989 | 7979.82 GB | 30 Nov 2021 - 23:59 |
Unknown robot (identified by 'crawl') | 63,773 | 276.55 GB | 30 Nov 2021 - 22:19 |
Bingbot | 44,858 | 578.94 GB | 30 Nov 2021 - 23:53 |
Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 28,539 | 202.57 GB | 30 Nov 2021 - 23:50 |
SemrushBot | 25,855 | 87.94 GB | 30 Nov 2021 - 23:59 |
Common *nix tool for automating web document retireval. Most likely a bot. | 12,199 | 91.24 GB | 30 Nov 2021 - 23:37 |
AhrefsBot | 10,677 | 164.82 GB | 30 Nov 2021 - 22:56 |
check_http (nagios) | 8,199 | 26.49 MB | 30 Nov 2021 - 23:59 |
Unknown robot (identified by empty user agent string) | 6,460 | 96.13 GB | 30 Nov 2021 - 23:36 |
Nutch | 5,816 | 18.34 GB | 23 Nov 2021 - 20:59 |
Googlebot | 5,455 | 79.12 GB | 30 Nov 2021 - 23:49 |
DotBot, Open Site Explorer | 4,462 | 35.40 GB | 30 Nov 2021 - 23:45 |
Python-urllib | 2,968 | 146.17 GB | 30 Nov 2021 - 23:30 |
Unknown robot (identified by 'spider') | 2,143 | 32.97 GB | 30 Nov 2021 - 23:47 |
Yandex Bot | 1,997 | 48.84 GB | 30 Nov 2021 - 23:38 |
Common Crawl | 1,217 | 12.41 GB | 30 Nov 2021 - 13:17 |
Suspected bot masqurading as Mozilla | 715 | 1.30 GB | 29 Nov 2021 - 15:36 |
Java (Often spam bot) | 679 | 2.87 GB | 29 Nov 2021 - 01:12 |
MJ12bot | 534 | 21.28 GB | 30 Nov 2021 - 15:26 |
YaCy | 373 | 7.40 GB | 30 Nov 2021 - 10:35 |
Suspected bot masqurading as Mozilla | 367 | 138.87 MB | 30 Nov 2021 - 22:19 |
SEOkicks Webcrawler | 313 | 6.19 GB | 02 Nov 2021 - 16:50 |
ltx71 | 308 | 25.84 GB | 30 Nov 2021 - 21:28 |
BaiDuSpider | 264 | 18.01 GB | 30 Nov 2021 - 15:47 |
Bing Preview bot | 208 | 40.37 GB | 30 Nov 2021 - 21:31 |
FavIconizer | 197 | 689.67 KB | 30 Nov 2021 - 23:01 |
FaceBook bot | 192 | 33.18 GB | 30 Nov 2021 - 12:26 |
HTTrack off-line browser | 184 | 1.25 GB | 27 Nov 2021 - 14:19 |
Mail.ru bot | 183 | 48.33 MB | 28 Nov 2021 - 07:08 |
Yahoo Slurp | 145 | 23.13 GB | 24 Nov 2021 - 19:26 |
Custo | 142 | 468.59 KB | 30 Nov 2021 - 20:38 |
LinkChecker | 103 | 203.82 MB | 30 Nov 2021 - 16:20 |
SeznamBot | 98 | 24.52 MB | 30 Nov 2021 - 22:04 |
Web Core / Roots | 74 | 3.75 GB | 30 Nov 2021 - 22:57 |
Alexa (IA Archiver) | 73 | 19.35 MB | 28 Nov 2021 - 12:46 |
MojeekBot | 73 | 15.55 MB | 30 Nov 2021 - 23:29 |
archive.org bot | 49 | 3.15 GB | 30 Nov 2021 - 09:27 |
Perl tool | 43 | 4.48 GB | 30 Nov 2021 - 22:50 |
Googlebot-Image | 35 | 324.44 KB | 30 Nov 2021 - 17:24 |
ning | 26 | 436.15 MB | 12 Nov 2021 - 12:20 |
Google Web Preview | 21 | 71.59 KB | 30 Nov 2021 - 15:21 |
Unknown robot (identified by a space or one of the characters _+:,.;/\- followed by 'bot') | 20 | 198.85 MB | 25 Nov 2021 - 17:13 |
BLEXBot, seems to belong to the WebMeUp backlink tool | 17 | 23.64 MB | 30 Nov 2021 - 21:04 |
twitterbot | 16 | 2.37 GB | 30 Nov 2021 - 10:29 |
TITAN | 13 | 32.88 KB | 08 Nov 2021 - 02:39 |
Unknown robot (identified by 'robot') | 9 | 618.04 MB | 20 Nov 2021 - 18:32 |
Sogou Spider | 8 | 26.54 KB | 26 Nov 2021 - 19:34 |
Netcraft | 4 | 13.27 KB | 21 Nov 2021 - 03:40 |
LinkBot | 4 | 31.60 MB | 27 Nov 2021 - 15:40 |
yeti | 4 | 49.98 KB | 30 Nov 2021 - 00:04 |
Powermarks | 3 | 10.05 KB | 27 Nov 2021 - 14:37 |
Simmany Robot Ver1.0 | 3 | 12.57 KB | 13 Nov 2021 - 14:24 |
Motor | 2 | 3.46 MB | 06 Nov 2021 - 05:29 |
CFNetwork | 2 | 872.00 MB | 12 Nov 2021 - 11:06 |
WordPress | 2 | 5.55 KB | 30 Nov 2021 - 07:15 |
InfoBot | 2 | 8.65 KB | 10 Nov 2021 - 04:25 |
A PHP script | 1 | 6.11 MB | 07 Nov 2021 - 09:19 |
Fish search | 1 | 471.44 MB | 05 Nov 2021 - 22:48 |
JBot Java Web Robot | 1 | 0 | 25 Nov 2021 - 21:18 |
|