68 different robots | Hits | Bandwidth | Last visit |
WGet tools | 5,946,280 | 29807.97 GB | 30 Sep 2020 - 23:59 |
Common *nix tool for automating web document retireval. Most likely a bot. | 141,648 | 253.68 GB | 30 Sep 2020 - 23:59 |
SemrushBot | 59,760 | 271.64 GB | 30 Sep 2020 - 23:59 |
Bingbot | 29,315 | 165.99 GB | 30 Sep 2020 - 23:59 |
Unknown robot (identified by empty user agent string) | 28,265 | 354.53 GB | 30 Sep 2020 - 23:35 |
AhrefsBot | 13,250 | 292.48 GB | 30 Sep 2020 - 23:45 |
check_http (nagios) | 8,499 | 27.46 MB | 30 Sep 2020 - 23:55 |
Unknown robot (identified by 'crawl') | 7,937 | 120.26 GB | 30 Sep 2020 - 23:59 |
Python-urllib | 6,325 | 10.87 GB | 30 Sep 2020 - 23:50 |
Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 4,673 | 35.51 GB | 30 Sep 2020 - 23:56 |
Googlebot | 4,597 | 28.36 GB | 30 Sep 2020 - 23:59 |
Yandex Bot | 3,342 | 62.44 GB | 30 Sep 2020 - 22:18 |
DotBot, Open Site Explorer | 2,943 | 19.34 GB | 30 Sep 2020 - 22:57 |
Java (Often spam bot) | 1,156 | 12.72 GB | 30 Sep 2020 - 22:56 |
Alexa (IA Archiver) | 1,022 | 36.45 GB | 25 Sep 2020 - 23:58 |
MagpieRSS | 815 | 162.15 GB | 30 Sep 2020 - 18:01 |
MJ12bot | 740 | 6.86 GB | 30 Sep 2020 - 22:08 |
SEOkicks Webcrawler | 564 | 20.74 GB | 27 Sep 2020 - 20:23 |
Common Crawl | 515 | 558.84 MB | 26 Sep 2020 - 23:03 |
Perl tool | 453 | 4.80 GB | 30 Sep 2020 - 15:29 |
Web Core / Roots | 423 | 8.15 GB | 30 Sep 2020 - 02:16 |
Unknown robot (identified by 'spider') | 338 | 13.16 MB | 30 Sep 2020 - 13:51 |
FavIconizer | 286 | 1.01 MB | 30 Sep 2020 - 19:29 |
ltx71 | 258 | 47.74 GB | 30 Sep 2020 - 19:32 |
360spider | 200 | 185.01 MB | 29 Sep 2020 - 00:19 |
BaiDuSpider | 198 | 12.06 GB | 30 Sep 2020 - 12:30 |
Mail.ru bot | 197 | 33.41 MB | 29 Sep 2020 - 12:41 |
Suspected bot masqurading as Mozilla | 195 | 4.41 GB | 30 Sep 2020 - 20:35 |
Suspected bot masqurading as Mozilla | 150 | 138.29 KB | 30 Sep 2020 - 06:57 |
SeznamBot | 143 | 4.04 GB | 30 Sep 2020 - 19:43 |
LinkChecker | 122 | 0 | 30 Sep 2020 - 11:47 |
BLEXBot, seems to belong to the WebMeUp backlink tool | 95 | 68.46 MB | 20 Sep 2020 - 16:13 |
FaceBook bot | 80 | 1.04 GB | 30 Sep 2020 - 07:07 |
MojeekBot | 70 | 3.49 MB | 29 Sep 2020 - 20:21 |
Sogou Spider | 63 | 187.31 MB | 30 Sep 2020 - 10:12 |
YaCy | 61 | 2.76 GB | 30 Sep 2020 - 12:05 |
ning | 52 | 99.95 MB | 23 Sep 2020 - 11:09 |
SMTBot | 45 | 62.52 MB | 12 Sep 2020 - 05:47 |
Unknown robot (identified by a space or one of the characters _+:,.;/\- followed by 'bot') | 32 | 721.73 MB | 21 Sep 2020 - 23:04 |
Sift | 30 | 152.49 KB | 29 Sep 2020 - 04:29 |
Bing Preview bot | 25 | 920.44 MB | 28 Sep 2020 - 02:42 |
Netcraft | 22 | 72.68 KB | 25 Sep 2020 - 20:50 |
Googlebot-Image | 15 | 26.65 KB | 30 Sep 2020 - 21:39 |
Cfetch | 14 | 2.90 GB | 30 Sep 2020 - 14:29 |
twitterbot | 12 | 1.78 GB | 29 Sep 2020 - 22:58 |
MSIECrawler | 12 | 39.82 KB | 10 Sep 2020 - 13:17 |
Anthill | 12 | 74.78 KB | 29 Sep 2020 - 09:08 |
CFNetwork | 10 | 638.92 MB | 18 Sep 2020 - 20:36 |
Unknown robot (identified by 'robot') | 7 | 15.47 MB | 23 Sep 2020 - 05:51 |
Jakarta commons-httpclient | 7 | 107.56 MB | 07 Sep 2020 - 09:59 |
WordPress | 7 | 634.02 KB | 30 Sep 2020 - 01:58 |
Powermarks | 6 | 19.85 KB | 13 Sep 2020 - 21:31 |
VSE | 5 | 346.31 MB | 01 Sep 2020 - 10:41 |
yeti | 4 | 7.86 KB | 26 Sep 2020 - 21:23 |
HTTrack off-line browser | 4 | 14.45 KB | 25 Sep 2020 - 19:43 |
Grapeshot Crawler | 3 | 9.92 KB | 04 Sep 2020 - 23:09 |
archive.org bot | 3 | 8.24 KB | 27 Sep 2020 - 23:45 |
GigablastOpenSource, an Open Source Search Engine(Wiki) | 3 | 139.42 MB | 12 Sep 2020 - 07:44 |
Unknown robot (identified by 'discovery') | 2 | 6.70 KB | 29 Sep 2020 - 21:27 |
Fish search | 2 | 636.96 MB | 05 Sep 2020 - 18:52 |
Motor | 2 | 188.22 MB | 24 Sep 2020 - 16:02 |
Google AdSense | 1 | 3.35 KB | 04 Sep 2020 - 23:02 |
Ruby script | 1 | 2.95 MB | 09 Sep 2020 - 07:32 |
Monster | 1 | 4.48 KB | 26 Sep 2020 - 11:15 |
Custo | 1 | 208.96 MB | 04 Sep 2020 - 22:27 |
Unknown robot (identified by 'scanner') | 1 | 18.04 KB | 18 Sep 2020 - 16:42 |
Nutch | 1 | 3.35 KB | 10 Sep 2020 - 12:37 |
InternetArchive | 1 | 0 | 09 Sep 2020 - 04:38 |
|