74 different robots | Hits | Bandwidth | Last visit |
WGet tools | 7,228,279 | 30188.02 GB | 30 Jun 2020 - 23:59 |
Unknown robot (identified by 'crawl') | 506,872 | 2657.88 GB | 30 Jun 2020 - 23:59 |
DotBot, Open Site Explorer | 14,455 | 180.91 GB | 30 Jun 2020 - 23:59 |
SemrushBot | 73,599 | 279.89 GB | 30 Jun 2020 - 23:59 |
Yandex Bot | 2,765 | 76.51 GB | 30 Jun 2020 - 23:59 |
Bingbot | 13,180 | 181.19 GB | 30 Jun 2020 - 23:59 |
Common *nix tool for automating web document retireval. Most likely a bot. | 124,000 | 313.05 GB | 30 Jun 2020 - 23:59 |
Unknown robot (identified by empty user agent string) | 28,774 | 534.81 GB | 30 Jun 2020 - 23:54 |
Python-urllib | 15,926 | 28.41 GB | 30 Jun 2020 - 23:52 |
Googlebot | 4,514 | 74.81 GB | 30 Jun 2020 - 23:51 |
check_http (nagios) | 8,570 | 27.68 MB | 30 Jun 2020 - 23:49 |
BaiDuSpider | 1,122 | 87.08 GB | 30 Jun 2020 - 23:44 |
AhrefsBot | 18,345 | 843.34 GB | 30 Jun 2020 - 23:42 |
Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 3,845 | 30.25 GB | 30 Jun 2020 - 23:40 |
Web Core / Roots | 169 | 3.52 GB | 30 Jun 2020 - 23:25 |
Netcraft | 18 | 32.99 KB | 30 Jun 2020 - 23:09 |
Sogou Spider | 207 | 1.11 GB | 30 Jun 2020 - 22:51 |
Suspected bot masqurading as Mozilla | 111 | 50.44 KB | 30 Jun 2020 - 22:39 |
FavIconizer | 375 | 1.22 MB | 30 Jun 2020 - 22:35 |
VSE | 2 | 6.57 KB | 30 Jun 2020 - 22:16 |
Suspected bot masqurading as Mozilla | 410 | 9.94 GB | 30 Jun 2020 - 21:54 |
LinkChecker | 112 | 0 | 30 Jun 2020 - 21:53 |
MJ12bot | 481 | 13.09 GB | 30 Jun 2020 - 21:49 |
Java (Often spam bot) | 1,910 | 38.38 GB | 30 Jun 2020 - 21:33 |
Perl tool | 375 | 3.80 GB | 30 Jun 2020 - 20:05 |
Common Crawl | 18 | 1.46 MB | 30 Jun 2020 - 17:03 |
Unknown robot (identified by 'spider') | 735 | 12.75 GB | 30 Jun 2020 - 16:56 |
SeznamBot | 132 | 1.79 GB | 30 Jun 2020 - 14:39 |
BLEXBot, seems to belong to the WebMeUp backlink tool | 1,685 | 3.89 GB | 30 Jun 2020 - 11:34 |
MagpieRSS | 753 | 146.77 GB | 30 Jun 2020 - 11:27 |
Cliqzbot | 17 | 10.20 MB | 30 Jun 2020 - 11:01 |
Alexa (IA Archiver) | 118 | 1.47 GB | 30 Jun 2020 - 10:32 |
MojeekBot | 57 | 3.55 MB | 30 Jun 2020 - 09:36 |
YaCy | 25 | 2.00 GB | 30 Jun 2020 - 09:21 |
ltx71 | 11 | 55.39 KB | 30 Jun 2020 - 00:00 |
360spider | 130 | 562.83 MB | 29 Jun 2020 - 23:52 |
Heritrix | 4 | 18.44 KB | 29 Jun 2020 - 21:57 |
Bing Preview bot | 280 | 19.27 GB | 29 Jun 2020 - 20:15 |
Mail.ru bot | 297 | 436.50 MB | 29 Jun 2020 - 19:51 |
Googlebot-Image | 84 | 389.95 KB | 29 Jun 2020 - 18:50 |
Sift | 57 | 222.29 KB | 29 Jun 2020 - 17:02 |
Jakarta commons-httpclient | 16 | 255.12 KB | 29 Jun 2020 - 09:28 |
Unknown robot (identified by 'robot') | 3 | 17.27 KB | 29 Jun 2020 - 08:17 |
twitterbot | 21 | 5.35 GB | 27 Jun 2020 - 09:32 |
A PHP script | 3 | 9.80 KB | 27 Jun 2020 - 08:31 |
Emacs-w3 Search Engine | 2 | 732.54 MB | 27 Jun 2020 - 01:45 |
Cfetch | 2 | 210.83 MB | 27 Jun 2020 - 00:17 |
FaceBook bot | 97 | 590.95 MB | 26 Jun 2020 - 22:00 |
yeti | 2 | 5.55 KB | 25 Jun 2020 - 20:58 |
CFNetwork | 9 | 293.18 MB | 25 Jun 2020 - 07:13 |
WebCopy | 2 | 4.43 KB | 24 Jun 2020 - 22:36 |
Google AdSense | 2 | 5.61 GB | 24 Jun 2020 - 20:07 |
InfoBot | 3 | 14.21 KB | 23 Jun 2020 - 23:29 |
MSIECrawler | 3 | 429.27 KB | 23 Jun 2020 - 11:42 |
Yahoo Slurp | 1 | 11.77 MB | 23 Jun 2020 - 11:19 |
ning | 65 | 285.48 MB | 23 Jun 2020 - 01:08 |
Fish search | 1 | 3.35 KB | 22 Jun 2020 - 20:02 |
Custo | 12 | 1.44 GB | 22 Jun 2020 - 12:49 |
lwp-request | 2 | 0 | 21 Jun 2020 - 19:37 |
Phantom | 25 | 68.60 KB | 21 Jun 2020 - 17:06 |
rpt-httpclient | 1 | 27.47 KB | 20 Jun 2020 - 18:00 |
TechBOT | 3 | 9.92 KB | 20 Jun 2020 - 12:15 |
SEOkicks Webcrawler | 186,262 | 1502.39 GB | 19 Jun 2020 - 16:11 |
Xenu's Link Sleuth (TM), see Wikipedia | 3 | 6.57 KB | 15 Jun 2020 - 12:41 |
StackRambler | 3 | 15.89 MB | 15 Jun 2020 - 10:17 |
SMTBot | 46 | 100.89 MB | 15 Jun 2020 - 03:25 |
Nutch | 2 | 6.70 KB | 14 Jun 2020 - 20:23 |
InternetArchive | 1 | 0 | 13 Jun 2020 - 04:29 |
Unknown robot (identified by 'scanner') | 1 | 18.04 KB | 12 Jun 2020 - 01:32 |
ichiro | 8 | 712.32 MB | 11 Jun 2020 - 00:50 |
HTTrack off-line browser | 1 | 7.13 KB | 07 Jun 2020 - 04:03 |
archive.org bot | 1 | 4.70 KB | 06 Jun 2020 - 23:14 |
NG 1.x (Exalead) | 2 | 0 | 06 Jun 2020 - 20:52 |
Sven | 13 | 185.85 MB | 05 Jun 2020 - 04:20 |
|