64 different robots | Hits | Bandwidth | Last visit |
Python-urllib | 809,930+1 | 6.02 GB | 31 Dec 2022 - 23:10 |
SemrushBot | 50,825+1920 | 1.24 GB | 31 Dec 2022 - 23:10 |
Common *nix tool for automating web document retireval. Most likely a bot. | 1,012,009+1 | 4.09 GB | 31 Dec 2022 - 23:10 |
Bingbot | 66,447+255 | 2.68 GB | 31 Dec 2022 - 23:10 |
DotBot, Open Site Explorer | 27,707+2119 | 9.72 GB | 31 Dec 2022 - 23:10 |
BLEXBot, seems to belong to the WebMeUp backlink tool | 73,492+369 | 2.79 GB | 31 Dec 2022 - 23:10 |
Grabber (SDSC) | 51,735 | 52.14 MB | 31 Dec 2022 - 23:09 |
AhrefsBot | 74,914+440 | 2.26 GB | 31 Dec 2022 - 23:08 |
Unknown robot (identified by empty user agent string) | 7,739+3 | 2.50 GB | 31 Dec 2022 - 23:06 |
SeznamBot | 375+86 | 8.73 MB | 31 Dec 2022 - 23:04 |
WGet tools | 104,393+40 | 317.28 GB | 31 Dec 2022 - 23:03 |
LinkChecker | 462+22 | 147.37 KB | 31 Dec 2022 - 23:03 |
Unknown robot (identified by 'spider') | 893+118 | 215.93 MB | 31 Dec 2022 - 23:02 |
Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 97,274+1163 | 5.92 GB | 31 Dec 2022 - 23:02 |
Googlebot | 20,127+206 | 3.64 GB | 31 Dec 2022 - 23:01 |
MJ12bot | 64,451+1863 | 13.01 GB | 31 Dec 2022 - 22:56 |
Yandex Bot | 4,108+496 | 580.54 MB | 31 Dec 2022 - 22:46 |
Unknown robot (identified by 'crawl') | 6,539+593 | 859.31 MB | 31 Dec 2022 - 21:51 |
MojeekBot | 101+102 | 1.94 MB | 31 Dec 2022 - 21:17 |
BaiDuSpider | 1,158 | 232.84 MB | 31 Dec 2022 - 18:49 |
WordPress | 267 | 12.99 MB | 31 Dec 2022 - 18:22 |
Perl tool | 756 | 26.20 MB | 31 Dec 2022 - 18:01 |
yeti | 26+39 | 70.33 MB | 31 Dec 2022 - 17:56 |
Unknown robot (identified by hit on 'robots.txt') | 0+2021 | 429.83 KB | 31 Dec 2022 - 17:51 |
Sogou Spider | 317+12 | 16.51 MB | 31 Dec 2022 - 16:13 |
Googlebot-Image | 27 | 923.23 KB | 31 Dec 2022 - 15:16 |
HTTrack off-line browser | 8 | 63.67 KB | 31 Dec 2022 - 15:14 |
FaceBook bot | 1,404 | 24.95 MB | 31 Dec 2022 - 15:04 |
LinkBot | 35+18 | 290.59 KB | 31 Dec 2022 - 12:54 |
twitterbot | 67+33 | 1.89 MB | 31 Dec 2022 - 12:32 |
Bing Preview bot | 285 | 11.33 MB | 31 Dec 2022 - 08:09 |
YaCy | 51+33 | 1.92 MB | 31 Dec 2022 - 06:15 |
Mail.ru bot | 2,484+359 | 41.80 MB | 31 Dec 2022 - 01:44 |
Web Core / Roots | 1,427 | 74.00 MB | 30 Dec 2022 - 20:29 |
Emacs-w3 Search Engine | 1 | 119.24 KB | 30 Dec 2022 - 20:26 |
Unknown robot (identified by a space or one of the characters _+:,.;/\- followed by 'bot') | 17+8 | 339.77 KB | 30 Dec 2022 - 14:07 |
WebCopy | 12 | 0 | 30 Dec 2022 - 09:40 |
InternetArchive | 4 | 0 | 30 Dec 2022 - 02:31 |
MagpieRSS | 0+10 | 2.13 KB | 29 Dec 2022 - 18:16 |
Suspected bot masqurading as Mozilla | 18 | 1.54 MB | 29 Dec 2022 - 13:10 |
ning | 5 | 0 | 29 Dec 2022 - 01:03 |
Linkdex | 7+2 | 196.22 KB | 28 Dec 2022 - 16:51 |
InfoBot | 0+34 | 7.24 KB | 28 Dec 2022 - 09:00 |
GetBot | 3 | 32.89 MB | 27 Dec 2022 - 13:25 |
Unknown robot (identified by 'robot') | 11+1 | 460.06 KB | 25 Dec 2022 - 13:30 |
Xenu's Link Sleuth (TM), see Wikipedia | 1 | 0 | 25 Dec 2022 - 11:49 |
Java (Often spam bot) | 1+5 | 19.12 KB | 24 Dec 2022 - 17:57 |
Motor | 134 | 3.94 MB | 23 Dec 2022 - 16:58 |
Y!J Yahoo Japan | 0+9 | 1.92 KB | 23 Dec 2022 - 12:34 |
Nutch | 1+1 | 17.19 KB | 23 Dec 2022 - 02:26 |
Daum | 0+9 | 1.92 KB | 22 Dec 2022 - 03:57 |
Unknown robot (identified by 'discovery') | 1 | 16.97 KB | 21 Dec 2022 - 07:20 |
FavIconizer | 4 | 76.79 KB | 20 Dec 2022 - 20:51 |
CFNetwork | 11 | 239.80 KB | 20 Dec 2022 - 01:00 |
Feedburner | 257 | 0 | 15 Dec 2022 - 10:33 |
SEOkicks Webcrawler | 8+2 | 213.86 KB | 14 Dec 2022 - 07:35 |
ASpider (Associative Spider) | 0+3 | 654 Bytes | 13 Dec 2022 - 16:10 |
Yahoo Slurp | 1 | 23.85 KB | 12 Dec 2022 - 05:25 |
archive.org bot | 0+2 | 436 Bytes | 11 Dec 2022 - 10:10 |
ips-agent Verisign(?) - no reliable information found. | 0+1 | 218 Bytes | 09 Dec 2022 - 20:17 |
Common Crawl | 3,327+24 | 159.48 MB | 09 Dec 2022 - 17:21 |
MS Search 6.0 Robot (MS SharePoint Portal Server?) | 2+2 | 32.81 KB | 09 Dec 2022 - 09:40 |
Google Web Preview | 5 | 49.25 KB | 05 Dec 2022 - 16:23 |
BBot | 2+2 | 144.63 KB | 04 Dec 2022 - 03:17 |
|