66 different robots | Hits | Bandwidth | Last visit |
Common *nix tool for automating web document retireval. Most likely a bot. | 1,003,725+1 | 4.21 GB | 31 Jan 2023 - 23:11 |
Python-urllib | 902,390+5 | 6.37 GB | 31 Jan 2023 - 23:11 |
MJ12bot | 81,866+1778 | 11.29 GB | 31 Jan 2023 - 22:17 |
WGet tools | 70,867+33 | 306.67 GB | 31 Jan 2023 - 23:08 |
AhrefsBot | 68,755+439 | 2.36 GB | 31 Jan 2023 - 23:59 |
Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 65,549+995 | 3.50 GB | 31 Jan 2023 - 23:10 |
BLEXBot, seems to belong to the WebMeUp backlink tool | 64,018+183 | 3.04 GB | 31 Jan 2023 - 23:59 |
Grabber (SDSC) | 51,302 | 56.51 MB | 31 Jan 2023 - 23:11 |
SemrushBot | 41,538+1708 | 675.46 MB | 31 Jan 2023 - 22:48 |
Bingbot | 38,762+239 | 1.56 GB | 31 Jan 2023 - 22:59 |
Googlebot | 23,486+197 | 3.34 GB | 31 Jan 2023 - 23:10 |
DotBot, Open Site Explorer | 19,025+1994 | 3.78 GB | 31 Jan 2023 - 23:07 |
Yandex Bot | 9,859+654 | 1.61 GB | 31 Jan 2023 - 23:08 |
Unknown robot (identified by empty user agent string) | 8,408+7 | 2.80 GB | 31 Jan 2023 - 22:20 |
FaceBook bot | 5,651 | 122.60 MB | 31 Jan 2023 - 20:05 |
Unknown robot (identified by 'crawl') | 3,847+58 | 130.41 MB | 31 Jan 2023 - 03:55 |
Unknown robot (identified by 'spider') | 1,785+119 | 258.73 MB | 31 Jan 2023 - 22:42 |
Unknown robot (identified by hit on 'robots.txt') | 0+1418 | 301.88 KB | 31 Jan 2023 - 22:48 |
Common Crawl | 1,261+17 | 46.10 MB | 31 Jan 2023 - 14:44 |
Perl tool | 741 | 28.15 MB | 31 Jan 2023 - 18:20 |
Web Core / Roots | 638 | 41.64 MB | 31 Jan 2023 - 07:22 |
WordPress | 614 | 30.77 MB | 30 Jan 2023 - 07:22 |
Unknown robot (identified by a space or one of the characters _+:,.;/\- followed by 'bot') | 537+8 | 7.38 MB | 31 Jan 2023 - 14:07 |
Yahoo Slurp | 540 | 10.04 MB | 25 Jan 2023 - 11:11 |
LinkChecker | 466+8 | 115.86 KB | 31 Jan 2023 - 17:07 |
MojeekBot | 231+222 | 3.64 MB | 31 Jan 2023 - 20:51 |
SeznamBot | 285+77 | 5.74 MB | 31 Jan 2023 - 21:22 |
Motor | 246 | 7.35 MB | 31 Jan 2023 - 12:04 |
YaCy | 179+65 | 4.11 MB | 31 Jan 2023 - 21:19 |
Sogou Spider | 175+4 | 3.78 MB | 31 Jan 2023 - 11:11 |
Mail.ru bot | 51+94 | 10.79 MB | 31 Jan 2023 - 22:31 |
Bing Preview bot | 113 | 5.29 MB | 29 Jan 2023 - 01:58 |
LinkBot | 55+28 | 511.18 KB | 31 Jan 2023 - 21:38 |
Google Web Preview | 82 | 2.58 MB | 29 Jan 2023 - 18:41 |
twitterbot | 55+12 | 1.74 MB | 31 Jan 2023 - 04:27 |
Heritrix | 45+2 | 756.62 KB | 26 Jan 2023 - 05:55 |
InfoBot | 0+28 | 5.96 KB | 30 Jan 2023 - 12:46 |
BaiDuSpider | 27 | 190.58 KB | 31 Jan 2023 - 00:06 |
ning | 26 | 669.32 KB | 29 Jan 2023 - 11:39 |
Unknown robot (identified by 'robot') | 25 | 1.09 MB | 27 Jan 2023 - 18:09 |
WebCopy | 25 | 33.95 KB | 29 Jan 2023 - 09:06 |
yeti | 9+16 | 325.44 KB | 31 Jan 2023 - 20:19 |
Googlebot-Image | 23 | 5.74 MB | 30 Jan 2023 - 21:06 |
Hometown Spider Pro | 15 | 254.62 KB | 09 Jan 2023 - 17:42 |
MagpieRSS | 0+12 | 2.55 KB | 29 Jan 2023 - 04:04 |
Suspected bot masqurading as Mozilla | 11 | 670.97 KB | 18 Jan 2023 - 04:43 |
Linkdex | 9+2 | 176.50 KB | 28 Jan 2023 - 17:11 |
CFNetwork | 10 | 143.68 KB | 28 Jan 2023 - 00:09 |
Y!J Yahoo Japan | 1+9 | 16.75 KB | 19 Jan 2023 - 10:59 |
Suspected bot masqurading as Mozilla | 9 | 22.08 MB | 30 Jan 2023 - 09:49 |
Java (Often spam bot) | 7+1 | 165.54 KB | 25 Jan 2023 - 05:02 |
Unknown robot (identified by 'checker') | 0+8 | 1.70 KB | 31 Jan 2023 - 01:51 |
Nutch | 3+4 | 51.78 KB | 23 Jan 2023 - 08:57 |
Custo | 7 | 873.33 KB | 12 Jan 2023 - 00:52 |
arks | 6 | 50.92 KB | 24 Jan 2023 - 15:21 |
FavIconizer | 5 | 76.81 KB | 31 Jan 2023 - 06:20 |
InternetArchive | 5 | 0 | 21 Jan 2023 - 08:36 |
Daum | 0+4 | 872 Bytes | 17 Jan 2023 - 14:39 |
Emacs-w3 Search Engine | 3 | 51.00 KB | 21 Jan 2023 - 11:44 |
HTTrack off-line browser | 3 | 10.91 KB | 28 Jan 2023 - 10:45 |
Monster | 2 | 16.97 KB | 09 Jan 2023 - 17:41 |
Unknown robot (identified by 'discovery') | 2 | 33.95 KB | 25 Jan 2023 - 14:41 |
Fish search | 2 | 16.97 KB | 24 Jan 2023 - 14:27 |
Microsoft URL Control | 1 | 15.29 KB | 26 Jan 2023 - 05:36 |
pita (pain in the ass?) | 1 | 16.97 KB | 09 Jan 2023 - 17:41 |
Raven Search | 1 | 16.97 KB | 09 Jan 2023 - 17:41 |
|