56 different robots | Hits | Bandwidth | Last visit |
WGet tools | 2,975,414 | 11547.36 GB | 28 Feb 2023 - 23:59 |
SemrushBot | 134,007 | 56.77 GB | 28 Feb 2023 - 23:59 |
Unknown robot (identified by 'crawl') | 90,031 | 504.09 GB | 28 Feb 2023 - 09:19 |
Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 26,229 | 297.91 GB | 28 Feb 2023 - 23:53 |
Bingbot | 24,736 | 876.27 GB | 28 Feb 2023 - 23:50 |
AhrefsBot | 18,270 | 33.11 GB | 28 Feb 2023 - 23:13 |
Googlebot | 13,405 | 248.10 GB | 28 Feb 2023 - 23:46 |
Common *nix tool for automating web document retireval. Most likely a bot. | 11,846 | 162.58 GB | 28 Feb 2023 - 23:24 |
check_http (nagios) | 7,976 | 27.86 MB | 28 Feb 2023 - 23:59 |
DotBot, Open Site Explorer | 7,774 | 117.50 GB | 28 Feb 2023 - 22:05 |
Yandex Bot | 7,577 | 306.81 GB | 28 Feb 2023 - 23:56 |
Unknown robot (identified by empty user agent string) | 6,510 | 89.19 GB | 28 Feb 2023 - 23:35 |
Python-urllib | 3,399 | 20.98 GB | 28 Feb 2023 - 23:31 |
Common Crawl | 1,900 | 3.85 GB | 09 Feb 2023 - 10:18 |
MJ12bot | 1,677 | 37.08 GB | 28 Feb 2023 - 15:23 |
Unknown robot (identified by 'spider') | 1,565 | 62.85 GB | 28 Feb 2023 - 23:27 |
Suspected bot masqurading as Mozilla | 971 | 978.22 MB | 28 Feb 2023 - 23:21 |
BLEXBot, seems to belong to the WebMeUp backlink tool | 856 | 197.89 MB | 26 Feb 2023 - 12:27 |
SEOkicks Webcrawler | 375 | 3.56 GB | 26 Feb 2023 - 17:10 |
Java (Often spam bot) | 368 | 10.85 KB | 28 Feb 2023 - 06:01 |
Googlebot-Image | 314 | 98.59 MB | 28 Feb 2023 - 11:13 |
BaiDuSpider | 292 | 23.58 GB | 27 Feb 2023 - 01:31 |
Suspected bot masqurading as Mozilla | 263 | 2.99 GB | 28 Feb 2023 - 19:40 |
Mail.ru bot | 189 | 117.87 MB | 28 Feb 2023 - 21:09 |
Perl tool | 162 | 27.21 GB | 28 Feb 2023 - 12:00 |
YaCy | 160 | 10.69 GB | 23 Feb 2023 - 02:59 |
FaceBook bot | 114 | 12.72 GB | 28 Feb 2023 - 01:29 |
Custo | 105 | 378.16 KB | 28 Feb 2023 - 22:06 |
Feedburner | 103 | 0 | 28 Feb 2023 - 21:18 |
MojeekBot | 100 | 46.33 MB | 28 Feb 2023 - 20:57 |
SeznamBot | 95 | 7.38 MB | 28 Feb 2023 - 22:21 |
FavIconizer | 93 | 332.74 KB | 28 Feb 2023 - 22:24 |
LinkChecker | 89 | 323.34 MB | 28 Feb 2023 - 06:10 |
Unknown robot (identified by a space or one of the characters _+:,.;/\- followed by 'bot') | 41 | 362.15 MB | 25 Feb 2023 - 13:30 |
yeti | 40 | 159.70 KB | 28 Feb 2023 - 13:07 |
GigablastOpenSource, an Open Source Search Engine(Wiki) | 34 | 7.73 KB | 14 Feb 2023 - 03:26 |
Web Core / Roots | 31 | 16.29 GB | 27 Feb 2023 - 17:16 |
Bing Preview bot | 15 | 326.90 MB | 28 Feb 2023 - 17:57 |
twitterbot | 12 | 9.70 GB | 25 Feb 2023 - 20:52 |
Sogou Spider | 9 | 32.56 KB | 26 Feb 2023 - 19:07 |
JBot Java Web Robot | 7 | 498.62 MB | 19 Feb 2023 - 01:43 |
Netcraft | 7 | 21.33 KB | 28 Feb 2023 - 20:54 |
WordPress | 5 | 85.46 KB | 26 Feb 2023 - 19:39 |
Heritrix | 4 | 14.29 KB | 16 Feb 2023 - 07:32 |
Motor | 3 | 4.47 GB | 22 Feb 2023 - 22:17 |
WebCopy | 2 | 0 | 28 Feb 2023 - 15:20 |
LinkBot | 2 | 3.31 KB | 05 Feb 2023 - 02:16 |
archive.org bot | 1 | 3.62 KB | 09 Feb 2023 - 05:37 |
Emacs-w3 Search Engine | 1 | 387.98 MB | 26 Feb 2023 - 10:28 |
Nutch | 1 | 12.82 KB | 21 Feb 2023 - 12:44 |
fido | 1 | 3.49 KB | 25 Feb 2023 - 20:47 |
Ask | 1 | 3.49 KB | 06 Feb 2023 - 18:12 |
InternetArchive | 1 | 5.35 KB | 19 Feb 2023 - 20:18 |
RoadHouse Crawling System | 1 | 3.49 KB | 03 Feb 2023 - 17:12 |
MSNBot-media | 1 | 3.62 KB | 04 Feb 2023 - 12:10 |
linkedinbot | 1 | 3.62 KB | 13 Feb 2023 - 09:18 |
|