| 73 different robots | Hits | Bandwidth | Last visit |
|---|
| Common *nix tool for automating web document retireval. Most likely a bot. | 1,217,864+3 | 5.28 GB | 31 Aug 2022 - 23:12 |
| Python-urllib | 918,811+5 | 7.12 GB | 31 Aug 2022 - 23:11 |
| Bingbot | 136,104+309 | 3.81 GB | 31 Aug 2022 - 23:12 |
| Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) | 123,570+1107 | 6.36 GB | 31 Aug 2022 - 23:12 |
| AhrefsBot | 107,899+466 | 2.76 GB | 31 Aug 2022 - 23:01 |
| WGet tools | 82,115+37 | 327.54 GB | 31 Aug 2022 - 23:10 |
| SemrushBot | 75,643+2552 | 4.27 GB | 31 Aug 2022 - 23:12 |
| MJ12bot | 67,643+1779 | 8.81 GB | 31 Aug 2022 - 23:07 |
| BLEXBot, seems to belong to the WebMeUp backlink tool | 65,318+2387 | 2.95 GB | 29 Aug 2022 - 00:56 |
| Grabber (SDSC) | 55,411 | 58.75 MB | 31 Aug 2022 - 23:11 |
| Googlebot | 36,333+197 | 4.73 GB | 31 Aug 2022 - 23:10 |
| DotBot, Open Site Explorer | 15,007+2088 | 4.95 GB | 31 Aug 2022 - 23:11 |
| Unknown robot (identified by 'crawl') | 9,410+459 | 481.35 MB | 31 Aug 2022 - 22:40 |
| Yandex Bot | 6,501+528 | 790.04 MB | 31 Aug 2022 - 22:25 |
| Unknown robot (identified by 'robot') | 6,941+33 | 857.39 MB | 31 Aug 2022 - 22:55 |
| Unknown robot (identified by empty user agent string) | 5,308+14 | 2.27 GB | 31 Aug 2022 - 22:23 |
| Common Crawl | 4,024+44 | 228.41 MB | 20 Aug 2022 - 06:44 |
| Unknown robot (identified by 'spider') | 3,017+104 | 1002.10 MB | 31 Aug 2022 - 22:52 |
| Unknown robot (identified by hit on 'robots.txt') | 0+1501 | 318.70 KB | 31 Aug 2022 - 22:35 |
| BaiDuSpider | 1,271 | 58.89 MB | 31 Aug 2022 - 22:32 |
| SeznamBot | 369+839 | 7.95 MB | 31 Aug 2022 - 08:32 |
| Web Core / Roots | 1,032 | 66.67 MB | 31 Aug 2022 - 14:33 |
| Mail.ru bot | 602+309 | 69.61 MB | 31 Aug 2022 - 08:52 |
| Perl tool | 852 | 28.23 MB | 31 Aug 2022 - 22:29 |
| FaceBook bot | 692 | 142.20 MB | 31 Aug 2022 - 20:47 |
| MojeekBot | 352+317 | 4.41 MB | 31 Aug 2022 - 22:39 |
| Motor | 434 | 12.81 MB | 31 Aug 2022 - 20:58 |
| LinkChecker | 337+8 | 115.86 KB | 31 Aug 2022 - 13:07 |
| yeti | 272+61 | 295.08 MB | 30 Aug 2022 - 04:58 |
| SEOkicks Webcrawler | 277+4 | 3.66 MB | 24 Aug 2022 - 13:45 |
| WordPress | 244 | 11.69 MB | 31 Aug 2022 - 20:03 |
| Unknown robot (identified by a space or one of the characters _+:,.;/\- followed by 'bot') | 213+8 | 82.05 MB | 31 Aug 2022 - 18:53 |
| Sogou Spider | 205+6 | 3.76 MB | 31 Aug 2022 - 22:00 |
| LinkBot | 91+47 | 883.52 KB | 31 Aug 2022 - 20:51 |
| twitterbot | 62+30 | 12.49 MB | 31 Aug 2022 - 09:48 |
| Bing Preview bot | 85 | 25.87 MB | 31 Aug 2022 - 01:35 |
| YaCy | 45+38 | 2.80 MB | 30 Aug 2022 - 10:50 |
| ning | 47 | 1.65 MB | 25 Aug 2022 - 00:02 |
| Suspected bot masqurading as Mozilla | 46 | 2.41 MB | 29 Aug 2022 - 21:50 |
| TITAN | 27 | 684.56 KB | 10 Aug 2022 - 11:40 |
| Googlebot-Image | 26 | 1.83 MB | 31 Aug 2022 - 14:24 |
| CFNetwork | 23 | 224.14 KB | 31 Aug 2022 - 02:10 |
| Nutch | 6+16 | 233.55 KB | 31 Aug 2022 - 18:27 |
| Suspected bot masqurading as Mozilla | 20 | 0 | 25 Aug 2022 - 11:51 |
| Yahoo Slurp | 17+2 | 429.81 KB | 31 Aug 2022 - 19:12 |
| archive.org bot | 14 | 1005.15 KB | 17 Aug 2022 - 00:57 |
| Linkdex | 12+2 | 367.72 KB | 26 Aug 2022 - 00:27 |
| AdsBot-Google | 0+12 | 2.55 KB | 21 Aug 2022 - 20:46 |
| Y!J Yahoo Japan | 1+9 | 16.75 KB | 28 Aug 2022 - 07:55 |
| BUbiNG | 3+7 | 83.66 KB | 12 Aug 2022 - 07:16 |
| InfoBot | 0+8 | 1.70 KB | 16 Aug 2022 - 06:12 |
| BBot | 4+3 | 289.05 KB | 21 Aug 2022 - 02:13 |
| WebCopy | 6 | 144.21 KB | 24 Aug 2022 - 15:51 |
| FavIconizer | 4 | 73.89 KB | 22 Aug 2022 - 04:12 |
| Proximic Spider | 0+4 | 872 Bytes | 29 Aug 2022 - 10:16 |
| Unknown robot (identified by 'checker') | 0+4 | 872 Bytes | 23 Aug 2022 - 03:49 |
| Daum | 0+4 | 872 Bytes | 30 Aug 2022 - 06:11 |
| microsoft-webdav-miniredir | 4 | 4.29 KB | 20 Aug 2022 - 01:24 |
| MagpieRSS | 0+4 | 872 Bytes | 12 Aug 2022 - 14:40 |
| HTTrack off-line browser | 3 | 10.91 KB | 26 Aug 2022 - 20:57 |
| Google AdSense | 3 | 54.54 KB | 16 Aug 2022 - 19:39 |
| Heritrix | 0+2 | 436 Bytes | 24 Aug 2022 - 00:51 |
| Jakarta commons-httpclient | 2 | 134.99 KB | 03 Aug 2022 - 13:07 |
| Feedburner | 2 | 0 | 21 Aug 2022 - 03:25 |
| Xenu's Link Sleuth (TM), see Wikipedia | 2 | 0 | 28 Aug 2022 - 20:57 |
| Phantom | 2 | 29.33 KB | 27 Aug 2022 - 21:49 |
| InternetArchive | 1 | 0 | 18 Aug 2022 - 01:17 |
| Custo | 1 | 22.31 KB | 26 Aug 2022 - 11:25 |
| yie8 | 1 | 127.23 KB | 18 Aug 2022 - 02:31 |
| NG 1.x (Exalead) | 1 | 25.36 KB | 13 Aug 2022 - 17:16 |
| linkedinbot | 0+1 | 218 Bytes | 04 Aug 2022 - 15:59 |
| Emacs-w3 Search Engine | 1 | 10.36 KB | 17 Aug 2022 - 12:23 |
| ComputingSite Robi/1.0 | 1 | 16.97 KB | 06 Aug 2022 - 19:10 |
|