|
|
| Yahoo! Slurp (yahoo.com) |
261044 |
50.55% |
|
| MSNBot (msn.com) |
179065 |
34.68% |
|
| Spider(?), faked user-agent |
26146 |
5.06% |
|
| GoogleBot (google.com) |
10132 |
1.96% |
|
| Unknown |
9207 |
1.78% |
|
| Ask Jeeves/Teoma (ask.com) |
2921 |
0.57% |
|
| Spider-on-the-fly (discoverlife.org) |
2441 |
0.47% |
|
| Stack Rambler (rambler.ru) |
2418 |
0.47% |
|
| ExaBot (exava.com) |
1759 |
0.34% |
|
| Yahoo! Slurp China (yahoo.com.cn) |
1751 |
0.34% |
|
| Charlotte (searchme.com) |
1572 |
0.3% |
|
| MSNBot Media (msn.com) |
1549 |
0.3% |
|
| Twiceler (cuill.com) |
1311 |
0.25% |
|
| NV32ts |
906 |
0.18% |
|
| DotBot (dotnetdotcom.org) |
749 |
0.15% |
|
| Voyager (kosmix.com) |
714 |
0.14% |
|
| Libwww-perl |
618 |
0.12% |
|
| ZyBorg (wisenut.com) |
582 |
0.11% |
|
| Fast Enterprise Crawler (fast.no) |
540 |
0.1% |
|
| Aotech.net E-mail collector |
514 |
0.1% |
|
| Yandex.ru robot |
493 |
0.1% |
|
| Speedy Spider (entireweb.com) |
475 |
0.09% |
|
| Sogou web spider (sogou.com) |
408 |
0.08% |
|
| BecomeBot (become.com) |
352 |
0.07% |
|
| IA Archiver (alexa.com) |
340 |
0.07% |
|
| Envolk spider (envolk.com) |
336 |
0.07% |
|
| Google Translate (google.com) |
271 |
0.05% |
|
| Yeti (naver.com) |
259 |
0.05% |
|
| VoilaBot (voila.fr) |
250 |
0.05% |
|
| Indy Library (E-mail collector) |
247 |
0.05% |
|
| Fast Web Crawler (fast.no) |
232 |
0.04% |
|
| LWP Simple |
222 |
0.04% |
|
| Yahoo Seeker (yahoo.com) |
209 |
0.04% |
|
| JyxoBot (jyxo.cz) |
205 |
0.04% |
|
| GigaBot (gigablast.com) |
188 |
0.04% |
|
| Java VM |
175 |
0.03% |
|
| Win Http Services (microsoft.com) |
169 |
0.03% |
|
| Voyager (lisa.co.jp) |
168 |
0.03% |
|
| GrubNG (grub.org) |
162 |
0.03% |
|
| Kalooga Bot (kalooga.com) |
162 |
0.03% |
|
| BaiDu Spider (baidu.com) |
129 |
0.02% |
|
| ScoutJet (scoutjet.com) |
128 |
0.02% |
|
| DoCoMo (docomo.ne.jp) |
122 |
0.02% |
|
| Lynx (file transferring tool) |
111 |
0.02% |
|
| SnapBot (snap.com) |
97 |
0.02% |
|
| NiceBot |
88 |
0.02% |
|
| Grub-client (grub.org) |
87 |
0.02% |
|
| IRLbot (irl.cs.tamu.edu) |
87 |
0.02% |
|
| Convera Crawler (authoritativeweb.com) |
83 |
0.02% |
|
| Atrax Solutions atraxbot (atraxsolutions.com) |
82 |
0.02% |
|
| DataFountains/DMOZ (ucr.edu) |
82 |
0.02% |
|
| Ichiro (help.goo.ne.jp) |
81 |
0.02% |
|
| DiscoBot (discoveryengine.com) |
79 |
0.02% |
|
| Session |
78 |
0.02% |
|
| Findlinks (wortschatz.uni-leipzig.de) |
74 |
0.01% |
|
| WebList.ru robot |
72 |
0.01% |
|
| MJ12bot (majestic12.co.uk) |
71 |
0.01% |
|
| Unknown (caching agent) |
70 |
0.01% |
|
| WGet (downloading tool) |
70 |
0.01% |
|
| SimilarPages Nutch Crawler (similarpages.com) |
64 |
0.01% |
|
| Page Bites Hyper Bot (pagebites.com) |
61 |
0.01% |
|
| PSBot (picsearch.com) |
61 |
0.01% |
|
| GaisBot (gais.cs.ccu.edu.tw) |
59 |
0.01% |
|
| NaverBot (naver.com) |
58 |
0.01% |
|
| POE-Component-Client-HTTP |
58 |
0.01% |
|
| NutchCVS (apache.org) |
57 |
0.01% |
|
| MLBot (metadatalabs.com) |
56 |
0.01% |
|
| Heritrix (accelobot.com) |
55 |
0.01% |
|
| Jakarta Commons-HttpClient |
51 |
0.01% |
|
| NutchCVS (nutch.org) |
51 |
0.01% |
|
| Ginger Crawler (gingersoftware.com) |
50 |
0.01% |
|
| Factbot (factbites.com) |
49 |
0.01% |
|
| Sosospider (soso.com) |
43 |
0.01% |
|
| FollowSite Bot (followsite.com) |
42 |
0.01% |
|
| Szukacz (szukacz.pl) |
42 |
0.01% |
|
| Fast MetaWeb Crawler (fastsearch.com) |
41 |
0.01% |
|
| NEWT ActiveX (Web Collector) |
41 |
0.01% |
|
| AipBot (aipbot.com) |
40 |
0.01% |
|
| Almaden Crawler (almaden.ibm.com) |
39 |
0.01% |
|
| TurnitinBot (turnitin.com) |
39 |
0.01% |
|
| Unknown E-mail collector |
39 |
0.01% |
|
| Xenu Link Sleuth (link cheching) |
38 |
0.01% |
|
| Baypup (baypup.com) |
37 |
0.01% |
|
| Exalead NG (aol.com) |
37 |
0.01% |
|
| Yanga World Search Bot (yanga.co.uk/) |
37 |
0.01% |
|
| Sapphire Web Nutch Crawler (boston.lti.cs.cmu.edu) |
36 |
0.01% |
|
| MSNBot Products (msn.com) |
35 |
0.01% |
|
| Net Research Server (loopimprovements.com) |
35 |
0.01% |
|
| GoogleBot-Mobile (google.com) |
33 |
0.01% |
|
| CCBot (commoncrawl.org) |
30 |
0.01% |
|
| Python-urllib fetcher (Naver Japan / Korea) |
30 |
0.01% |
|
| Sproose bot (sproose.com) |
29 |
0.01% |
|
| Holmes / Onet Szukaj (szukaj.onet.pl) |
28 |
0.01% |
|
| Scooter (altavista.com) |
28 |
0.01% |
|
| SpiderMan (yahoo.com) |
27 |
0.01% |
|
| Jewelry City Spider (jewelrycity.org). |
26 |
0.01% |
|
| JoeDog |
26 |
0.01% |
|
| Nu_tch-princeton crawler (cs.princeton.edu) |
26 |
0.01% |
|
| Nutch Crawler (nutch.org) |
25 |
0% |
|
| YoudaoBot (youdao.com) |
25 |
0% |
|
| Total |
516399 |
|