Citat:
Originally posted by stefanaxe@Apr 16 2006, 11:18
Följer inte AA robot-anvisningar som alla andra sökmotorer?
|
Errhmm..
Kod:
User-agent: Googlebot
Disallow: /
Allow: /li
Allow: /vi
Allow: /list
Allow: /view
Allow: /hjalp
Allow: /*.htm
Allow: /*.html
User-agent: ia_archiver
Disallow:
User-agent: slurp
Disallow /hjalp
Disallow /aw
Disallow /cgi-bin
User-agent: msnbot
Disallow /hjalp
Disallow /aw
Disallow /cgi-bin
User-agent: *
Disallow: /