[hnn]

Originally posted by stefanaxe@Apr 16 2006, 11:18
Följer inte AA robot-anvisningar som alla andra sökmotorer?

Errhmm..

User-agent: Googlebot
Disallow: /
Allow: /li
Allow: /vi
Allow: /list
Allow: /view
Allow: /hjalp
Allow: /*.htm
Allow: /*.html

User-agent: ia_archiver
Disallow:

User-agent: slurp
Disallow /hjalp
Disallow /aw
Disallow /cgi-bin

User-agent: msnbot
Disallow /hjalp
Disallow /aw
Disallow /cgi-bin

User-agent: *
Disallow: /