website/robots.txt

4 lines
90 B
Text
Raw Normal View History

User-agent: *
# exclude CGI stuff -- it really screws with searches.
Disallow: /cgi-bin/*