how to apply robots.txt?

[gsmdib@…]

This is probably not solely related to trac, but still, could you please advise how to make robots.txt available for a trac instance? Thanks.

[nkantrowitz]

How are you running the Trac? If via Apache or Lighty, use their normal methods for serving files. If you are using tracd, you can look at the RobotsTxt plugin.

[gsmdib@…]

Via apache. Could you please advise how to implement the 'normal method'? TIA.

[sid]

Doing a simple web search will provide a lot of documentation how to do this. example search query for 'apache robots.txt

This is not a Trac issue.

[anonymous]

When hosting trac with apache and mod_python you have to explicitly unset any Python handlers for the robots.txt file:

<Location "/robots.txt">
    SetHandler None
</Location>

[anonymous]

Also sone that you have to put that entry (Location "/robots.txt"...) *after* the Python handler, otherwise it won't help.

[anonymous]

I put the robots.txt file in the htdocs/ directory. Trac 0.10.4 can find it there.