E ciudat ce ziceti voi.
Chiar pe pagina lor de help http://help.yahoo.com/help/us/ysearch/slur...p/slurp-02.html se face referire la
http://www.robotstxt.org/wc/norobots.html unde zice asa:
Disallow
...
Any empty value, indicates that all URLs can be retrieved. At least one Disallow field needs to be present in a record.
Imi e greu sa cred ca fac referire la niste reguli pe care apoi le incalca.

Ar mai fi o varianta mentionata la http://www.robotstxt.org/wc/exclusion-admin.html de a crea robots.txt gol. De asta ce ziceti?