Salutare! Poate va ajuta configurarea urmatoare pentru robots.txt - eu nu am avut probleme deocamdata.


User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /xmlrpc.php
Disallow: /wp-
Disallow: /trackback/
Disallow: ?wptheme=
Disallow: ?comments=
Disallow: ?replytocom
Disallow: /comment-page-
Disallow: /?s=
Disallow: /wp-content/

User-agent: *
Disallow: /feeds/rss/search
Disallow: /search
Disallow: /cgi-bin/

User-agent: CompSpyBot
User-agent: Curious George
User-agent: CybEye.com
User-agent: DoCoMo
User-agent: ExB Language Crawler
User-agent: Ezooms
User-agent: Flamingo_SearchEngine
User-agent: Genieo
User-agent: Genio
User-agent: LWNutch
User-agent: LexxeBot
User-agent: OpenWebIndex
User-agent: RediffNewsBot
User-agent: SEOENGWorldBot
User-agent: Scanmine
User-agent: Screaming Frog SEO Spider
User-agent: ShopWiki
User-agent: ShowyouBot
User-agent: Sosospider
User-agent: WocBot
User-agent: Yeti
User-agent: YoudaoBot
User-agent: daumoa
User-agent: gsa-crawler
User-agent: libcrawl
User-agent: linkdex
User-agent: magpie-crawler
User-agent: repparser
User-agent: sindice-site-manager
User-agent: sogou spider
User-agent: sogou
User-agent: woriobot
User-agent: yacybot
User-agent: yolinkBot
Disallow: /

User-agent: Mediapartners-Google
Disallow: /