Indexare gresita wordpress

Am vrut sa vad ce pagini am indexate de google la un blog pe wordpress.

Si am gasit o gramada de posturi (articole) indexate in genul: site.com/titlu_post/209/ sau site.com/titlu_post/105/

Apar indexate si articolele cu url normal, fara acele numere in terminatie. Am verificat la permalinks si are structura: /%postname%/

Deci sunt duplicat.

Blogul are o versiune veche All in One SEO Pack 2.2.3.1.

Titlurile la articole au terminatia "part 209", "part 105".

robots.txt are
User-Agent: *

Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /archives/
Disallow: /page/
Disallow: /comments/feed/
Disallow: /index.php
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /tag
Disallow: /author
Disallow: /trackback
Disallow: /*trackback
Disallow: /*trackback*
Disallow: /*/trackback
Disallow: /feed/
Disallow: /xmlrpc.php
Disallow: /?p=*

Ce pot sa fac?

Ruleaza mai intai un xenu si zice el de unde au fost generate.
Apoi vezi ce minune le putea genera.
Spor

Am rezolvat, era de la un script care afisa comentariile in sidebar.

Daca url-ul avea in coada numere /245/, general url la comentarii cu tot cu acele numere in coada.

Bun acel program "xenu", nu cred ca gaseam fara el.

Pune si link ca eu am uitat, poate au nevoie si altii Find broken links on your site with Xenu's Link Sleuth (TM)
Duce pana la 2 milioane de linkuri interne si externe. Pentru mai mult, pe bani dar bun https://www.screamingfrog.co.uk/seo-spider/pricing/
Pune si tu numele acelui "script" (plugin) poate cauta si altii.
Weekend fain sa avem :)

Era un script facut de mine.

Chestia asta: <?php comment_ID() ?> genera acel numar in coada. Insa nu stiam pentru ca nu era afisat daca postul nu avea acele numere. Cred ca un post din blog a avut un permalink stricat, de genul site.com/264/ si astfel s-au generat link-urile alea si s-au propagat mai departe in alte link-uri.

Treaba de programatori :) Nu ma bag peste ei.

Salutare! Poate va ajuta configurarea urmatoare pentru robots.txt - eu nu am avut probleme deocamdata.


User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /xmlrpc.php
Disallow: /wp-
Disallow: /trackback/
Disallow: ?wptheme=
Disallow: ?comments=
Disallow: ?replytocom
Disallow: /comment-page-
Disallow: /?s=
Disallow: /wp-content/

User-agent: *
Disallow: /feeds/rss/search
Disallow: /search
Disallow: /cgi-bin/

User-agent: CompSpyBot
User-agent: Curious George
User-agent: CybEye.com
User-agent: DoCoMo
User-agent: ExB Language Crawler
User-agent: Ezooms
User-agent: Flamingo_SearchEngine
User-agent: Genieo
User-agent: Genio
User-agent: LWNutch
User-agent: LexxeBot
User-agent: OpenWebIndex
User-agent: RediffNewsBot
User-agent: SEOENGWorldBot
User-agent: Scanmine
User-agent: Screaming Frog SEO Spider
User-agent: ShopWiki
User-agent: ShowyouBot
User-agent: Sosospider
User-agent: WocBot
User-agent: Yeti
User-agent: YoudaoBot
User-agent: daumoa
User-agent: gsa-crawler
User-agent: libcrawl
User-agent: linkdex
User-agent: magpie-crawler
User-agent: repparser
User-agent: sindice-site-manager
User-agent: sogou spider
User-agent: sogou
User-agent: woriobot
User-agent: yacybot
User-agent: yolinkBot
Disallow: /

User-agent: Mediapartners-Google
Disallow: /