Indexare gresita wordpress

[Seph]

Am vrut sa vad ce pagini am indexate de google la un blog pe wordpress.

Si am gasit o gramada de posturi (articole) indexate in genul: site.com/titlu_post/209/ sau site.com/titlu_post/105/

Apar indexate si articolele cu url normal, fara acele numere in terminatie. Am verificat la permalinks si are structura: /%postname%/

Deci sunt duplicat.

Blogul are o versiune veche All in One SEO Pack 2.2.3.1.

Titlurile la articole au terminatia "part 209", "part 105".

robots.txt are
User-Agent: *

Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /archives/
Disallow: /page/
Disallow: /comments/feed/
Disallow: /index.php
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /tag
Disallow: /author
Disallow: /trackback
Disallow: /*trackback
Disallow: /*trackback*
Disallow: /*/trackback
Disallow: /feed/
Disallow: /xmlrpc.php
Disallow: /?p=*

Ce pot sa fac?

[Sorin Frumuseanu]

Ruleaza mai intai un xenu si zice el de unde au fost generate.
Apoi vezi ce minune le putea genera.
Spor

[Seph]

Am rezolvat, era de la un script care afisa comentariile in sidebar.

Daca url-ul avea in coada numere /245/, general url la comentarii cu tot cu acele numere in coada.

Bun acel program "xenu", nu cred ca gaseam fara el.

[Sorin Frumuseanu]

Pune si link ca eu am uitat, poate au nevoie si altii Find broken links on your site with Xenu's Link Sleuth (TM)
Duce pana la 2 milioane de linkuri interne si externe. Pentru mai mult, pe bani dar bun https://www.screamingfrog.co.uk/seo-spider/pricing/
Pune si tu numele acelui "script" (plugin) poate cauta si altii.
Weekend fain sa avem

[Seph]

Era un script facut de mine.

Chestia asta: <?php comment_ID() ?> genera acel numar in coada. Insa nu stiam pentru ca nu era afisat daca postul nu avea acele numere. Cred ca un post din blog a avut un permalink stricat, de genul site.com/264/ si astfel s-au generat link-urile alea si s-au propagat mai departe in alte link-uri.

[Sorin Frumuseanu]

Treaba de programatori Nu ma bag peste ei.

[vertedi]

Salutare! Poate va ajuta configurarea urmatoare pentru robots.txt - eu nu am avut probleme deocamdata.


User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /xmlrpc.php
Disallow: /wp-
Disallow: /trackback/
Disallow: ?wptheme=
Disallow: ?comments=
Disallow: ?replytocom
Disallow: /comment-page-
Disallow: /?s=
Disallow: /wp-content/

User-agent: *
Disallow: /feeds/rss/search
Disallow: /search
Disallow: /cgi-bin/

User-agent: CompSpyBot
User-agent: Curious George
User-agent: CybEye.com
User-agent: DoCoMo
User-agent: ExB Language Crawler
User-agent: Ezooms
User-agent: Flamingo_SearchEngine
User-agent: Genieo
User-agent: Genio
User-agent: LWNutch
User-agent: LexxeBot
User-agent: OpenWebIndex
User-agent: RediffNewsBot
User-agent: SEOENGWorldBot
User-agent: Scanmine
User-agent: Screaming Frog SEO Spider
User-agent: ShopWiki
User-agent: ShowyouBot
User-agent: Sosospider
User-agent: WocBot
User-agent: Yeti
User-agent: YoudaoBot
User-agent: daumoa
User-agent: gsa-crawler
User-agent: libcrawl
User-agent: linkdex
User-agent: magpie-crawler
User-agent: repparser
User-agent: sindice-site-manager
User-agent: sogou spider
User-agent: sogou
User-agent: woriobot
User-agent: yacybot
User-agent: yolinkBot
Disallow: /

User-agent: Mediapartners-Google
Disallow: /