How an search engine marketing Mounted a Bizarre Crawled At present Not Listed Subject

How an SEO Fixed a Weird Crawled Currently Not Indexed Issue

A technical search engine marketing printed a case examine of how he solved a curious Crawled At present Not Listed drawback on his website. Whereas the answer he discovered won’t be common to others experiencing this drawback, his methodology for figuring out the issue and fixing it presents a helpful walkthrough for fixing technical search engine marketing issues.

What occurred to his website indexing was actually bizarre. However his answer was easy and is smart.

I found an outline of this drawback on a tweet by Adam Gent (@Adoubleagent)


Proceed Studying Under

Crawled – At present Not Listed

There are a lot of anecdotal stories of Crawled At present Not Listed on Fb, Twitter and even in John Mueller’s Workplace-hours hangouts.

In a latest Workplace-hours hangout somebody requested why Google Search Console (GSC) was exhibiting Crawled Not Listed however once you click on by they turn into listed. John Mueller answered that it’s only a lag between stories.

And in one other Workplace-hours hangout John Mueller pointed out that it’s totally regular for a website to have many web page not be listed.

He famous:

“…if in case you have a smaller website and also you’re seeing a big a part of your pages should not being listed, then I’d take a step again and attempt to rethink the general high quality of the web site and never focus a lot on technical points for these pages.

The opposite factor to remember on the subject of indexing, is it’s fully regular that we don’t index every part off of the web site.

And over time, once you get to love 200 pages in your web site and we index 180 of them, then that proportion will get somewhat bit smaller.”


Proceed Studying Under

Whereas each of these are good causes to clarify why the Crawled Not Listed difficulty is occurring to some folks, that’s not the rationale Adam Gent found.

Adam Gent found a completely totally different drawback that seemed to be an algorithm difficulty at Google itself. There was nothing fallacious with the positioning itself, the issue was with Google’s indexing.

Why Crawled – At present Not Listed

Adam reviewed the GSC Index Protection report and found that Google was crawling and indexing his feeds as in the event that they have been HTML pages.

He took random phrases from these pages and did a website: search with these phrases and found that the feed web page content material was certainly listed.

To make issues worse, Google had apparently canonicalized the content material on the RSS feed over the precise net web page, accounting for why the true net pages have been crawled however not listed.

The RSS feed Was Generated by WordPress

An odd factor about this case is that once you have a look at the feed web page it renders like an internet web page and never how an XML file often renders.

Screenshot of Cache of RSS Feed

Screenshot of a cached RSS page

I could be fallacious however that doesn’t seem like a standard RSS feed. It appears like an HTML web page.


Proceed Studying Under

Though the underlying code actually is XML that’s not  how most feeds usually look.

Might which have performed a task in why Google selected to canonicalize the feed?

It’s arduous to grasp how that would occur as a result of there are such a lot of alerts like inner linking that underneath traditional circumstances would trigger Google to favor the HTML pages as canonical.

How Adam Mounted the Drawback

After Adam found out what occurred he eliminated these WordPress generated feed pages, submitted the feed URLs for a crawl after which 404’d the pages.

After these pages have been dropped from the index he subsequent submitted the right URLs to Google and inside a number of days the issue was mounted.


Proceed Studying Under

What Triggered the Drawback?

Adam wrote that the issue seems to be on Google’s facet.

I requested round and somebody informed me that apparently a number of years in the past Google began indexing feeds however that he thought this drawback had been mounted.

I’m not an skilled on XML nevertheless it appears uncommon that the feed resembles an HTML web page as an alternative of the conventional XML format that reveals up with out HTML styling.

The feed doesn’t look regular so it looks as if that no matter is making it seem like that could be an underlying trigger.

Regardless, when you’re having Crawled At present Not Listed issues, that is yet one more factor to test in case it’s additionally occurring to you.


Proceed Studying Under


Learn the unique submit that walks by fixing the issue:

A Curious Case of Canonicalization

Source link

Leave A Comment



Our purpose is to build solutions that remove barriers preventing people from doing their best work.

Giza – 6Th Of October
(Sunday- Thursday)
(10am - 06 pm)