Last week’s announcement of the coming float of the S&P500 index provider, Vanguard, caused an immediate reaction. Indexes are a key driver behind the success of ETFs, which is why the potential drop in indexing uptake has serious implications for the ETF industry.
Infrastructure is the support structure that allows our society to run on a daily basis. It includes both man-made structures such as buildings, bridges, highways, and utility poles and naturally occurring structures such as mountains, rivers, and forests. Infrastructure has become one of the most important and yet underreported topics in the news, and it’s no wonder why. When a person hears the words “infrastructure” or “crisis”, they immediately think of the state of our nation’s aging and broken infrastructure. However, as Americans, we should not be fixated on one single issue, but instead be reminded that there are many issues that need to be addressed.
Google’s John Mueller responded to a question about the decrease in crawl speed after major changes to a website’s infrastructure, in this case moving to a new content delivery network (CDN).Muller explains why indexing seems to be slower after such changes.
The questioner noted that he had created URL redirects and switched to a new content delivery network (CDN) and shortly thereafter saw a noticeable drop in indexing numbers.
General information about indexing websites on Google
Indexing is the process by which Google’s crawler, the Googlebot, visits a web page and loads it into Google’s search index.
View
More information below
Google’s search index is where candidate pages are located that can be ranked. The search results come from the Google search index.
That’s why Google’s indexing level is so important. Google needs to explore and discover new pages to rank them.
It is considered a serious problem if anything prevents Google from crawling the website.
According to Google’s help page on crawling and indexing:
We use software called web crawlers to detect publicly accessible websites. Crawlers search web pages and click on links just as you browse the web’s content.
You go from link to link and send data about these web pages to Google’s servers.
…We take into account key signals – from keywords to page freshness – and track it all in the search index.
View
More information below
Changing the content delivery network is a big change
A content delivery network is a network of servers located all over the world. The goal is to speed up the delivery of website content by delivering it from a server that is close to the person who wants to visit the site.
In theory, Google should be able to visit the site and index it even faster, which is a reasonable expectation.
The question is this:
After the redirection and the change of CDN, we see a dramatic drop in crawl rate.
John Mueller explains the dramatic drop increep
John Mueller of Google replied:
Yes! It makes perfect sense.
If you change the infrastructure of your website, we change our search engine.
On the one hand, we have to be a bit conservative in the beginning and make sure we don’t cause any problems, and then we automatically start to pick up the pace.
So if you switch to a different CDN, that’s a significant change in infrastructure. We acknowledge this change and pause the exploration for a while, only to start it again when we feel it is fast.
Google avoids being disruptive
The drastic reduction in indexing for major infrastructure changes is intended to make life easier for the updated site.
Sometimes a site making a major change can go through several changes in a week or more.
Indexing a site that is not fully updated or ready to be indexed can be confusing for Google if the site is constantly changing.
So it’s a good thing that Google slows down crawling after major changes to the site’s infrastructure.
View
More information below
Summons
See Mueller’s response to the question at 29:10 minute.
Related Tags:
google search declineguest post google penaltysearch engine journal guest postgoogle guest postingguest post seohiring an seo expert,People also search for,Privacy settings,How Search works,google search decline,guest post google penalty,search engine journal guest post,google guest posting,guest post seo,hiring an seo expert,hiring seo,how to hire an seo
