What are the Crawling, Google Indexing and SEO Ranking

What are the Crawling, Google Indexing and SEO Ranking

Crawling, Indexing and Ranking

Wrote this "What to do when a webmaster submits an article with a "URL is not in Google services" message?" written a few months ago, explaining why Google Search Console is crawling slowly, leading to error messages. I didn't expect this to have a follow-up, and the development was unexpected.

Although a dedicated article has been written to explain it, the client continues to ask Search Console related questions. After talking for a long time, she found out that she found that the latest articles published on the website could not be searched by Google. She suspected that there was something wrong with the settings of the website administrator, which prevented the new articles from being indexed.

 Why does the article collection time slow down?

Things go something like this:

  Example: Brother ~ I'm sorry, I have something else to ask you ~ My website is currently experiencing retrieval problems. When I submit a sitemap, it shows that everything is normal, but when I enter individual article URLs, it shows that it is not in the index. Help solve it? grateful

        W: It doesn't matter, you can refer to my article (link at the beginning of the article)

    E: I did everything mentioned in the article, but I couldn't improve it... Because I couldn't even search for it this time, I couldn't see where the problem was at all, so I came to ask for your help again~ There are some Strange error I can't understand (comes the search console image)

    E: The latest two articles can't be found at all~ The only thing that I think is more likely to cause problems is this (the picture of the ads.txt error message)

        W: I think you can find it by searching (pass to her the picture of "site:article URL") Don't worry so much~ It's just a little late, you can rest assured

    E: Not that one... It's an article published after 9/22~

        W: It's only 9/23 today, don't be in such a hurry

    E: What should I say, this problem has never happened before, so I wondered if there is any problem, and it should be solved as soon as possible. This is the first time that the article can't be found by the site method.

The following dialogue is omitted for now. It turned out that after so many rounds of dialogue, the client finally knew the true intention of the client. After she published a new article in the past, she could quickly find it on Google (maybe within a day), but now It often takes several days to find them. So I began to wonder where the settings were wrong, but when I handled the case by myself, I couldn't find the murderer. What the client really wants to know is how to do it so that the article can be quickly indexed by Google. So from the last article, right up to the last sentence of this conversation above, we've been spinning on the wrong thread. If she could take me directly to the crime scene, it would save this time.

1. Article inclusion speed

In fact, I have also discovered the same phenomenon in the past year. When searching for my latest article, it often takes several days to see it, which has never happened before. So my past cognition is actually similar to that of the client: "The website with a relatively high weight is indexed by Google faster, and my articles are often included within a few minutes." But what has not happened before does not mean that it will not happen in the future; the workload of Google in the past is not equal to the workload of Google now. So although I found that the article was indexed by Google slower, I didn't feel uncomfortable. I can fully appreciate how fast the website is increasing every day, and I can imagine the number of articles around the world increasing every day, which is absolutely astronomical compared to before. The Google servers must then have to make trade-offs to handle such a large amount of data.

2. Does the blogging platform have any influence?

So what are the trade-offs for Google's servers? What kind of sites will be indexed first? The client later asked questions: "My blogger friends who use WP don't have this problem, so I think it's weird." Me: "This has nothing to do with WP, you need to prove that all WP sites can be searched, instead of considering the status of one WP site as all WP sites." Client: "I have asked five questions here" Me: "How does the traffic of these five websites compare to yours?" Client: "There are two taller than me" It seems that things are a bit strange, because I don't know the status of the client's friend's WP website, so I can't say anything, and the 5 websites are small samples after all, maybe her friend did some basic action that she didn't do. The only thing I can think of is that WP may have a plug-in to automatically submit articles to Google, that is, we usually have to go to Google to manually submit a single article, and the WP website can automatically execute it. But aside from the plug-in factor, the speed at which Google indexes website articles is unlikely to be different for different blogging platforms. First of all, how to determine which platform is eligible for priority indexing? If this kind of thing is made public to Google, it will not be scolded. Secondly, there are so many platforms in the world. Can you tell what platform it is from the source code when indexing?

3. The impact of article update speed

In fact, as long as you understand the working principle of the Google search engine, you can know how long the crawler will visit and index the website.

    If a website has an average of one new article a week, then it is impossible for the crawler to check whether there is a new article.

    If the website takes a long time to write an article, maybe a month, then the crawler naturally does not need to waste the workload of the server, just check it again after a long time.

    If there are new articles on the website every day, the crawler will also be afraid of missing new articles, and will report every day to check whether there are new articles.

    For websites like news media that have new articles in minutes, crawlers are checking them all the time, and they will receive new articles in seconds.

The above principle is quite simple and easy to understand. If you want articles to be quickly included in one day, you have to have new articles every day to attract Google to index. There are many such concepts on the Internet, just to mention two articles:

How often Google indexes the site
What Is Crawl Frequency?

The speed of updating articles on this site is quite slow. At present, there will be an article in nearly a week, so I will not force the article to be included in Google immediately. After checking the speed of updating articles in the background of the client's website, I found that it has been slower than before, no wonder the situation of this article happened.

Fourth, is it necessary to deliberately update frequently

1. Significantly improved SEO

The update speed of this site is not the same from the past to the present, and there was a period of time when the update was more frequent, for example, there were new articles every 2~3 days.

Since the traffic source of this site is mainly based on SEO, if the traffic changes, it is obvious that the SEO weight is adjusted by Google, whether it is up or down.

In the period of frequent updates, I can also feel that the website traffic has increased significantly. This means that when the website is updated quickly, in addition to the faster Google indexing, it will also provide additional SEO Bonus rewards for the website, bringing traffic above the average.

2. The SEO boost is only a short-term phenomenon

However, don’t get me wrong, the traffic brought by frequent website updates is only a temporary phenomenon, and does not mean that the SEO weight of the website has been permanently improved.

To give a simple example, the boss has two ways of rewarding for good performance at work in the company: Salary Increase: This is a permanent reward that increases your income every month. The one-time amount may be smaller, but the total revenue will increase a lot in the long run. It usually happens when the boss feels that your abilities have improved significantly, or you have obtained certain qualifications. Give a bonus: This is a one-time bonus, although the single amount may be large, it will not increase the total income in the long run. Usually occurs in, monthly full attendance bonus, independent event bonus, etc. For the "frequent website update", what Google gives is only a bonus (Bonus). You have to keep doing it in order to continue to get the bonus, just like the attendance bonus, as long as the bonus is stagnant, it will be gone. The real goal of the website is to get a “salary increase”, which can only be achieved by substantially enhancing the gold content of the article. In the past, I discussed an article on this topic, “Can blog sites improve SEO to bring traffic?”, All kinds of SEO tips are not long-term solutions.

3. The donkey chasing the carrot

Finally, let’s make a summary. Although I know the benefits of “frequent website updates”, will I encourage readers to do this? Draw conclusions in several directions: Blogging for a living: If you rely on blogging for food and clothing, then of course you have to be really good at filling your stomach. Writing a lot of articles and writing well is a basic requirement. Any SEO skills also need to be used, so of course a website To update faster than others, the article should be better than others. Blogging is about writing about physical health: if blogging is just a hobby and doesn’t affect your regular work, then of course it doesn’t matter if you don’t pay attention to SEO skills, you don’t need to force articles out for traffic. Want to make some advertising money: I think this is the most embarrassing group. The more traffic, the more advertising revenue. Although they do not make a living from the website, this group will also find ways to study various SEO techniques, expecting any Increase the possibility of extra money. Then this group just fell into the trap of Google. Google is holding all kinds of carrots that increase SEO traffic in the short term. More indexed content he wanted to collect, increasing Google's revenue. But whether the time and health costs paid are proportional to the advertising revenue and whether the basic hourly salary can be obtained, I don't know.