Author: LION Digital

Should we make a UK version of our English blog?

Q. (03:15) The company I work at is working on expanding its market in the UK and recently launched a UK subsite. 70% of our US traffic comes from our blog, and we don’t currently have a blog on the UK subsite. Would translating our US blog post into Queen’s English be beneficial for UK exposure?

• (03:42) I doubt that would make a big difference. So, in particular, when it comes to hreflang, which is the way that you would connect different language or country versions of your pages, we would not be ranking these pages better just because we have a local version. It’s more that if we understand that there is a local version, and we have that local version indexed, and it’s a unique page, then we could swap out that URL at the same position in the search results to the more local version. So it’s not that your site will rank better in the UK if you have a UK version. It’s just that we would potentially show the link to your UK page instead of the current one. For, essentially, informational blog posts, I suspect you don’t really need to do that. And one of the things I would also take into account with internationalisation is that fewer pages are almost always much better than having more pages. So, if you can limit the number of pages that you provide by not doing a UK version of the content that doesn’t necessarily need to have a UK version, then that’s almost always better for your site, and it’s easier for you to maintain.

How is a language mismatch between the main content & the rest of a page treated?

Q. (05:04) How might Google treat a collection of pages on a site that is in one language per URL structure, for example, example.com/de/blogarticle1, and the titles may be in German, but the descriptions are in English, and the main content is also in English?

• (05:25) So the question goes on a little bit with more variations of that. In general, we do focus on the primary content on the page, and we try to use that, essentially, for ranking. So, if the primary content is in English, that’s a really strong sign for us that this page is actually in English. That said, if there’s a lot of additional content on the page that is in a different language, then we might either understand this page is in two languages, or we might be a little bit confused if most of the content is in another language and say, well, perhaps this page is actually in German, and there’s just a little bit of English on it as well. So that’s one thing to kind of watch out for. I would really make sure that, if the primary content is in one language, that it’s really a big chunk of primary content, and that it’s kind of useful when people go to that page who are searching in that language. The other thing to watch out for is the titles of the page and things like the primary headings on the page. They should also match the language of the primary content. So if the title of your page is in German, and the primary content of your page is in English, then it’s going to be really, really hard for us to determine what we should show in the search results because we try to match the primary content. That means we would try to figure out what would be a good title for this page. That also means that we would need to completely ignore the titles that you provide. So if you want to have your title shown, make sure that they also match the primary language of the page.

Does an age-interstitial block crawling or indexing?

Q.  (08:20)  If a website requires users to verify their age before showing any content by clicking a button to continue, is it possible that Google would have problems crawling the site? If so, are there any guidelines around how to best handle this?

• (08:40) So, depending on the way that you configure this, yes, it is possible that there might be issues around crawling the site. In particular, Googlebot does not click any buttons on a page. So it’s not that Google would be able to navigate through an interstitial like that if you have something that is some kind of a legal interstitial. And especially if it’s something that requires verifying an age, then people have to enter something and then click Next. And Googlebot wouldn’t really know what to do with those kinds of form fields. So that means, if this interstitial is blocking the loading of any other content, then probably that would block indexing and crawling as well. A really simple way to test if this is the case is just to try to search for some of the content that’s behind that interstitial. If you can find that content on Google, then that probably means that we were able to actually find that content. From a technical point of view, what you need to watch out for is that Google is able to load the normal content of the page. And, if you want to show an interstitial on top of that, using JavaScript or HTML, that’s perfectly fine. But we need to be able to load the rest of the page as well. So that’s kind of the most important part there. And that also means that if you’re using some kind of a redirect to a temporary URL and then redirecting that to your page, that won’t work. But, if you’re using JavaScript/CSS to kind of display an interstitial on top of your existing content that’s already loaded, then that would work for Google Search. And, from a kind of a policy point of view, that’s fine. That’s not something that we would consider to be cloaking because the content is still being loaded there. And especially if people can get to that content after navigating through that interstitial, that’s perfectly fine.

Is using the Indexing API for a normal website good or bad?

Q. (15:35)  Is using API index or the indexing API good or bad for a normal website?

• (08:40) So the indexing API is meant for very specific kinds of content, and using it for other kinds of content doesn’t really make sense. That’s similar, I think, I don’t know– using construction vehicles as photos on your website. Sure, you can put it on a medical website, but it doesn’t really make sense. And, if you have a website about building houses, then, sure, put construction vehicles on your website. It’s not that it’s illegal or that it will cause problems if you put construction vehicles on your medical website, but it doesn’t really make sense. It’s not really something that fits there. And that’s similar with the indexing API. It’s really just for very specific use cases. And, for everything else, that’s not what it’s there for.

Does Googlebot read the htaccess file?

Q. (16:31) Does Googlebot read the htaccess file?

• (16:36) The short answer is no because, usually, a server is configured in a way that we can’t access that file, or nobody can access that file externally. The kind of longer answer is that the htaccess file controls how your server responds to certain requests, assuming you’re using an Apache server, which uses this as a control file. And essentially, if your server is using this file to control how it responds to certain requests, then, of course, Google and anyone will see the effects of that. So it’s not– again, assuming this is a control file on your server, it’s not that Google would read that file and do something with it, but rather Google would see the effects of that file. So, if you need to use this file to control certain behavior on your website, then go for it.

How does Google Lens affect SEO?

Q. (17:39) How does Multisearch in Google Lens can affect SEO?

• (17:44) So this is something, I think, that is still fairly new. We recently did a blog post about this, and you can do it in Chrome, for example, and on different types of phones. Essentially, what happens is you can take a photo or an image from a website, and you can search using that image. For example, if you find a specific piece of clothing or a specific don’t know anything, basically, that you would like to find more information on, you can highlight that section of the image and then search for more things that are similar. And, from an SEO point of view, that’s not really something that you would do manually to make that work, but rather, if your images are indexed, then we can find your images, and we can highlight them to people when they’re searching in this way. So it’s not that there’s like a direct effect on SEO or anything like that. But it’s kind of like, if you’re doing everything right, if your content is findable in Search, if you have images on your content, and those images are relevant, then we can guide people to those images or to your content using multiple ways.

I’m unsure what to do to make my Blogger or Google Sites pages indexable.

Q. (21:24) I’m unsure what is needed to have my Blogger and Google Sites pages searchable. I assumed Google would crawl its own platforms

• (21:34) So I think the important part here is that we don’t have any special treatment for anything that is hosted on Google’s systems. And, in that regard, you should treat anything that you host on Blogger or on Google Sites or anywhere else essentially the same as any content that you would host anywhere on the web. And you have to assume that we need to be able to crawl it. We need to be able to, well, first, discover that it actually exists there. We need to be able to crawl it. We need to be able to index it, just like any other piece of content. So just because it’s hosted on Google’s systems doesn’t give it any kind of preferential treatment when it comes to the search results. It’s not that there is a magic way that all of this will suddenly get indexed just because it’s on Google, but rather, we have these platforms. You’re welcome to use them, but they don’t have any kind of special treatment when it comes to Searching. And also, with these platforms, it’s definitely possible to set things up in a way that won’t work as well for Search as they could. So, depending on how you configure things with Blogger and with regards to Google Sites, how you set that up, and which kind of URL patterns that you use, it may be harder than the basic setup. So just because something is on Google’s systems doesn’t mean that there’s a preferential way of that being handled.

Why is a specific URL on my site not crawled?

Q. (25:29) Is there anything like a URL format penalty. I’m facing a weird problem where a particular URL doesn’t get crawled. It’s the most linked page on a website, and still, not even one internal link is found for this URL, and it looks like Google is simply ignoring this kind of URL. However, if we slightly change the URL, like an additional character or word, the URL gets crawled and indexed. The desired URL format is linked within the website, present and rendered HTML, and submitted in a sitemap as well, but still not crawled or indexed.

• (26:06) So it’s pretty much impossible to say without looking at some examples. So this is also the kind of thing where I would say, ideally, go to the Help forums and post some of those sample URLs and get some input from other folks there. And the thing to also keep in mind with the Help forums is that the product experts can escalate issues if they find something that looks like something is broken in Google. And sometimes things are broken in Google, so it’s kind of good to have that path to escalate things.

Does adding the location name to the description help ranking?

Q. (26:50) Does adding the location name in the meta description matter to Google in terms of a ranking if the content quality is maintained?

• (26:59) So the meta description is primarily used as a snippet in the search results page, and that’s not something that we would use for ranking. But, obviously, having a good snippet on a search results page can make it more interesting for people to actually visit your page when they see your page ranking in the search results.

Which structured data from schema do I add to a medical site?

Q. (27:20) How does schema affect a medical niche’s website? What kind of structure data should be used there?

• (27:27) So I would primarily when it comes to structured data, I would primarily focus on the things that we have documented in our developer documentation and the specific features that are tied to that. So, instead of saying, what kind of structured data should I use for this type of website, I would kind of turn it around and say, what kind of visible attributes do I want to have found in the search results? And then, from there, look at what are the requirements for those visual attributes, and can I implement the appropriate structure data to fulfill those requirements? So that’s kind of the direction I would head there.

Does every page need schema or structured data?

Q. (28:06) Does every page need schema or structured data?

• (28:10) No, definitely not. As I mentioned, use the guide of what visual elements I want to have visible for my page, and then find the right structured data for that. It’s definitely not the case that you need to put structured data on every page.

Sign up for our Webmaster Hangouts today!

Crawling of Website.

Q. (00:55) My question is about the crawling of our website. We have different numbers from the crawling of the Search Console in our server log. For instance, we have three times the number of the crawling from Google in our server, and in Search Console, we have one of the three bars. Could it be possible that there maybe something wrong as the numbers are different.

• (1:29) I  think the numbers would always be different, just because of the way that in Search Console we report on all crawls that go through the infrastructure of Googlebot. But that also includes other types of requests. So for example, I think Adsbot also uses the Googlebot infrastructure. Those kinds of things. And they have different user agents. So if you look at your server logs, and only look at the Googlebot user agent that we use for web search, those numbers will never match what we show in that Search Console.

Why Did Discover Traffic Drop?

Q. (03:10) I have a question in mind. We have a website. And from the last three to four months, we have been working on Google Web Stories.  It was going very well. On the last one, the 5th of March, actually, we were having somewhere around 400 to 500 real-time results coming from Google Discover on our Web Stories. But suddenly, we saw an instant drop in our visitors from Google Discover, and our Search Console is not even showing any errors. So what could be a possible reason for that?

• (3:48) I don’t think there needs to be any specific reason for that, because, especially with Discover, it’s something that we would consider to be additional traffic to a website. And it’s something that can change very quickly. And anecdotally, I’ve seen that from, or I’ve heard that from sites in these Office Hours that sometimes, they get a lot of traffic from Discover and then suddenly it goes away. And then it comes back again. So it doesn’t necessarily mean that you’re doing anything technically wrong. It might just be that, in Discover, things have slightly changed, and then suddenly you get more traffic or suddenly you get less traffic. We do have a bunch of policies that apply to content that we show in Google Discover, and I would double-check those, just to make sure that you’re not accidentally touching on any of those areas. That includes things, I don’t know offhand, but I think something around the lines of clickbaity content, for example, those kinds of things. But I would double-check those guidelines to make sure that you’re all in line there. But even if everything is in line with the guidelines, it can be that suddenly you get a lot of traffic from Discover, and then suddenly, you get a lot less traffic.

Can We Still Fix the robots.txt for Redirects?

Q.  (14:55) We have had a content publishing website since 2009, and we experienced a bad migration in 2020, where we encountered a huge drop in organic traffic. So the question here is that we had a lot of broken links, so we use the 301 redirect to redirect these broken links to the original articles. But what we did in robots.txt, we disallowed these links so that the crawling budget won’t be gone on crawling this for all four pages. So the main thing here, if we fixed all these redirects, we redirected to the same article with the proper name, can we remove these links from the robots.txt, and how much time does it take to actually be considered by Google.

• (15:53) So if the page is blocked by the robots.txt, we wouldn’t be able to see the redirect. So if you set up a redirect, you would need to remove that block in the robots.txt. With regards to the time that takes, there is no specific time, because we don’t crawl all pages at the same speed. So some pages we may pick up within a few hours, and other pages might take several months to be recrawled. So that’s, I think, kind of tricky. The other thing, I think, is worth mentioning here is, that if this is from a migration that is two years back now, then I don’t think you would get much value out of just making those 404 links show content now. I can’t imagine that would be the reason why a website would be getting significantly less traffic. Mostly, because it’s– unless these pages are the most important pages of your website, but then you would have noticed that. But if these are just generic pages on a bigger website, then I can’t imagine that the overall traffic to a website would drop because they were no longer available.

Some of the Blogs Posts Aren’t Indexed, What Can We Do?

Q. (18:54) My question is a crawling question pertaining to discovered not indexed. We have run a two-sided marketplace since 2013 that’s fairly well established. We have about 70,000 pages, and about 70% of those are generally indexed. And then there’s kind of this budget that crawls the new pages that get created, and those, we see movement on that, so that old pages go out, new pages come in. At the same time, we’re also writing blog entries from our editorial team, and to get those to the top of the queue, we always use this request indexing on those. So they’ll go quicker. We add them to the sitemap, as well, but we find that we write them and then we want them to get in to Google as quickly as possible.  As we’ve kind of been growing over the last year, and we have more content on our site, we’ve seen that that sometimes doesn’t work as well for the new blog entries. And they also sit in this discovered not indexed queue for a longer time. Is there anything we can do to internal links or something? Is it content-based, or do we just have to live with the fact that some of our blogs might not make it into the index?

• (20:13) Now, I think overall, it’s kind of normal that we don’t index everything on a website. So that can happen to the entries you have on the site and also the blog post on the site. It’s not tied to a specific kind of content. I think using the Inspect URL tool to submit them to indexing is fine. It definitely doesn’t cause any problems. But I would also try to find ways to make those pages as clear as possible that you care about them. So essentially, internal linking is a good way to do that. To really make sure that, from your home page, you’re saying, here are the five new blog posts, and you link to them directly. So that it’s easy for Googlebot when we crawl and index your home page, to see, oh, there’s something new, and it’s linked from the home page. So maybe it’s important. Maybe we should go off and look at that.

Can a Low Page Speed Score Affect the Site’s Ranking?

Q. (27:28) Does low-rating mobile results on Google page speed like LCP, FID, might have affected our website rank after the introduction of the new algorithm last summer? Because we were like the fourth in my city? If I check a web agency or a keyword that we saw after the introduction of this algorithm and go on Google Search Console, we find out that these parameters like LCP, FID for mobile, have a bad rating, like 48, not for desktop, there is 90. So it’s OK. So could this be the problem?

• (28:24) Could be. It’s hard to say just based on that. So I think there are maybe two things to watch out for. The number that you gave me sounds like the PageSpeed Insights score that is generated, I think, on desktop and mobile. Kind of that number from 0 to 100, I think. We don’t use that in search, for the rankings. We use the Core Web Vitals, where there is LCP FID and CLS, I think. And the metrics that we use are based on what users actually see. So if you go into the Search Console, there’s the Core Web Vitals report. And that should show you those numbers. If it’s within good or bad, kind of in those ranges.

Can Google Crawl Pagination With “View More” Buttons?

Q. (39:51) I recently redesigned my website and changed the way I list my blog posts and other pages from pages one, two, three, four to a View More button. Can Google still crawl the ones that are not shown on the main blog page? What is the best practice? If not, let’s say those pages are not important when it comes to search and traffic, would the whole site as a whole be affected when it comes to how relevant it is for the topic for Google?

• (40:16) So on the one hand, it depends a bit on how you have that implemented. A View More button could be implemented as a button that does something with JavaScript, and those kinds of buttons, we would not be able to crawl through and actually see more content there. On the other hand, you could also implement a View More button, essentially as a link to page two of those results, or from page two to page three. And if it’s implemented as a link, we would follow it as a link, even if it doesn’t have a label that says page two on it. So that’s, I think, the first thing to double-check. Is it actually something that can be crawled or not? And with regards to if it can’t be crawled, then usually, what would happen here is, we would focus primarily on the blog posts that would be linked directly from those pages. And it’s something where we probably would keep the old blog posts in our index because we’ve seen them and indexed them at some point. But we will probably focus on the ones that are currently there. One way you can help to mitigate this is if you cross-link your blog post as well. So sometimes that is done with category pages or these tag pages that people add. Sometimes, blogs have a mechanism for linking to related blog posts, and all of those kinds of mechanisms add more internal linking to a site and that makes it possible that even if we, initially, just see the first page of the results from your blog, we would still be able to crawl to the rest of your website. And one way you can double-check this is to use a local crawler. There are various third-party crawling tools available. And if you crawl your website, and you see that oh, it only picks up five blog posts, then probably, those are the five blog posts that are findable. On the other hand, if it goes through those five blog posts. And then finds a bunch more and a bunch more, then you can be pretty sure that Googlebot will be able to crawl the rest of the site, as well.

To What Degree Does Google Follow the robots.txt Directives?

Q. (42:34) To what degree does Google honour the robots.txt? I’m working on a new version of my website that’s currently blocked with a robots.txt file and I intend to use robots.txt to block the indexing of some URLs that are important for usability but not for search engines. So I want to understand if that’s OK.

• (42:49) That’s perfectly fine. So when we recognise disallow entries in a robots.txt file, we will absolutely follow them. The only kind of situation I’ve seen where that did not work is where we were not able to process the robots.txt file properly. But if we can process the robots.txt file properly, if it’s properly formatted, then we will absolutely stick to that when it comes to crawling. Another caveat here is, that usually, we update the robots.txt files, maybe once a day, depending on the website. So if you change your robots.txt file now, it might take a day until it takes effect. With regards to blocking crawling– so you mentioned blocking indexing, but essentially, the robots.txt file would block crawling. So if you blocked crawling of pages that are important for usability but not for search engines, usually, that’s fine. What would happen, or could happen, is that we would index the URL without the content. So if you do a site query for those specific URLs, you would still see it. But if the content is on your crawlable pages, then for any normal query that people do when they search for a specific term on your pages, we will be able to focus on the pages that are actually indexed and crawled, and show those in the search results. So from that point of view, that’s all fine.

If 40% Of Content Is Affiliate, Will Google Consider Site a Deals Website?

Q. (53:27) So does the portion of content created by a publisher matter, and I mean that in the sense of affiliate, or maybe even sponsored context. Context is a Digiday newsletter that went out today that mentioned that publishers were concerned that if you have, let’s say 40%, of your traffic or content as commerce or affiliate, your website will become or considered by Google, a deals website, and then your authority may be dinged a little bit. Is there such a thing that’s happening in the ranking systems algorithmically?

• (54:05) I don’t think we would have any threshold like that. Partially, because it’s really hard to determine a threshold like that. You can’t, for example, just take the number of pages and say, this is this type of website because it has 50% pages like that. Because the pages can be visible in very different ways. Sometimes, you have a lot of pages that nobody sees. And it wouldn’t make sense to judge a website based on something that, essentially, doesn’t get shown to users.

Sign up for our Webmaster Hangouts today!

Search Console Says I’m Ranking #1, but I Don’t See It.

Q. (00:56) The question is about the average position in Google Search Engine. A few days ago, I realised that over 60 queries show that I am in position #1. I still don’t understand how that works because some of the keywords I am searching on search results do not appear first. I have seen some of them coming from the Google Business Profile, but some of them are not really appearing. And I don’t really understand why it says that the average position is #1. What does it really mean? Does it sometimes say I am on position #1 and it takes time to appear, or how does it really work?

• (54:13): The average top position is based on what was actually shown to users. It is not a theoretical ranking where you would appear at #1 if you did the right kind of thing. It is really: “We showed it to users, and there was ranking at #1 at that time”. Sometimes it is tricky to reproduce that. Which I think kind of makes it a little bit hard in that, and sometimes it is for users in a specific location or the search in a specific way – may be just mobile or just on the desktop. Sometimes it is something in image one box or knowledge graph or local business entry that you mentioned. All of these things also paces where could be the link to your website. And if you see something listed ranking #1, usually what I recommend doing is trying to narrow that down into what exactly was searched by using the different filters in the Search Console and figure out which country that was in and what type of query was it – was it mobile or on a desktop and see if you can try to reproduce it that way. Another thing that I sometimes do is look at the graph over time, so specifically for the query, your keywords’ average position ranking #1 but the total impressions and the total clicks don’t seems like make much sense, and you think that a lot of people are searching, but you should expect to see a lot of traffic, a lot of impressions at least, that also could be a sign of that you were shown at position #1 in whatever way, but you aren’t always shown that way. That could be something very temporary or something that fluctuates a little bit, anything along those lines. But it is not the case that it is a theoretical position. You are shown in the search results, and when we were shown, you were shown like this.

Why Is My Homepage Ranking Instead of an Article?

Q. (05:55) When I am checking the Search Console for the pages, the only page that is actually shown is mostly the home page. Some of the queries of home pages are ranking have their own pages.

• (6:21) I guess it is just our algorithms that see your home page as a more important page overall. Sometimes it is something more if you continue to work on your website and make it better and better, it is easier for our systems to recognise that actually there is a specific page that is more suited for this particular query. But it is a matter of working on your site. Working on your site shouldn’t be something that just generates more and more content than you actually creating something better. And that could be by deleting a bunch of content and combining things together. So that is improving your website isn’t the same as adding more content.

Does Locating a Sitemap in a Folder Affect Indexing?

Q.  (07:11) We have our sitemaps in the subfolder of our website. I have noticed recently that a lot more pages say ‘indexed’ but not ‘submitted’. Do you think that might be due to moving the sitemaps into the subfolder? We used to have them in our ‘root’ but due to technology change, we had to move them.

• (7:30) The locations of the sitemaps shouldn’t really matter. It is something we can put in a subdirectory or subdomain. It depends on how you submitted the sitemap file. For example, list it in your ‘robots.txt’ file. You can put it anywhere; it doesn’t matter.

I Have a New Blog, No Links. Should I Submit in Search Console?

Q. (07:59) I am new to blogging and starting a new blog. It has been an amazing experience to start from scratch. Brand new blog, no links to it. Would you recommend that you submit URLs as you published them using the Google Search Console and then request indexing for new blogs that have no links to them, or there is no point, and it is not really helpful?

• (8:30) It is not that there is any disadvantage to doing that. If it is a new site that we really have absolutely no signals, no information about it at all, at least telling us about URLs is the way of getting down the initial foot in the door, but it is not a guarantee that it will pick that up. That is something where you probably know someone else who is blogging, and you can work together and maybe get the link to your site. Something along those lines probably would do a lot more than just going to Search Console and saying, ‘I want this URL indexed immediately’. 

How Fast Does the Links Report Update?

Q. (09:22) How long does it typically take for Google to add the links on a brand new blog on the Google Search Console ‘Links’ report?

• (9:53) A lot of the reports on Search Console are recalculated every 3-4 days. In terms of about a week, you should probably see some data there. The tricky part is we show the samples of the links on your website, and it doesn’t mean that we immediately populate that one links that we found to your side. It is not a matter of months, and it is not a matter of hours as it is in the performance report. From one up to a week is a reasonable time.

Should the Data in Search Console Align With Analytics?

Q. (10:56) Google and my website have stated that the data we are getting from Google Analytics and Google Search Console will not match exactly, but they will make sense directly. This means for your organic search. All your clicks always are under the sessions that you get it. Is there any understanding, correct?

• (11:17) I guess it depends on what you have set up and what you are looking at specifically. Suppose you are looking at it at the site level that is about to be correct. If you are looking at it per year level on a very large site, it could happen that those individuals are not tracked in the Search Console, and you would see slightly different changes, or you would see changes over time. Some days it is tracked, and some days it is not tracked, particularly if you have a very large website. But if you look at the website overall, that should be pretty close. Search Console measures what is shown on the search results – the clicks and impressions from there, and Analytics uses Java Script to track what is happening on the website side. Those tracking methods are slightly different and probably have slightly different ways of deduplicating things. So I would never expect two of these two to line up completely. However, overall, they should be pretty close.    

Why Are Disallowed Pages Getting Traffic?

Q. (16:58) And my next question is, in my robots file, what I’ve done is, I have disallowed some of the pages, certain pages that I’ve disallowed. But it is quite possible that Google had probably in the past indexed those pages. And when I have blocked them, disallowed crawling, today, to this date, I see them getting organic sessions. Why is that happening, and how can I fix that? And read there is something called ‘‘noindex’’ directive. But is there the right way to go about it? Or how should I pursue this?

• (17:30) If these are pages that you don’t want to have indexed, then using ‘noindex’ would be better than using the disallow and robots.txt. The ‘noindex’ would be a metatag on the page though. So it’s a robots metatag with ‘noindex’. And you would need to allow crawling for that to happen.

Does Google Use Different Algorithms per Niche?

Q. (23:47) Is it true that Google has different algorithms for the indexing and ranking of different niches? We have two websites of the same type, and we’ve built them with the same process. The only difference is that the two sites are different niches. And currently, one is working while the other one has lost all ranking.

• (24:07) So I don’t think we have anything specific with regards to different niches. But obviously, different kinds of content is differently critical to our search results. And if you look at something like our quality raters guidelines, we talk about things like your money your life sites, where we do kind of work to have a little bit more critical algorithms involved in the crawling, indexing, and ranking of their sites. But it’s not the case that you would say it’s like a bicycle shop has completely different algorithms than, I don’t know, a shoe store, for example. They’re essentially both e-commerce type stores. But the thing that you also mentioned in the question is that these are content aggregator sites, and they’re built with the same process. And some of them do work, and some of them don’t. That, to me, feels like it’s– I don’t know your sites. It feels a bit like low effort affiliate sites, where you’re just taking content feeds and publishing them. And that’s the kind of thing where our algorithms tend not to be so invested in making sure that we can crawl and index all of that content. Because essentially, it’s the same content that we’ve already seen elsewhere on them. So from that point of view, if you think that might apply to your site, I would recommend focusing on making fewer sites and making them significantly better. So that it’s not just aggregating content from other sources, but actually that you’re providing something unique and valuable in the sense that if we were to not index your website properly, then the people on the internet would really miss a resource that provides them with value. Whereas, if it’s really the case that if we didn’t index your website, then people would just go to one of the other affiliate aggregator sites, then there is no real reason for us to focus and invest on crawling and indexing your site. So that’s something where, again, I don’t know your websites. But that’s something that I would look into a little bit more rather than just, oh, “Google doesn’t like bicycle stores; they like shoe stores instead”.

What Counts as a Click in FAQ Rich Snippets?

Q.  (26:23) Two related questions. What counts as a click for an FAQ rich snippet? Does Google ever add links to FAQ answers, even if there isn’t one included in the text?

• (26:29) You link to the help centre article on that, which I think is pretty much the definitive source on the clicks and impressions and position counting in Search Console. In general, we count it as a click if it’s really a link to your website and someone clicked on it. And with regards to the rich results, I can’t say for sure that we would never add a link to a rich result that we show in the search results. Sometimes I could imagine that we do. But it’s not the case that we would say, Oh, there’s a rich result on this page. Therefore we’ll count it as a click, even though nobody clicked on it. It’s really, if there’s a link there and people click on it and go to your website, that’s what we would count. And similarly, for impressions, we would count it as an impression if one of those links to your sites were visible in the search results. And it doesn’t matter where it was visible on the page if it’s on the top or the bottom of the search results page. If it’s theoretically visible to someone on that search results page, we’ll count it as an impression.

Why Do Parameter Urls Get Indexed?

Q. (30:31) Why do parameter URLs end up in Google’s index even though we’ve excluded them from crawling with the robots.txt file and with the parameter settings in Search Console. How do we get parameter URLs out of the index again without endangering the canonical URLs?

•  (30:49) So, I think there’s a general assumption here that parameter URLs are bad for a website. And that’s not the case. So it’s definitely not the case that you need to fix the indexed URLs of your website to get rid of all parameter URLs. So from that point of view, it’s like, I would see this as something where you’re polishing the website a little bit to make it a little bit better. But it’s not something that I would consider to be critical. With regards to the robots.txt file and the parameter handling tool, usually, the parameter handling tool is the place where you could do these things. My feeling is the parameter handling tool is a little bit hard to find and hard to use by people. So personally, I would try to avoid that and instead use the more scalable approach in the robots.txt file. But you’re welcome to use it in Search Console. With the robots.txt file, you essentially prevent the crawling of those URLs. You don’t prevent indexing of those URLs. And that means that if you do something like a site query for those specific URLs, it’s very likely that you’ll still find those URLs in the index, even without the content itself being indexed. And I took a look at the forum thread that you started there, which is great. But there, you also do this fancy site query, where you pull out these specific parameter URLs. And that’s something where if you’re looking at URLs that you’re blocking by robots.txt, then I feel that is a little bit misleading. Because you can find them if you look for them, it doesn’t mean that they cause any problems, and it doesn’t mean that there is any kind of issue that a normal user would see in the search results. So just to elaborate a little bit. If there is some kind of term on those pages that you want to be found for, and you have one version of those pages that is indexable and crawlable and another version of the page that is not crawlable, where we just have that URL indexed by itself, if someone searches for that term, then we would pretty much always show that page that we actually have crawled and indexed. And the page that we theoretically also have indexed, because it has– it’s blocked by robots.txt, and theoretically, it could also have that term in there, that’s something where it wouldn’t really make sense to show that in search results because we don’t have as much confirmation that it matches that specific query. So from that point of view for normal queries, people are not going to see those ‘robotic’ URLs. And it’s more if someone searches for that exact URL, or does a specific site query for those parameters, then they could see those pages. If it’s a problem that these pages are findable in the search results, then I would use the URL removal tool for that, if you can. Or you would need to allow crawling and then use a ‘noindex’ directive, robots.txt directive, to tell us that you don’t want these pages indexed. But again, for the most part, I wouldn’t see that as a problem. It’s not something where you need to fix that with regards to indexing. It’s not that we have a cap on the number of pages that we index for a website. It’s essentially, that we’ve seen a link to these. We don’t know what is there. But we’ve indexed that URL should someone search specifically for that URL.

Does Out-Of-Stock Affect the Ranking of a Product Page?

Q. (37:41) Let’s say my product page is ranking for a transactional keyword. Would it affect its ranking if the product is out of stock?

• (37:50) Out of stock, it’s possible. Let’s kind of simplify like that. I think there are multiple things that come into play when it comes to products themselves, in that they can be shown as a normal search result. They can also be shown as an organic shopping result as well. And if something is out of stock, I believe the organic shopping result might not be shown. Not 100% sure. And when it comes to the normal search results, it can happen that when we see that something is out of stock, we will assume it’s more like a soft 404 error, where we will drop that URL from the search results as well. So theoretically, it could essentially affect the visibility in Search if something goes out of stock. It doesn’t have to be the case. In particular, if you have a lot of information about that product anyway on those pages, then that page can still be quite relevant for people who are searching for a specific product. So it’s not necessarily that something goes out of stock, and that page disappears from search. The other thing that is also important to note here is that even if one product goes out of stock, the rest of the site’s rankings are not affected by that. So even if we were to drop that one specific product, because we think it’s more like a soft 404 page, then people searching for other products on the site, we would still show those normally. It’s not that there would be any kind of a negative effect that swaps over into the other parts of the site.

Could a Banner on My Page Affect Rankings?

Q. (39:30) Could my rankings be influenced by a banner popping up on my page?

• (39:35) And yes, they could be as well. There are multiple things that kind of come into play with regards to banners. On the one hand, we have within the Page Experience Report; we have that aspect of intrusive interstitials. And if this banner comes across as an intrusive interstitial, then that could negatively affect the site there. The other thing is that often with banners, you have side effects on the cumulative layout shift, how the page renders when it’s loaded, or with regards to the – I forgot what the metric is when we show a page, the LCP I think, also from the Core Web Vitals side with regards to that page. So those are different elements that could come into play here. It doesn’t mean it has to be that way. But depending on the type of banner that you’re popping up, it can happen.

Do Links on Unindexed Pages Count?

Q. (49:03) How about you have been linked from some pages, but those pages have not been indexed. But those mentions or the link has been already present on those particular things. So it is still counted just because the page is not indexed, and so those links won’t be counted? Or even if the page is not indexed but if there is a link, those things can be counted as well?

• (49:39) Usually, that wouldn’t count. Because for a link, in our systems at least, we always need a source and a destination. And both of those sides need to be canonical indexed URLs. And if we don’t have any source at all in our systems, then essentially, that link disappears because we don’t know what to do with it. So that means if the source page is completely dropped out of our search results, then we don’t really have any link that we can use there. Obviously, of course, if another page were to copy that original source and also show that link, and then we go off and index that other page, then that would be like a link from that other page to your site. But that original link, if that original page is no longer indexed, then that would not count as a normal link.

What Can We Do About Changed Image Urls?

Q. (50:50) My question is on the harvesting of images for the recipe gallery. Because we have finally identified something which I think has affected some other bloggers and it’s really badly affected us, which is that if you have lots and lots of recipes indexed in the recipe gallery, and you change the format of your images, as the metadata is refreshed, you might have 50,000 of the recipes get new metadata. But there is a deferral process for actually getting the new images. And it could be months before those new images have been picked up. And while they’re being harvested, you don’t see anything. But when you do a test on Google Search Console, it does it in real-time and says, yeah, everything’s right because the image is there. So there’s no warning about that. But what it means is you better not make any changes or tweaks to slightly improve the formatting of your image URL. Because if you do, you disappear.

• (51:39) Probably what is happening there is the general crawling and indexing of images, which is a lot slower than normal web pages. And if you remove one image URL and you add a new one on a page, then it does take a lot of time to be picked up again. And probably that’s what you see there. What we would recommend in a case like that is to redirect your old URLs to new ones, also for the images. So if you do something, like you have an image URL which has the file size attached to the URL, for example, then that URL should redirect to a new one. And in that case, it’s like we can keep the old one in our systems, and we just follow the redirect to the new one.

Does Crawl Budget Affect Image Re-Indexing?

Q. (54:25) Does crawl budget affect image re-indexing?

• (54:31) Yeah, I mean, what you can do is make sure that your site is really fast in that regard. And that’s something in the crawl stats report; you should be able to see some of that, where you see the average response time. And I’ve seen sites that have around 50 milliseconds. And other sites that have 600 and 700 milliseconds. And obviously, if it’s faster, it’s easier for us to request a lot of URLs. Because otherwise, we just get bogged down because we send, I don’t know, 50 Google bots to your site at one time. And we’re waiting for responses before we can move forward.

Sign up for our Webmaster Hangouts today!

Changing Page Title and Descriptions

Q.  (1:01) A few days ago we optimised a page title and description, after that we saw the title and description changed when using the site to search in Google. After a long while, the title and description have been the one in the first place. In this case, does Google think the formal title and description are better than the one we optimised and would there be any other possible reasons that may cause this?

• (1:36) I would not necessarily assume that if Google changes it to something that Google thinks it’s better, then you should use that title too. It is more than our systems have selected a different title. And usually, the title is picked on a per-page basis.

Best Practice to Have Content Display

Q. (4:29) Every franchisee has a microsite after the domain, like domain/branch1 and their corresponding URLs. What would be the best practice to have content displayed well? It is really hard to have 100 branches and 100 different contents. Is there a problem from the perspective of Google? What does Google think about it?

• (05:48) Guidelines, usually doorway pages are more about swapping out one word on a page essentially, where the rest of the content is the same. If you have a specific service you create pages for every city nearby or every street or every region nearby, and you create hundreds of these pages that are essentially just driving traffic to the same business. I think franchises probably that’s a lesser issue because these are essentially separate businesses. It is essential to separate the businesses.

Decrease in Getting Articles Indexed

Q. (09:25) I work in a content creation agency. And we work with different blogs. And the agency has been running for more than 10 years. And we’ve never had problems with getting our articles indexed. But for the last six months, all of the new blogs especially are having big problems getting their articles indexed. So recently, we’re working on organic traffic. And if we’re not getting our articles indexed, then we can’t work on that or optimise any content. I was just wondering if there’s any specific reason, maybe there’s been some sort of change, or if we just have to wait and see what happens? Because we have done a lot of technical revisions of the site amps and everything. But we have just noticed a decrease in articles getting indexed.

• (10:14) It is hard to say in general without being able to kinda look at some of these example sites. If you have examples of pages that you have that are not super fresh, a couple of weeks old that are still not getting indexed I would love to get some of those examples. In general, though, I think what I see with a lot of these questions that tend to come up around my content not being indexed is that from a technical point of view, a lot of these sites are really good, that they’re doing the right things about site maps, the internal linking is set up well. It is more than on our side from a quality point of view, it is something where it feels like the bar is slowly going up and that more people are creating content that is technically okay but not from a quality point of view.

SEO Tool in Duplicate Content

Q. (15:40) We have had a news publishing website since 2009. We post articles related to recipes, health, fitness and stuff like that. We have articles that are considered personal SEO tools as duplicate content. We tend to recreate another version of this recipe or tweak it around maybe sugar-free or salt-free everything related to that. What the SEO tool suggested is to remove it because none of the duplicate content is being ranked or indexed by Google. What is the solution for this?

• (16:53) To make assumptions with regards to what Google will do and what will happen. And sometimes those assumptions are okay and sometimes they are not correct. This kind of feedback from SEO tools is useful because it is still something that you can take a look at it and make a judgment call. You might choose to say, I’m ignoring the tool in this case and I’m kind of following the guidance in a different case. If you are seeing something even from a very popular SEO tool that tells you, you should disavow these links and delete this content. Always use your judgment first before blindly following that.

Ranking Service Pages to Get More Leads

Q.  (23:16) I’m currently working on a website that is based in India and we get leads from all over India. We can provide services all over the world, but first I want to rank my service pages to get more leads from the USA. Can you help me know what things I can do so that I can rank top of my competitors?

• (24:16) If you’re going from a country-specific website to something more global then it helps to make sure that from a technical point of view, your website is available for that. Using a generic top-level domain instead of a country-specific top-level domain can help. Any time when you go from a country-level website to a global-level website, the competition changes completely.

Getting the Best Approach for Client Credits

Q. (27:18) We are working with an eCommerce client, and it is an open-source online store management system. Their blog is WordPress. The main URL is example.com, whereas the blog is blog.example.com. What would be the best approach for this client to get credit from the blogs?

•  (28:06) Some SEOs have very strong opinions about subdomains and subdirectories and would probably want to put this all on the same domain, from our point of view you could do it like this as well. This setup would be fine. If you did want to move it into the same domain, then practically speaking, that usually means you have to do some technical tricks, where essentially you proxy one subdomain as a subdirectory somewhere else. You have to make sure that all of the links work.

Describing Products Using Alt Text

Q. (37:22) Should I write alt text for products for an e-Commerce site since there is already text beneath that describes the product?

• (37:35) The alt text is meant as a replacement or description of the image. That is particularly useful for people who cannot look at individual images, who use things like screen readers. It also helps search engines to understand what this image is about

Using Alt Tags for Longer Text With Decoration

Q.  (40:07) Would you use alt tags for images that use only decoration within the longer text? How would you treat those mostly stock images?

• (40:28) From an SEO point of view, the alt text helps us to understand the image better for image search. If you do not care about this image for image search then that is fine. You would focus more on the accessibility aspect there rather than the pure SEO aspect. It is not the case that we would say a textual webpage has more value. It is just well, we see the alt text and we apply it to the image.

Added Links in an Underscore Cell

Q.  (44:09) So one of my technical members has added the links in the form of an underscore target or underscore equally to blank. How are Google bots able to crawl these things? Do they understand that there are links added to this particular node that typed something over there?

• (46:15) I think we just ignore it. Because it makes more sense from a browser point of view what happens. The target attribute refers to how that link should be open. If you have a frame on a page then it will open that link in a frame. What we focus on is if there is a href value given there, then essentially that link goes to the same page and we ignore that.

Home Page Disappearance and Ranking

Q.  (50:33) For every query there is only the — know my home page is getting ranked, and all of the other pages are ignored, suddenly disappear from this. How’s Google treating it?

• (51:19) Sometimes we think the home page is a better match for that specific query and it could be that some of the information is on the home page itself. The more detailed page is seen as not such a good page from our point of view. It is something where you can experiment with removing some of that information from the home page.

Sign up for our Webmaster Hangouts today!

Page Disappearing From Top Results

Q.  (0:35) It is related to the pages which are 8 at 10 or 12 and suddenly they disappear from the sub. Is it really like Google is still trying to understand the overall website quality? Or there could be some other technical issues? Because when the pages are live, they are in the sublinks. When they get indexed they are placed at like 8th position, 10 or 12 or 15., so being very close or near to the first page.

• (01:12) If you’re creating something new or if you are just updating something existing like if you have a page where you’re just updating the prices then you don’t need to make a new page for every price change that you make. You just update those prices. On the other hand, if you’re providing information then that feels like something that should live on a separate URL where people can go directly to that specific place of information. The advantage of keeping the same URL is that over time it builds a little bit more value and then people understand that it’s actually the place to go for this piece of information. For example, every day for a new price you would create a new page then if people search for, what is the current price for this product, then they’re going to find some of these, but it’s going to be unclear to us which one of them they should show. On the other hand, if you have one page, where you just update the current price, then we know, for the price, this is the page.

Traffic and Engagement and Its Impact on Rankings

Q. (6:57) We recently added a page for our site that is consistently driving significant traffic at levels we’ve never seen before. So it’s really through the roof. Can a single page with extremely high engagement and traffic have an influence on the domains as a whole? Do these signals trickle to other pages on the site and play a positive role at that domain level?

• (07:23)  I don’t think we would use engagement as a factor. But it is the case that, usually, pages within a website are interlinked with the rest of the website. And through those internal links, across the website, we do forward some of the signals. So if we see that a page is a really good page and we would like to show it in search a lot, maybe it also has various external links going there, then that gives us a lot of additional contexts about that page. And we can kind of forward some of that to the rest of the website. So usually that’s a good thing.

Core Web Vitals for Lower Value Pages Drag the Site Down?

Q. (8:37) We prioritise our high-search pages for product improvements like anyone else would do. We prioritise our high-search pages for product improvements like anyone else would do. Can a subset of pages with poor LCP or CLS, say only the video page on the site that aren’t the main or secondary or even tertiary search-traffic-driving pages on the site, impact the rest of the site’s overall Core Web Vitals score? So what I mean by this is, can a group of bad pages with little search traffic in the grand scheme of things actually impact—drag the overall score of the site down? And do we need to prioritise those bad pages even though they aren’t high-traffic pages?

• (09:14) Usually, that wouldn’t be the problem. There are two aspects there. On the one hand for the Core Web Vitals, we look at a sample of the traffic to those pages, which is done through, I don’t know, the Chrome User Experience Report functionality. I believe that’s documented on the Chrome side somewhere. It’s essentially a portion of the traffic to your website. That means that, for the most part, the things that we will look at the most are the pages that get the most visits. So if you have random pages on the side that nobody ever looks at and they are slow, then those wouldn’t be dragging your site down.

Internal Links Will Play a Role in Indexing Priority

Q. (16:39) We found that many category pages didn’t get indexed faster than other specific pages like product pages. And these category pages are in a formal place like they are close to the home page. Wondering if the theory is correct?

• (17:09) I think a difficult part there is also that kind of linked closer from their home page is a general rule of thumb. But it doesn’t have to be the case. Because we have a lot of systems also in play to try to figure out how often we should recrawl a page. And that depends on various factors. It also depends on how well it’s linked within the website, but also based on what we expect will happen with this page, how often do we think it will change, or how often do we think it will change significantly enough that it’s worthwhile to recrawl and re-index it.

Flat Hierarchy vs. URL Hierarchy

Q.  (18:13) Can you comment on the flat hierarchy versus a strict kind of URL? Because there is no such thing as a closer flat structure.

• (18:58) We don’t care so much about the folder structure. We essentially focus on the internal link. And it also kind of links from the home page, not links to the home page. So from that point of view, if you have a URL structure that doesn’t have any subdirectories at all, we still see that structure based on the internal linking. And a lot of times, the architecture of the website is visible in the URL structure, but it doesn’t have to be the case.

Website Quality

Q. (22:28) How do you improve the perceived quality of a whole website at Google side?

•  (22:53) I think we’ve given some types of things that you can focus on with the reviews updates think we’ve given some types of things that you can focus on with the reviews updates. that we’ve done for product reviews. Some of that might apply. But I don’t think there is one solution to improving the overall quality of any larger website. And especially on an e-commerce site, I imagine that’s quite tricky. There are sometimes things like working  to improve the quality of the reviews that people leave if it’s user-generated reviews, for example, making sure that you’re highlighting the user reviews, for example.

Crawl Statistics Report

Q. (25:47) We have looked at the crawl stats reports on the Search Console and have been trying to identify if there might be some issue on the technical side with Google crawling our website. What are some of the signals or things to identify that will point us to if Google is struggling to crawl something or if Googlebot is distracted by irrelevant files and things that it’s trying to index that are irrelevant to us?

• (26:34) Crawl reports will not be useful in that case. You are looking at an aggregate view of the crawling of your website. And usually, that makes more sense if you have something like, I don’t know, a couple hundred thousand pages. Then you can look at that and say, on average, the crawling is slow. Whereas if you have a website that’s maybe around 100 pages or so, then essentially, even if the crawling is slow, then those 100 pages, we can still get that, like once a day, worst case, maybe once a week. It’s not going to be a technical issue with regards to crawling. It’s essentially more a matter of understanding that the website offers something unique and valuable that we need to have indexed. So less of an issue about the crawling side, and more about the indexing side.

Google Search Console Not Matching Up to Real Time Search

Q. (30:09) On 1st March, my website home page, was gone from the search results, completely gone from there. OK, it’s a kind of Google thing. The home page was not in the search results. But the interesting thing, in the Google Search Console, for every keyword, Search Console is saying I’m still ranking in the first position for every keyword that was ranking before 1st March. But the significant amount of impressions and clicks had gone. About 90% had gone. Rankings and the CTR are the same. For about one week, I tried everything, but nothing works out for me. Google Search Console is saying I am ranking in the first position still.

• (32:06) Try to figure out whether it’s a technical issue or not. And one way you could try to find out more there is to use the URL inspection tool. it’s indexed but it’s not ranking, at least when you search. And the thing with, I think, the performance report in Search Console, especially the position number there, that is based on what people saw. 

Service and Recipe Website

Q.  (41:12) So on the service website, I have different FAQs based on different city pages for my services. Do I have to create separate pages for FAQs, or can I just add them to the same city pages? And from our point of view, you can do whatever you want?

• (41:35) If you make a separate page with the FAQ markup on it and that separate page is shown in the search results, then we can show it. If that separate page is not shown, then we wouldn’t be able to show it. And let’s see, I think the same applies to the recipe website example there. We’ve seen that most recipe websites are not providing very useful information in my country, and we try to change that by providing more useful information and going to the point of adding FAQs to every recipe.

Shortcut Handlings

Q.  (48:42) On our website, we have a lot of shortcuts like the English version will be EG, for example. How does Google handle that?

• (48:58) We don’t do anything special with those kinds of things. We essentially treat them as tokens on a page. And a token is essentially a kind of like a word or a phrase on a page. And we would probably recognise that there are known synonyms for some of these and understand that a little bit, but we wouldn’t do anything specific there in that we’d have a glossary of what this abbreviation means and handle that in a specific way. Sometimes plays a role with regards to elements that do have a visible effect on schema.org the requirements are sometimes not the same as in Google Search. So it will have some required properties and some optional properties and it kind of validates based on those Google Search sometimes we have a stricter set of requirements that we have documented in our Help Center as well. So from our point of view, if Google doesn’t show it in the search results then we would not show it in the testing tool and if the requirements are different, Google’s requirements are stricter and you don’t follow these guidelines then we would also flag that as I don’t know.

Sign up for our Webmaster Hangouts today!

Content Project

Q. If I have content that is updated daily, like the rates of cryptocurrency or similar. What is the best way to do it? Is it to create a new article every day? Or is it to update existing articles daily, so the URL stays the same?

• (01:12) If you’re creating something new or if you are just updating something existing like if you have a page where you’re just updating the prices then you don’t need to make a new page for every price change that you make. You just update those prices. On the other hand, if you’re providing information then that feels like something that should live on a separate URL where people can go directly to that specific place of information. The advantage of keeping the same URL is that over time it builds a little bit more value and then people understand that it’s actually the place to go for this piece of information. For example, every day for a new price you would create a new page then if people search for, what is the current price for this product, then they’re going to find some of these, but it’s going to be unclear to us which one of them they should show. On the other hand, if you have one page, where you just update the current price, then we know, for the price, this is the page.

Heading Hierarchy

Q. Can Google separate the heading from the main content?

• (05:44) Usually, that should just work now. In HTML 5 there’s also an element that you can use for the header and footer. Within those elements, you can have your headings again essentially. Even from a semantic point of view, you could set that up properly. It should be possible and I don’t think it would cause any problem on Google’s side either way. What usually happens with these kinds of common elements is we can recognise them across the site because they’re more or less repeated across the site. Then we can try to de-emphasise them when it comes to Search because we realise it’s the same text on all of these pages. We will just pick one of these pages to rank essentially for the text in their footer section.

Testing Tools

Q. Google Search Console is throwing an error with this error in a required structured data element. But when I check the same on validator.schema.org, it does not show any warnings or any errors. So the first question is, is it the right site to check the AMP implementation of a web page? And if so, there is a contradiction. What should be the step over?

• (08:12) Testing tools are for slightly different purposes. That’s probably why you’re seeing that difference. The testing tool in schema.org is more about understanding schema.org markup in general.

Internal Links in the Footer Section

Q. Is it likely to be seen as problematic by Google because the links are not contextual?

• (09:49) Most parts that wouldn’t cause any problems. I would see this more as these are links on these pages. They’re normal internal links. For example, if you have a larger website and essentially every page is linked with every other page, there’s no real context there. So it’s hard for us to understand what the overall structure is, and which of these pages is more important. Because if you’re linking to everything, then it’s like everything is not important. So that’s the kind of element that I would watch out for. Whether or not they’re in the footer from my point of view is irrelevant. If they’re generated by a plug-in or added manually, I don’t think that matters either. I would just kind of watch out, from a structural overall, based on the requirements that schema.org has. And the testing tool in Search Console is focused purely on what we can pull out of the structured data and use to show in a search feature. It is really focused on the Search part of that story. And within Search, we only use a small part of the schema.org markup. And sometimes we have slightly different requirements that maybe we require a specific element more than the base schema.org markup would require.

Page Speed

Q. We did one experiment on our website, wherein we improved the page speed of our website like we moved from somewhere close to 30, and we are now at 45, 50 on our [INAUDIBLE] PageSpeed Insights score. And in the next couple of days, we saw a massive improvement in our ranks. So I just wonder, can this be like– this correlation, is there a possibility that it is true or there could be other external factors that can also be impacting so quickly in two days and we are seeing a jump?

• (11:05) The speed aspect is something that we have to pick up through the Chrome User Experience Report data and that takes a while to collect and to be aggregated. That’s not something you would see within a couple of days.

Sitemap

Q. When Googlebot crawls the site map in the server address, does it first crawl the submitted sitemaps in the backend, or does it go directly to our server?

• (12:11) We don’t crawl them at the same time. It’s not that there’s a specific order form. It’s more that for individual sitemap files we try to figure out how often they change and how often they give us useful information. And based on that, for individual files, we will crawl them at different rates. So it can happen that one in your Search Console account is crawled very frequently, but also one that you submit directly is crawled frequently, and maybe another in Search Console is crawled very infrequently. So it doesn’t depend on where you submit it.

Not Showing Up for Branded Keyword

Q. We’ve got two sites. One’s a global site and one’s Australia’s site. Suddenly we were the Australia site ranking for our branded term for our number one position for two, three months. And suddenly, in the last one week, it was replaced by the global website, .com website, for a couple of days. And I just wanted to understand why that could be the case?

• (18:50)  It’s hard to know without looking at the sites. But, I mean, if these are two sites that are essentially part of the same bigger group, we can switch between which one we would show for a ranking like that. With annotations, you can give us a little bit more information on how you want us to treat that pair of pages. Geotargeting can help a little bit. But then, it can happen that we show a global version of a page in a country where you actually also have a local version of the page, perhaps just because the global version is so much stronger than the local version.

Category Pages Update

Q. We run an eCommerce website and we are now in a stage where we want to make major updates to our category pages. One draft wants—or in one draft we want to get rid of the product listings. So you have the product listing with the faceted search where you can filter for the products you are looking for. My question is when we remove the whole products listing of category pages would we have a disadvantage in the ranking because first of all the other competitors have these kinds of product listings? Second, my guess is this is such an established element like for eCommerce pages that the users expect something like this.

• (23:05) From an SEO point of view. I think there are different things you would want to watch out for, which probably you will so that we can still find all of the individual products that we have clean links to there. But if you’re just kind of redesigning this kind of a category page and making it look more like an informational page, I wouldn’t expect my problems with that. From Google’s point of view, you’re just changing the design.

Breadcrumb Setup

Q.  If you have structured data for a breadcrumb is internal linking still important for SEO?

• (25:27) It’s something where internal linking is supercritical for SEO. I think it’s one of the biggest things that you can do on a website to kind of guide Google and guide visitors to the pages that you think are important. And what you think is important is totally up to you. You can decide to make things important where you earn the most money, or you can make things important where you’re the strongest competitor, or maybe you’re the weakest competitor. With internal linking, you can kind of focus things on those directions and those parts of your site. And that’s not something that you can just replace with structured data.

Multiple Product Schemas

Q. For product listing can we implement multiple product schemas on the product listing page?

• (30:01) From Google’s point of view I don’t think you should be doing that, at least the last time I checked the policies around structured data because for product structured data, we really want that to apply to the primary element of the page. And if you have multiple products on a page, it’s not that one of them is the primary element of the page. You should not use multiple products structured data elements on a category page or something like that.

Sitemap Renovate

Q. If we have a really huge website with millions of URLs, and, right now, we are currently—the sitemaps are being renovated. And our IT team is considering storing the new files, as in the new sitemap files, in our cloud service. That means from example.com/sitemaps to cloud.com/sitemaps, we are wondering, is that a problem if we store the sitemaps in the cloud? And if that’s not a problem, shall we also create a permanent redirect for the old URL for this example.com/sitemap, or how should we plan the move?

(47:13) It’s definitely possible to host the sitemap file somewhere also. There are two ways that you can do that. One is if you have both of those domains verified in Search Console, then that works. The other way is if you submit it with the robots.txt file, where you 

specify the sitemap colon and then the URL of the sitemap. That can also go to a different domain. if you have a separate server where you’re creating sitemap files, or if you have a staging setup where it crawls and checks the files and then creates a sitemap file somewhere else, that would all work. I would also redirect the old sitemap file to the new location just to be clean, but probably even if you just delete the old sitemap URL and make sure to submit the new one properly, then that should just work. What might be a little bit tricky.

Sign up for our Webmaster Hangouts today!

Sub-Feature Which Is Present in Google

Q. If a link for any of [Big Query] if a particular URL has been ranked at People Also Ask,  is it counted as a click or impression? And how that feature has been added to the Search Console part is it tracked or monitored? If the users click on some other questions then your query gets added to that particular list? How is this thing monitored or calculated?

• (01:39) Whenever there is a URL shown in the search results, we do try to show that in the Search Console. With the People Also Ask sections, if you expand it then it shows a link to your site with that question and the snippet there. That means whenever someone expands that People Also Ask section and sees the URL for your site, then that is counted as an impression and it doesn’t matter if you’re kind of the first one in the People Also Ask, or if people clicked 15 different ones and then suddenly yours shows up that would be counted as an impression. And also, when a click happens, that gets counted as a click. The ranking, the position is based on the overall position of that group of elements. The Search Console — the only thing to watch out for so the performance reports shows the ranking of the topmost items. So if different URLs are listed multiple times in a search page results, it tracks the position of the first one.

Sub-Domain for Google Search

Q. When you have subdomains it could be a blog or a help forum or resources, how good is Google to recognize this subdomain it’s not a different one but it’s a subsidiary of this particular domain?

• (04:34) Specifically around the links there, I don’t think we would differentiate that much. It’s not that we would say, this is an internal link or an external link. It’s essentially just we found this link from this one page. From our point of view, those are just pages essentially. We don’t say this is a home page or this is kind of a product page or category page. We essentially say just this is a page and this is what it’s relevant for.

Search Console for New Website

Q. In the Search Console I do see at least one link from the former owner that’s still there. Does that mean that this link would actually still count as Search Console does show it?

• (06:27) I don’t know if it counts. But the important part  with Search Console and the link report there’s we try to show all the links that we know of to that site. It’s not a sign that we think these are important links or that they count. So, in particular things like no follow links they would still be listed. Disavowed links would still be listed. Links that we just ignite for other reasons they could still be listed as well. So just because it’s listed doesn’t mean it’s actually a relevant or helpful link on the site.

Building Mobile Site on Amp

Q. Will it be still cached by Google the same way as a site with an AMP version would be and it would appear in search results in the Google Cache Viewer? What happens if there is an error that invalidates the page? Will it just not be shown at all in the search results?

• (07:37) So I think you need to watch out for the difference between having pages that are purely on AMP and pages that are kind of this paired setup. And if a page is purely on AMP from our point of view it’s essentially a normal HTML page. It can also be a valid AMP page. And if it is a valid AMP page, then we can use the AMP cache and all of the AMP functionality in search as well.

Internal Links in Footer Section

Q. Is it likely to be seen as problematic by Google because the links are not contextual?

• (09:49) Most parts that wouldn’t cause any problems. I would see this more as these are links on these pages. They’re normal internal links. For example if you have a larger website and essentially every page is linked with every other page, there’s no real context there. So it’s hard for us to understand what the overall structure is, which of these pages is more important. Because if you’re linking to everything, then it’s like everything is not important. So that’s the kind of element that I would watch out for. Whether or not they’re in the footer from my point of view it is irrelevant. If they’re generated by a plug-in or added manually, I don’t think that matters either. I would just kind of watch out, from a structural point.

Hidden and Invisible Text

Q. The support doc about the topic addresses the intent to deceive or trick bots by including excessive keywords to establish tropical relevance. Is all hidden text against webmaster guidelines?

• (14:36) I don’t think that would be problematic. Hidden text, from our point of view, is more problematic when it’s really about deceiving the search engines with regards to what is actually on a page. So the extreme example would be that, you have a page about shoes and there’s a lot of hidden text there that is about Olympics or something like that. Then suddenly your shoe page starts ranking for these Olympic terms but when a user goes there there’s nothing about the Olympics and that would be kind of problematic from our point of view. We do a reasonable job in recognizing hidden text and trying to avoid that from happening but that’s kind of the reason we have this kind of element in the webmaster guidelines.

Page Performance

Q. We want to improve page performance with some metrics like LCP,FID. We want to change the way of loading description and some reviews content from synchronous to asynchronous on user guide. But for the Googlebot side this content will still be synchronous. Do you think when we’re doing this, it will have any impact to the site from the SEO side?

• (24:16) That’s essentially similar to the dynamic rendering setup that we used to recommend, where you use JavaScript for the client side and you statically render the content for Googlebot and other search engines from our Google it is the same content you’re just delivering it in a slightly different way that’s perfectly fine.

Indexing, Image File Names & Ranking

Q. Do you care about image search? Will image alt text and captions be sufficient for Google to understand without an appropriate image, file name and title?

• (24:59) We’re using an intelligent CDN provider which has been replacing the image file names with unique numbers and noticed that all the images are 404s in the Search Console. Disabling the CDN would significantly degrade overall site performance. There are also two things that I would look at, one hand if these are images that you need to have indexed in image search then you should definitely make sure that you have a stable file name for your images. Kind of the most important element here you don’t mention that these numbers or URLs change but sometimes these CDNs essentially provide a kind of session-based ID for each image. If the image URL changes every time we crawl then essentially we’ll never be able to index those images properly. Mostly because for images we tend to be a little bit slower with regards to crawling and indexing. So if we see an image once and we say we will just drop that image from our search results from the image ranking and essentially we’ll this image that we thought was here is actually no longer here anymore. For web search, we don’t need to be able to crawl and index the images because we essentially just look at the web pages. If all of the images were 404 all the time or blocked by robots.txt we would still treat that page exactly the same as if we were able to index all those images.

Search Results

Q. We’re still getting flagged for explicit content. Is there a way to get reconciled via Google? How do we go about this?

• (28:22) Most part, this kind of understanding of which content we would filter by SafeSearch is handled automatically. If we recognize that a site has changed significantly then we will treat it differently in search results as well. We will remove that SafeSearch bit for the site or for those pages depending on when you made this change. It might be that you just need to be a little bit more patient and it’ll settle down properly.

Same Keyword on Two Different Pages

Q. Is it okay to target the same main keyboard on those two different pages?

• (29:51) Whatever keywords that you want from our point of view we’re not going to hold you back. There are guidelines that you should not do this but it’s more that if you have multiple pieces of content that are ranking for the same query with the same intent then you’re essentially kind of diluting the value of the content that you’re providing across multiple pages. That could mean that these individual pages are not that strong when it comes to competing with other people’s websites. If you have two pages and they’re both targeting the same keyword and they have different intents then that seems kind of reasonable because people might be searching for that keyword with extra text added for one intent and extra text added for other intent. They’re essentially unique pages.

SEO Rankings, Filtering & Sorting

Q. How does Google treat these pages within the website? How do the search results affect the overall ranking? Is it enough to submit sitemaps for ranking? Or we should take additional consideration to help Googlebot to gather all of the returnable URLs?

• (37:58) The last question, I would not rely on sitemaps to find all of the pages of your website. Sitemaps should be a way to give additional information about your website. Internal linking is super important and something you should definitely watch out for. Make sure that you set things up when someone crawls your website. They’re able to find your content and not rely on the sitemap.

Poor Core Web Vitals

Q. My website had a drop in visitors due to poor Core Web Vitals. What page experience ranking to desktop? How important is it compared to the other ranking factors?

• (42:40) The Page Experience ranking factor is essentially something that gives us a little bit of extra information about these different pages that could show up in the search results in situations where we have strong, clear kind of intent from the query where we can understand that they really want to go to this website then from that point of view we kind of can ease off using Page Experience as a ranking factor. If all of the content is very similar in the search results page then probably using Page Experience helps a little bit to understand which of these are fast pages or reasonable pages with regards to the user experience. Less reasonable pages to show in the search results. With that kind of situation helps us with the desktop rollout, I believe this is going to be a slower rollout over the course of something like a month which means you would not be seeing a strong effect from one day to the next but rather you would see that effect over a period of time.

Ranking Signals

Q. How important is it compared to other ranking signals?

• (44:52) Websites would not see a big visible change when it comes to the Core Web Vitals. Even if your website goes from being kind of reasonable to being in that poor bucket in the Core Web Vitals from one day to the next I would not expect to see that as a kind of a giant change in the search results. Maybe changing a few positions seems kind of the right change there. But I would not see it as a page going from ranking number 2 to ranking number 50. If you are seeing a drastic change like that I would not focus on purely Core Web Vitals. I would step back and look at the overall picture and see what else could be involved and try to figure out what you can do to improve things overall.

Validating & Building Schema

Q. I’m building out schema and doing it exactly like it should be done.But when I go over the Rich Results Test for Google it doesn’t validate. When I go to Google Search Console it’s not showing up. What should I do to actually get this to validate?

• (49:00) There are two main things that play a role. ONe hand the validator schema.org is set up to validate theoretical schemes that you can provide with schema.org The validator in Search Console is based purely on functionality that has visible changes in Google Search and that’s usually very small subset of the bigger scheme.org set of things that you can mark up. Example you are making things up that don’t have a visible effect in the search results in terms of maybe it starts, shows video or something like that then that would be something where Search Console would say I don’t see anything here which might fall into that category that you’re seeing there. Sometimes plays a role with regards to elements that do have a visible effect on schema.org the requirements are sometimes not the same as in Google Search. So it will have some required properties and some optional properties and it kind of validates based on those Google Search sometimes we have stricter set of requirements that we have documented in our Help Center as well. So from point of view, if Google doesn’t show it in the search results then we would not show in the testing tool and if the requirements are different, Google’s requirements are stricter and you don’t follow these guidelines then we would laso flag that as I don’t know.

Sign up for our Webmaster Hangouts today!

Indexing API

Q. Using API for things other than job posting and broadcasting events doesn’t bring any benefits.

• (06:49) The person asking the question is interested whether he can use API for pages different than job posting and broadcasting events, like, for example, news articles and blogs. John says that people try doing that, but essentially what Google has documented is what it uses the API for. If there isn’t content that falls into those categories, then the API isn’t really going to help. But trying that won’t negatively affect the content.

Unlinked Brand Mention

Q. Unlinked brand mention doesn’t really affect SEO.

• (12:01) When there is an article, that mentions the website without linking to it, it is not a bad thing. It is only a little inconvenient for users, because they have to search for the website being mentioned. But, otherwise, John says he wouldn’t assume there’s some SEO factor that is trying to figure out where someone is mentioning the website’s name. 

User Reviews and Comments

Q. Non-spammy comments help Google understand the page a little better.

• (12:58) A really useful thing about the user comments is that oftentimes people will write about the page in their own words, and that gives Google a little bit more information on how it can show this page in the search results. From that point of view, comments are a good thing on a page. Obviously, finding a way to maintain them in a reasonable way is something tricky, because people also spam those comments, and all kinds of crazy stuff happens in them. But overall once a way to maintain comments on a web page is found, that gives the page a little bit more context and helps people who are searching in different ways to also find this content. 

SSL Certificates

Q. Any kind of valid SSL certificate works fine for Google.

• (14:03) Different types of SSL certificates are not important for SEO, and just free SSL certificates are perfectly fine. The different types of certificates is more a matter of what is wanted with this certificate. From Google point of view, it just watches out for whether the certificate is valid or not.

Multiple Job Postings

Q. Posting the same job posting in different subdomains from the same root domain is fine.

• (14:47) John assumes that posting the same job posting in different subdomains with the same job posting data structure is perfectly fine, because it’s very common to have the same job posted on different websites, and those different websites could have structured data on them as well. From that point of view, just having it posted different times on the same website or in different subdomains should be perfectly fine as well. However, John mentions that he doesn’t know all the details of all the guidelines around job postings, so it might be that there’s some obscure mention that it should be only listed once on each website. 

• John also says that usually Google tries to de-dupe different listings, and that is done for all kinds of listings. So if it’s an image or if it’s a web page, or anything else, if Google can recognise that it’s the same primary content, it will try to just show it once. He assumes the same rule applies for Google Jobs.

Internal Duplicate Content

Q. Having the same content as a PDF on the website and as a block article is not viewed as duplicate content.

• (17:10) The person asking the question has a situation where she has put a piece of content as PDF file on her website and wants to use the same content to present it as an HTML block article on the same website and is worried that it might be viewed as duplicate content. John assures her that Google wouldn’t see it as duplicate content, because it’s different content. One is an HTML page, on is a PDF. Even if the primary piece of content on there is the same, the whole thing around it is different. From that level, Google wouldn’t see it as duplicate content. At most, the difficulty might be that in the search results it can happen that both of these show up at the same time. From SEO point of view, it is not necessarily a bad things, but maybe there’s a personal strategic reasons to have either the PDF or the HTML page more visible.

Paginated Content

Q. In paginated content Google views first pages of content as more important than pages that come after.

• (19:39) The person asking the question has a website with discussions, where a thread can have too many comments to have them all in one long page. He wants to make it a paginated content but is not sure whether the newest comments should appear on the first page or on the last pages. John says that that is something that is ultimately up to the person asking the question and which comments he wants to prioritise. John assumes that if something is on page four, then Google would have to crawl page one, tow, three first to find that, and usually that would mean that it’s longer away from the main part of the website. From Google’s point of view, what would probably happen there is Google wouldn’t give it that much weight, and probably Google wouldn’t recrawl that page as often. Whereas if the newest comments should be the most visible ones, then maybe it makes sense to reverse that order and show them differently. Because if the news comments are right on the main page, then it’s a lot easier for Google to recrawl that more often and to give it a little bit more weight in the search results. That’s up to the website owner how he wants to balance that.

Page Number

Q. Google doesn’t have a specific ratio on how many pages or how many indexable pages a website should have.

• (28:48) From Google’s point of view, there’s no specific ratio that Google would call out for how many pages a website should have or how many indexable pages a website should have. That’s ultimately up to a website owner. What John says he tends to see is that fewer pages tend to perform better in the sense that if the value of the content is concentrated on fewer pages, then, in general, those few pages tend to be a lot stronger than if the content was to be diluted across a lot of different pages. That plays across the board, in the sense that, from a ranking point of view, Google can give these pages a little bit more weight. From a crawling point of view, it’s easier for Google to keep up with these. So especially if it’s a new website, John recommends starting off small focusing on something specific and then expanding from there, and not just going in and creating 500,000 pages that Google needs to index. Because especially for a new website, when it starts off with a big number of pages, then chances are, Google will just pick up a small sample of those pages, and whether or not that small sample is the pages most important to the website is questionable.

Referring Pages

Q. If URLs referring to the pages of the website are long-retired microsite domains, it is perfectly fine.

• (30:36) John says that URLs from long-retired microsite domains referring to the important pages on the website are not bothersome at all. So in particular, the referring page in the Inspection Tool is where Google first sees the mention of the pages, and if it first sees them on some random website, then that’s just where it saw them. That is what is listed there, it doesn’t mean that there’s anything bad with those pages. From Google’s point of view, that’s purely a technical thing. It’s not a sign that there is a need to make sure that the pages were first found on some very important part of the website. If the pages are indexed, that’s the important part there. Referring page is useful when there’s a question on how Google even found the page, or where it comes from. If there are weird URL parameters in there, or if there’s something really weird in the URL that Google should never have found in the first place, then looking at the referring URL is something that helps to figure out where this actually comes from. 

A Drop In Crawl Stats

Q. There are several things that Google takes into account when deciding on the amount of crawling it does on a website.

• (35:09) On the one hand, Google tries to figure out how much it needs to crawl from a website to keep things fresh and useful in its search results. That relies on understanding the quality of the website, how things change on the website. Google calls it the crawl demand. On the other hand, there are the limitations that Google sees from the server, from the website, from the network infrastructure with regard to how much can be crawled on a website. Google tries to balance these two.

• The restrictions tend to be tied to two main things – the overall response time to requests to the website and the number of errors, specifically, server errors, that can be seen during crawling. If Google sees a lot of server errors, then it will slow down crawling, because it doesn’t want to cause more problems. If it sees that the server is getting slower, then it will also slow down crawling. So those are the two main things that come into play there. The difficulty with the speed aspect is that there are two ways of looking at speed. Sometimes that gets confusing when looking at the crawl rate.

• Specifically for the crawl rate, Google just looks at how quickly it can request a URL from the server. The other aspect of speed is everything around Core Web Vitals and how quickly a page loads in a browser. The speed that it takes in a browser tends not to be related directly to the speed that it takes for Google to fetch an individual URL on a website, because in a browser the JavaScript needs to be processed, external files need to be pulled in, content needs to be rendered, and all of the positions of the elements on the page need to be recalculated. That takes a different amount of time than just fetching that URL. That’s one thing to watch out for.

• When trying to diagnose a change in crawl rate there’s no need to look at how long it takes for a page to render, instead, it’s better to look at just purely how long it takes to fetch that URL from the server.

• The other thing that comes in here as well – is that, from time to time – depending on what is done on the website, Google tries to understand where the website is actually hosted. If Google recognises that a website is changing hosting from one server to a different server – that could be to a different hosting provider, that could be moving to a CDN, or changing CDNs, anything like that – Google’s systems will automatically go back to some safe rate where it knows that it won’t cause any problems, and then, step by step, increase again.

Link Juice

Q. Link Juice is a great way to tell Google which pages on the website are important.

• (46:01) “Link Juice” is always one of those terms where people have very conflicting feelings about it because it’s not really how Google’s systems look at it. With regard to internal linking, this is one of the most important elements of a website because it is a great way to tell Google what is considered to be important on the pages. Most websites have a home page that is seen as the most important part of the website, and especially links that can be provided from those important pages to other pages that are thought to be important – that’s really useful for Google. It can be that these are temporary links too. For example, if there’s an e-commerce site, and a new product is linked to from the home page, then that’s a really fast way for Google to recognise those new products, to crawl and index them as quickly as possible, and to give them a little bit of extra weight. But of course, if those links are removed, then that internal connection is gone as well. With regard to how quickly that is picked up, that’s essentially picked up immediately as soon as Google recrawls and reindexes those pages.

Crawling

Q. There are a couple of ways to go around “discovered, not indexed” URLs.

• (58:21) John says Google finds all kinds of URLs across the web, and a lot of those URLs don’t need to be crawled and indexed, because maybe they’re just variations of URLs Google already knows, or maybe they’re just some random forum or scraper script that has copied URLs from the website and included them in a broken way. Google finds all of these linked all the time on the web. So it’s very normal to have a lot of these URLs that are either crawled and not indexed or discovered and not crawled, just because there are so many different sources of URLs across the web. John suggests to first of all download a list of a sample of those so that it is possible to look at individual examples and try to classify which of those URLs are actually ones that are important and which of these are ones that can be ignored. Anything that looks really weird as a URL is better to be ignored. Regarding the ones that are important, that’s something where it would be useful to try to figure out what can be done to better tie these to the website with regard to tying them to things like internal linking.

Sign up for our Webmaster Hangouts today!

Often changing title tags

Q. The problem with changing title tags very often lies in the fact that Google won’t be able to recrawl them that often.

• (01:02) The person asking the question talks about his website in the mutual fund industry, where it is necessary that the title tag changes every day depending on the number representing the stock prices. John says that it’s something where Google wouldn’t give any special weight if the title tag keeps changing. But the thing is that if the website owner is to change the titles on a daily basis, Google might not re-crawl that page on a daily basis. So it might be the case that the titles are changed every day, but in the search results, the title that Google shows is a few days old just because that’s the last version that it picked up from that page. It’s more like a practical effect rather than a strategic effect.

Reusing a domain

Q. There’s no strict time frame within which Google catches up with a domain being reused for different purposes.

• (04:04) Google catching up with the fact that a domain is being used for a different purpose is something that happens over time organically. If an existing domain is being reused, and there is new content that is different from the one before, then over time, Google will learn that it’s a new website and treat it accordingly. But there’s no specific time frame for that. 

• There are two things to watch out for in situations like this. The first one is whether the website was involved in some shady practices before, like, for example, around external links. That might be something that needs to be cleaned up.

• The other aspect is if there’s any webspam manual action, then, that’s something that needs to be cleaned up, so that the website starts from a clean slate. 

• Its’ never going to be completely clean if something else was already hosted on the website before, but at least it puts the website in some sort of reasonable state where it doesn’t have to drag that baggage along.

Robots.txt and traffic

Q. Failure in traffic might not be necessarily linked to technical failures, for example, robots.txt failure.

• (10:20) The person asking the question is concerned with lower traffic on his website and links it to several days of robots.txt failure and connectivity issues. John says there are two things to watch out for in situations like this. On the one hand, if there are server connectivity issues, Google wouldn’t see it as a quality problem. So it wouldn’t be that the ranking for the pages would drop. That’s the first step. So if there’s a drop in the ranking of the pages, then that would not be from the technical issue.

• On the other hand, what does happen with these kinds of server connectivity problems is that if Google can’t reach the robots.txt file for a while, then it will assume that it can’t crawl anything on the website, and that can result in some of the pages from the website being dropped from the index. That’s kind of a simple way to figure out whether it’s from a technical problem or not. Are the pages gone from the index, and if so, that’s probably from a technical problem. If it is from a technical problem, if these pages are gone, then usually Google will retry those missing pages after a couple of days maybe and try to index them again.

• If the problem has happened a while ago and there were steps taken in an attempt to fix that, and the problem keeps recurring, it is worthy to double-check with the Crawl Error section in Search Console to see if there is still, perhaps, a technical issue where sometimes maybe Googlebot is blocked.

Indexing the comment section

Q. It’s important to make sure that the way the comment section is technically handled on the page makes it easy for Google to index comments.

• (16:47) John says that it is up to a website owner whether he wants the comments to show in SERPs or not, but comments are essentially a technical element on the page. So it’s not that there’s a setting in Search Console to turn it on or off. It’s basically there are different ways of integrating comments on web pages, and some of those ways are blocked from indexing and some of those ways are easy to index. So if there’s a need to have the comments indexed, then it’s important to make sure to implement them in a way that’s easy to index. The Inspect URL tool in Search Console will show a little bit of what Google finds on the page, so it can be seen whether Google can index the comments.

URL not indexed

Q. If Google crawls the URL, it doesn’t automatically mean it will index it.

• (21:10) The person asking the question is concerned by the fact that even though his URLs get crawled, he gets the “URL discovered, not indexed”, or “URL crawled, not indexed” messages – he thinks that maybe the content is not good enough for Google to index it. John says that it is kind of an early easy assumption to say “Oh, Google looked at it but decided not to index it”. Most of the time Google crawls something, it doesn’t necessarily mean that it will automatically index it. John says these two categories of not indexed can be treated as a similar thing. It’s tricky because Google doesn’t index everything, so that can happen.

CDN

Q. Whenever a website moves to CDN or changes its current one, it affects crawling but doesn’t really affect rankings.

• (26:28) From a ranking point of view, moving to a CDN or changing the current one wouldn’t change anything. If the hosting is changed significantly, what will happen on Google’s side is the crawling rate will move into a more conservative area first, where Google will crawl a little bit less first because it saw a bigger change in hosting. Then, over time, in, probably, a couple of weeks, maybe a month or so, Google will increase the crawl rate again to see where it thinks it will settle down. Essentially that drop and craw rate overall for the move to a CDN or change of a CDN that can be normal.

• The crawl rate itself doesn’t necessarily mean that there is a problem, because if Google was crawling two million pages of the website before, it’s unlikely that these 2 million pages would be changing every day. So it’s not necessarily the case that Google would miss all of the new content on the website. It would just try to prioritise again and figure out which of these pages it actually needs to re-crawl on a day-to-day basis. So just because the crawl rate drops, it’s not necessarily a sign for concern.

• Some other indicators, like, for example, the change in average response time would be of more priority. Because the crawl rate that Google chooses is based on the average response time. It’s also based on server errors etc. if the average response time goes up significantly, Google will stick to a lower crawl rate.

Changing the rendering and ranking

Q. There might be a couple of reasons why after changing the client-side rendering to a server-side rendering the website doesn’t recover from a drop in rankings.

• (36:22) John says there might be two things at play whenever a website sees a drop in rankings. On the one hand, it might be that with the change of the infrastructure, the website layout and structure has changed as well. That could include things like internal linking, maybe even the URLs that are findable on the website. Those kind of things can affect ranking. The other thing could be that maybe there were just changes in ranking overall that were happening, and they just happened to coincide with when the technical changes were made.

HREF tag

Q. The image doesn’t matter as much as the Alt text, and Alt texts are treated the same way as an anchor text associated directly with the link.

• (40:33) With regards to an image itself, Google would probably not find a lot of value in that as an anchor text. If there is an anchor text associated with the image, then Google would treat that, essentially, the same as any anchor text that has been associated with the link directly. So from Google’s point of view, the Alt text would, essentially, be converted into a text on the page and be treated in the same way. It’s not that one or the other would have more value or not. They’re basically equivalent from Google’s side, and the order doesn’t matter as much. John says that it probably doesn’t matter at all. It’s essentially just both on the page. However, one thing he advises against doing is removing the visible text purely for usability reasons, since the visible text doesn’t matter as much or the same as Alt text. Because other search engines might not see it that way, and it might also be for accessibility reasons that it actually makes sense to have a visible text as well.

• So it’s not about blindly removing it to a minimum, but rather knowing that there’s no loss in having both of them there.

Moving Domains

Q. There are a couple of things that can be done to ensure that moving from one domain to another takes the value of the old domain with it.

• (42:04) There are two things related to moving from one domain to another. On the one hand, if there’s a movement from one website to another, and the redirects are used to move things over, and the various tools such as the Change of Address tool in search Console are used, then that helps Google to really understand that everything from the old domain should just be forwarded to the new one. 

• The other aspect there is on a per-page basis. Google also tries to look at cannonicalisation, and for that, it tries to look at a number of different factors that come in. on the one hand, redirects play a role, things like internal linking play a role, the rel=”canonical” on the pages play a role, but external links also play a role. So what could happen in probably more edge cases is that if Google sees a lot of external links going to the old URL and maybe even some internal links going to the old URL, it actually indeed the old URL instead of the new one. Because from Google’s point of view, it starts to look like the old URL is the right one to show, and the new one is maybe more of a temporary thing. Because of this, what John recommends for a migration from one domain to another is not to only set up the redirect and use the Change of Address tool, but also to go off and try to find the larger websites that were linking to the previous domain, and see if they can update those links to the new domain.

Robots.txt and indexing

Q. If the pages blocked by robots.txt are still indexed, it is not necessary to put a no-index tag on them.

• (44:25) If the URL is blocked by robots.txt, Google doesn’t see any of the meta tags on the page. It doesn’t see the rel=”canonical” on the page because it doesn’t crawl that page at all. So if the rel=”canonical” or a no-index on the page needs to be taken into account, the page needs to be crawlable. 

• The other aspect here is that oftentimes these pages may get indexed if they’re blocked by robots.txt, but they’re indexed without any of the content because Google can’t crawl it. Usually, that means that these pages don’t show up in the search results anyway. So if someone is searching for some kind of product that is sold on the website, then Google is not going to dig and see if there’s also a page that is blocked by robots.txt, which would be relevant because there are already good pages from the website that can be crawled and indexed normally that Google can show. On the other hand, if a suite query is done for that specific URL, then maybe the URL would be seen in the search results without any content. So a lot of times it is more of a theoretical problem rather than a practical problem, and theoretically, these URLs can get indexed without content, but in practice, they’re not going to cause any problems in search. And if they’re being seen showing up for practical queries on the website, then most of the time that’s more a sign that the rest of the website is really hard to understand.

• So if someone searches for one of the product types, and Google shows one of these roboted kinds of categories or facet pages, then that would be a sign that the visible content on the website is not sufficient for Google to understand that the normal pages that could have been indexed are actually relevant here. 

• That would be the first step there is to try and figure out whether normal users see these pages when they search normally. If they don’t see them, then that’s fine. It can be ignored. If they do see these pages when they search normally, then that’s a sign that maybe it is worth focusing on other things, on the rest of the website.

Google Favicon

Q. Google favicon picks up homepage redirects.

• (47:04) If the homepage is redirected or if the Favicon file is redirected to a different part of the website, Google should be able to pick that up. Because practically what would happen here is Google would follow that redirect, but it would probably still index it as the homepage anyway. So from a practical point of view, if the name of the website is searched for, probably Google would show the root URL even though it redirects to a lower=level page.

Product Review Images

Q. To stand out in terms of images in product reviews, using original photos is ideal.

• (52:49) The person asking the question wonders whether to stand out in terms of product review images, it is okay to have photoshopped versions of the images found online or it is better to upload original photos. John says that the guidelines that Google has for reviews recommend focusing on unique photos created of these products and not artificial review photos. He doesn’t think Google systems would automatically recognise that, but it’s something that Google would probably look at, at least on a manual basis from time to time. 

Sign up for our Webmaster Hangouts today!

Paid Links

Q. The way Google decides whether the link is a paid link or not does not just depend on someone reporting the link as a paid link.

• (00:42) Google takes a lot of different things into account when deciding whether the link is a paid link. It does not give every link that it finds full weight. So even if it is not sure, something can be somewhere in between, but it is a number of things that it takes into account there. It’s not just someone reporting this as a paid link, because random people on the internet report lots of things that aren’t necessarily true. At the same time, sometimes it’s useful information. So it’s a lot of things that come together with regards to paid links.

Internal Link Placement

Q. The placement of the internal link doesn’t really matter, but the placement of the content itself matters.

• (02:15) For internal links, on the one hand, Google uses them to understand the context better. So things like anchor text helps it, but another really important part is really just being able to crawl the website. For that, it doesn’t matter where that link is on a page to kind of crawl the rest of the website. Sometimes things are in the footer, sometimes in the header, sometimes in a shared menu or in a sidebar or within a body of content. All of those linked places are all fine from Google’s point of view. Usually what Google differentiates more with regards to location on a page is the content itself to try to figure out what is really relevant for this particular page. For that, it sometimes really makes sense to focus on the central part of the page, the primary piece of content that changes from page to page and not so much the headers and the sidebars and the footers or things like that. Because those are a part of the website itself, but it’s not the primary reason for this page to exist and the primary reason for Google to rank that page. That’s kind of the difference that Google takes when it comes to different parts of the page. As for links, it’s usually more to understand the context of pages and to be able to crawl the website, and for that Google doesn’t really need to differentiate between different parts of the page.

Product Review Websites

Q. There are not really strict guidelines or a checklist for websites to be classified as product page websites.

• (04:34) The person asking the question has a fashion website, from which he started to link products in different stores, as his viewers started asking where they can buy the products from their articles, he now is not sure if his website would classify as a product review website or not. John says that he doesn’t think Google would differentiate that much with these kinds of websites. It’s not that there is a binary decision on which type of website something is. From his point of view, it sounds like there is some kind of review content and informational content on the person’s website, as well as some affiliate content. All of these things are fine. It’s not a case that website owners have to pick one type of website and say that everything on their website fits some kind of criteria exactly. In most cases, on the web, there is a lot of grey room between the different types of websites. From Google’s point of view, that’s fine. That is kind of expected. John says it shouldn’t be really worrisome whether Google would think the website is a product review website. Essentially, it’s good to use the information that Google gives for product review websites, but it’s not something where there is a checklist that one has to fulfill for anything that is classified exactly as a product review website.

Local Directories

Q. Making sure that the information about the business is correct and matching across different websites and tools is important for Google to not get confused.

• (07:41) John is not sure how having the exact same business information across the web plays into Google Business Profile, local listings and that part of things. One place where he has seen a little in that direction, which might not be perfectly relevant for local businesses, but just generally in Google recognizing the entity behind a website or a business. For that, it does sometimes help to really make sure that Google has consistent information, that it can recognise that the information is correct because it found it in multiple places on the web. Usually, this plays more into the knowledge graph, the knowledge panel side of things, where if Google can understand, that this is the entity behind the website. If there are different mentions of that entity in different places, and the information there is kind of consistent, then Google can trust that information. Whereas if Google finds conflicting information across the web, then it’s harder. For example, if there is a situation where there is local business structure data on website pages with local profiles with opening hours or phone numbers, then on the website there is a marked up info conflicting with that. On Google’s side, it has to make a judgment call then and it doesn’t know what is correct. In those kinds of situations, it’s easy for Google’s systems to get confused and use the wrong information. Whereas if website owners find some way to consistently provide the correct information everywhere, then it’s a lot easier for Google to say what the correct information is.

Links

Q. Linking back to a website that has linked to you is not viewed as a linked scheme, as long as all the rules are followed.

• (11:24) The person asking the question is in a situation where there are a few websites that are linking to his website. He doesn’t know whether he is getting any value from that, but assuming he is, he wonders if linking to those websites, following all the rules, not making any illegal link exchanges, would result in him losing some value. John says that it is perfectly fine and natural, especially if this is a local business linking to its neighbours. Or if the website is mentioned in the news somewhere, and the person mentions that on his website that would be okay. Essentially, he is linking back and forth. It’s kind of a reciprocal link, which is a natural kind of link. It’s not something that is there, because there is some crazy linked scheme. If that is done naturally and there isn’t any weird deal behind the scenes, it should be fine.

Cleaning Up Website After Malware Attacks

Q. There are a few things that can be done to make the unwanted links drop out of the index after a hacker attack on a website.

• (24:10) The person asking the question has experienced a malware attack on his website that resulted in lots of pages that he doesn’t want to be indexed being indexed. He has cleaned up after the attack, but the results of the malware are still being shown and he can’t use a temporary removal tool, as there are too many links. John suggests that first of all he needs to double-check that the pages he removed were actually removed. Some types of website hacks are done in a way that if these are checked manually, it looks like the pages are removed. But actually, for Google, they are still there. It can be checked with the Inspect URL tool. Then for the rest, there are two approaches. On the one hand, the best approach is to make sure that the more visible pages are manually removed, that means searching for the company name, for the website name, primary products etc., seeing the pages that show up in the search results and making sure that anything that shouldn’t be shown is not shown. Usually, that results in maybe up to 100 URLs, where the website owner can say that these are hacked and need to be removed as quickly as possible. The removal tool takes those URLs out within about a day.

• The other part is the URLs that are remaining – they will be recrawled over time. But usually, when it comes to lots of URLs on a website, that’s something that takes a couple of months. So on the one hand, those could be just left to be, as they are not visible to people unless they explicitly search for the hacked content or do a site query of the website. These will drop out over time in half a year. Then they can be double-checked afterwards to see if they’re actually completely cleaned up. If that needs to be resolved as quickly as possible, the removal tool with the prefix setting can be used. It is essentially trying to find common prefixes for these pages, which might be a folder name or a filename or something that is in the beginning and filtering those out. The removal tool doesn’t take them out of Google’s index, so it doesn’t change anything for the ranking. But it doesn’t show them in the results anymore.

Emojis

Q. Using emojis in title tags and meta descriptions doesn’t really affect anything.

• (33:04) One can definitely use emojis in titles and descriptions of pages. Google doesn’t show all of these in the search results, especially if it thinks that it kind of disrupts the search results in terms of looking misleading perhaps and these kinds of things. But it’s not that emojis cause any problems, so it’s okay to keep them there. John doesn’t think they would give any significant advantage, because at most what Google tries to figure out is what is the equivalent of that emoji. Maybe Google will use that word as well associated with the page, but it’s not like the website will get an advantage from that. It doesn’t harm or help SEO in any way.

API and Search Console

Q. API and Search Console take their data from the same source but present it a little bit differently.

• (34:15) The data in the API and the data in the UI is built from the exact same database tables. So it’s not that there is any kind of more in-depth or more accurate data in the API than in the UI. The main difference between the API and the URL is that in the API there are more rows of examples that can be retrieved when downloading things. So sometimes that is useful. The other thing that is perhaps a little bit confusing with the API and the data in Search Console is that when looking at a report in Search Console, there will be numbers on top that give the number of clicks and impressions overall. The data that is provided in the API is essentially the individual rows that are visible in the table below the overall data in Search Console. For privacy reasons and various other reasons, Google filters out queries that have very few impressions. So in the UI in Search Console on top with the number Google includes the aggregate full count, but the rows that are shown there don’t include the filtered information. So what can happen is that if one is to look at the overall total in Search Console it’ll be a different number than if the totals are taken from the API, where all of the rows are added up. That’s something where it’s a little bit confusing at first, but essentially it’s the same data presented in a slightly different way in the API.

FAQ In Rich Results

Q. There are three criteria that need to be followed by FAQ schema to have an opportunity to be featured in rich results.

• (36:15) FAQ rich results are essentially similar to other types of rich results in that there are several levels that Google takes into account before it shows them in the search results. On the one hand, they need to be technically correct. On the other hand, they need to be compliant with Google policies. John says he doesn’t think there are any significant policies around FAQ-rich results other than that the content should be visible on the page. The third issue that sometimes comes into play here is that Google needs to be able to understand that the website is trustworthy, in that it can trust the data to be correct. That is sometimes something where kind of from a quality point of view, Google may be not convinced about a website and then it wouldn’t show the results. Those are the three steps to look at FAQ rich results.

Seasonal Pages

Q. Both removing seasonal pages when they are no longer relevant and leaving them be is okay. The thing to remember is to use the same URL every year.

• (37:38) On Google’s side, it’s totally up to the website owner to choose how to deal with seasonal pages. Keeping the pages there is fine, removing them after a while is fine if they’re no longer relevant. Essentially, what would be seen is that traffic to these pages will go down when it’s not seasonal, but no one is missing out on any impressions there. If the pages are made to be no index or 404 for a while and then brought back later, that’s essentially perfectly fine. The one thing to watch out for is to reuse the same URL year after year. So instead of having a page that is called Black Friday 2021 and then Black Friday 2022, it’s better to just have Black Friday. That way, if the page is reused, all of the signals that were associated with that page over the years will continue to work in the website’s favour. That’s the main recommendation there. It’s okay to delete these pages when they’re not needed and just recreate the same URL later and it’s okay to keep those pages for a longer period of time.

CLS Scores

Q. There are no fixed rankings for CLS scores.

• (40:19) There’s nothing like a fixed number with regards to how strong CLS scores work for websites. From Google’s point of view, these metrics are taken into account when it comes to the Core Web Vitals and the page experience ranking factor, and Google tries to look at them overall. Google tries to focus especially on the area where the website is in that reasonable area with regards to these scores. So if a website is not in, let’s call them “poor” or “bad” scores, bad section, then that’s something where Google decides it’s reasonable to take those into account. Google doesn’t have any fixed rankings or algorithmic functions where it would take ½ of FCP and ½ of CLS and take ⅓ of this into account.

• It’s really something where Google needs to look at the bigger picture.

Geo-redirects

Q. Geo-redirects make it hard for Google to find and index content.

• (53:26) Geo-redirects have a negative impact on the content being indexed, and that applies to all kinds of websites. From Google’s point of view, usually, the geo-redirects are more a matter of making it technically hard for Google to crawl the content. Especially if the users from the US are being redirected to a different version of a website, Google will just follow that redirect. Googlebot usually crawls from one location. Then it’s less a matter of quality signals or anything like that, it’s more that if Google can’t see the web pages, then it can’t index them. That’s essentially the primary reason why John says they don’t recommend doing these things. Maybe some big websites do a thing where they redirect some users and don’t redirect others, and maybe Googlebot is not being redirected – it’s possible. From Google’s point of view, it doesn’t do them any favours because it would usually end up in a situation where there are multiple URLs with exactly the same content in the search results. The website is competing with itself. Google sees it as a website duplicating itself with the content in multiple locations, Google doesn’t know which to rank best and makes a guess at that.

Sign up for our Webmaster Hangouts today!