Check If Certain Content on Your Webpage is SEO Friendly


Verify if specific content on your webpage is SEO friendly & is being picked by Googlebot during indexing.

Last Updated : July 14, 2020

What it means?

You generally expect Googlebot to index all the content on your page. But, the bot may not always index your page’s dynamic content. This is the content that requires user-action (such as scrolling) or JavaScript execution to appear on your page. With access to Google Search Console (or Google Webmaster) for the concerned site, you can use the tools there to check if the content you expect Googlebot to index is being indexed. And in case you don’t have Google Search Console access, there still are ways to do so.

[Google Search Console] Check if certain content appeared in the crawled copy of your page:

Use this method if the page in question is already live & has already been indexed by Google & if you have access to Google Search Console for the site. Login to Google Search Console and enter the URL you want to verify to access Google’s crawled copy of your page:


Viewing Google's crawled copy of your page (from Google Search Console)
Viewing Google’s crawled copy of your page (from Google Search Console)

Once you copy the HTML source of the crawled page, simply search for your desired content within the HTML source:

  • If this is a certain text, it should be present within the HTML source of your crawled page copy.
  • If this is an image, it should be present within the src attribute of an <img> tag or within srcset attribute for a <picture> tag.

If the content doesn’t appear as explained above, Googlebot may not be indexing that content.

[Google Search Console] Check if Googlebot will index certain content in next crawl:

If you seek to check a certain page / content of the page that isn’t yet indexed by Googlebot, you can vertify using ‘Test Live URL’ feature of Google Search Console:


Viewing how your live URL will appear to Googlebot during next crawl
Viewing how your live URL will appear to Googlebot during next crawl

For above-the-fold content, you can view how the content renders and for rest of the page, you can copy the HTML source of the page and search for your desired content within the HTML source.

[Without Google Search Console] Check if Google has indexed specific content on your live page:

The most elegant & sure-shot way to verify if Google is indexing a specific text from your live page is to search for that text on Google with site:<url-to-verify>. So, if I want to ensure that the text Viewing how your live URL on this page is being indexed, I shall search on Google as following: Viewing how your live URL site:https://www.tezify.com/how-to/check-if-certain-webpage-content-is-seo-friendly/.


Check within the indexed copy for specific text content via Google Search
Check within the indexed copy for specific text content via Google Search

To perform a similar verification for images, use Google’s reverse image search:


Check within indexed copy for specific image via Google Search
Check within indexed copy for specific image via Google Search

If Google doesn’t show the searched content as expected, try the approach detailed in the next section.

[Without Google Search Console] Check if Googlebot will index certain content in next crawl:

This is a powerful method because it works even if:

  • You don’t have access to site’s Google search console
  • The URL is or isn’t publicly accessible.
  • Google has or hasn’t yet indexed the content.

However, it requires accessing certain tech-tools to execute. First-up, keep the below user-agent strings handy:

Here’s the User-Agent string for mobile:

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

And here’s one for desktop:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/74.0.0.0 Safari/537.36

Now, our goal is to get the same HTML for our URL that Googlebot will be provided. We can do so via:

  • Chrome Devtools: Using the User Agent string above, follow this guide to override User-Agent string within Chrome devtools.
  • cURL command: Run the following cURL command on Linux command-line or online here:

Below is the curl command you can run (simply replace your URL) to obtain the HTML.

  • For Mobile (replace with your URL in the snippet below):
  • curl '<ENTER_YOUR_URL_HERE>' \
      -H 'cache-control: no-cache' \
      -H 'user-agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.0.1 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' \
      -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
      -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
    

  • For Desktop (replace with your URL in the snippet below):
  • curl '<ENTER_YOUR_URL_HERE>' \
      -H 'cache-control: no-cache' \
      -H 'user-agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/74.0.0.1 Safari/537.36' \
      -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
      -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
    

Once you obtain the HTML response from any of the above methods, you can search for the specific content within the HTML. If present as a regular HTML content / tag, it shall be picked by Googlebot during next crawl. If it requires execution of JavaScript, it may or may not be picked by the Googlebot during the next crawl immediately.



Didn't find what you are looking for?


Just email your query or suggestion to punit@tezify.com and we shall do the needful.

Contact Us Image

Also Read:
SEO Impact of Lazy Loading Images
Lazy loading images helps improve website speed. It is especially critical for pages with pagination or infinite scrolling. That stated, it is important to ensure the images lazy loading solution does not negatively affect the indexability of those images. This blog post evaluates these considerations and suggests a solution that reliably covers all bases. (..continue reading)



ABOUT THE AUTHOR
Punit Sethi has been working with large e-Commerce & B2C websites on improving their Site Speed & Scalability. He frequently tweets here.
Punit Sethi