Delete/Remove Indexed URLs from Google SERP on any or WordPress Website

There could be a number of reasons for you wanting to delete or remove the already indexed URL pages from Google SERP. This guide will help you to completely remove the crawled pages to be removed from Google indexing from a WordPress or any website.

I am writing WordPress because the example given in this article on removing the indexed pages show the WordPress backend. Here are some of the examples of the indexed pages that should usually be removed from Google:

  • Thin content pages (Pages with almost no content)
  • Pages with confidential data
  • Specific pages which add no value such as some policy pages
  • Some old page which adds no value to your website anymore or no more relevant

Which indexed pages should not be removed?

  • Pages with crawl errors – Google will drop those pages eventually
  • 404 pages – Google will also drop 404 pages eventually from SERP
  • Different versions of the same page – Use canonical tags instead
  • Paginated pages – Use canonical tags instead

Here are the steps to remove or deleted indexed pages from Google

  • Add Noindex, noarchive, nosnippet meta tags on those pages
  • Disallow those pages through robots.txt file
  • Make the page 404 by deleting or drafting pages
  • Submit to Google search console to remove the indexed page

Adding Noindex, noarchive, nosnippet meta tags on those pages

  • If you are using WordPress, you need to have Yoast plugin. Open those pages in the edit mode and scroll to the Yoast plugin.
  • Go to the Yoast settings, you will see advanced options for meta robots.
  • Under the option “Allow search engines to show this page in search results?”, select “No”.
  • “Should search engines follow links on this page?”, select no.
Yoast settings to prevent a page from being indexed on Google

Yoast settings to prevent a page from being indexed on Google

These two options will add noindex and nofollow meta tags on your page. noindex, though, allows the crawlers to crawl the page but it says the crawlers to not to index the page on google. nofollow tag will tell the crawler not to follow any links given on that page.

You need to add “No Image Index”, “No Archive”, and “No Snippet” meta tags to those pages. You get these options from the drop-down menu. NOINDEX tag tells Google not to index a specific page. NOARCHIVE meta tag tells Google not to store a cached copy of your page. NOSNIPPET meta tag tells Google not to show a snippet (description) under your Google listing, it will also not show a cached link in the search results

Note: You need to remove “Site-wide default: None” option which comes by default, otherwise when you update the page, with “Site-wide default: None” option, other tags would not get updated, so you need to remove that.

Disallow those pages through robots.txt file

You need to add Disallow rules in your robots.txt file, to disallow crawlers to crawl those pages. Robots.txt file uses regex matching. If you know how to write regex rules, you can group those pages in a common theme and disallow at once, otherwise, you can disallow them individually.

Sample robots.txt file

If you want to allow some pages and disallow the rest of the pages, then in the regex matching, first write the disallow rule and then write the allow the rule to make an exception to the crawlers. Furthermore, you can go to the robots.txt tester in the old search console, and you can individually check for the allowed or disallowed URLs at the bottom.

robots.txt tester

Delete or draft the pages

The next step would be to delete or draft the pages. I am not sure of the “privately published” option in the WordPress if that would work in this scenario. Before deleting the pages, have the URLs copies somewhere on a notepad to submit to Google search console.

Publish settings in WordPress

Publish settings in WordPress

Submit to Google search console to remove the indexed page

Login to the old search console. Under Google index, click on “remove URLs” which will show you Remove URLs option. Click on the “Temporarily hide” button. Here, you only need to give the relative path of the page.

Temporarily hide option in search console

For example, if the page that you want to remove is example.com/page-remove, then only put /page-remove. and then continue. You will see three request types for URL removals.

  • Clear URL from the cache and temporarily remove from Search
  • Clear URL from cache only
  • Clear cache and temporarily hide all URLs beginning with…
URL removal request types in Google search console

URL removal request types in Google search console

Clear URL from the cache and temporarily remove from Search

Hides the page from Google Search results for about 90 days, and also clears the cached copy of the page and snippet from the Google index. The page can reappear in Search results after the blackout period.

Notes:

The URL must match exactly, including the page extension (for example, .html). Therefore, if you specify path/mypage, then the following URLs will NOT be matched:

  • path/MyPage, path/mypage?1234
  • path/mypage.html
  • Anchors are never matched and should be omitted from your URL (mypage#anchor).

Google will recrawl the page during the blackout period and refresh the page cache and snippet, but will not show them until the blackout period expires, unless you make removal permanent.

Clear URL from cache only

Clears the cached page and snippet from the Google index, but does not remove the page from Search results. Google will refresh the page cache and snippet the next time the page is crawled.

Notes:

The URL must match exactly, including the page extension (for example, .html). Therefore, if you specify path/mypage, then the following URLs will NOT be matched:

  • path/MyPage, path/mypage?1234
  • path/mypage.html
  • Anchors are never matched and should be omitted from your URL (mypage#anchor).

Clear cache and temporarily hide all URLs beginning with…

Hides all URLs beginning with the specified prefix, and also clears the cached copies of the pages and snippets. For example, if your property is example.com, and you specify “foods/”, then all of the following URLs will match:

  • http://example.com/foods/pizza
  • https://www.example.com/foods/bread?type=whole_wheat
  • http://www.example.com/foods/pasta/spaghetti/bologonese.html

Notes:

URLs are hidden for about 90 days and the cached page and snippets are cleared. Matching results can reappear in search results after the blackout period. Google will recrawl the page during the blackout period and refresh the page cache and snippet, but will not show them until the blackout period expires, unless you make removal permanent by deleting the page. Here is a very important link from Google on how to find the indexed page and image URL.

Once, you are done, submit the request. The request might take around 24 hours to get updated. If everything is fine, then the submitted URL will get removed within 24 hours until then it will be in the pending status. Once removed, the status will change to removed.

But remember, this is only for the 90 days and beyond the blackout period, it might get reindexed on the Google SERP. So, to permanently remove the URL, delete them or make them noindex along with blocking them through robots.txt file. Once removed and that is by mistake, you can reinclude or reindex the URL again on Google.

Pending and removed status on remove URLs tool in search console

Pending and removed status on remove URLs tool in search console

What you should not do?

Do not change the URL of the page and redirect the old URL to the changed URL

Leave a Reply

Your email address will not be published. Required fields are marked *