Noindex, No Problems: 6 Good Uses for Noindex Tags

Noindex, Google, and You

Noindexed pages are pages of a website that specifically tell Google and other search engines not to include them in search results. Now at first glance, this seems like an SEO nightmare, but there are a few times when using a noindex tag can be a good thing. Read on for a few common noindex questions and my advice!

What kinds of pages should I noindex?

Pages containing Download links

One of the most popular content marketing CTA incentives is downloadable content. You don’t want people to directly access that content through Google instead of going through your conversion channel. Noindexing your content download page ensures visitors are getting to your content the right way.

Archive Pages

This one can be a little counterintuitive at first. Archive everything past page 2? There’s good stuff in there! You’re not wrong (hopefully!), but the question is in terms of search results: will anyone searching for anything find page/23/ useful for their query? You don’t really want just list pages from your archive ranking, or even worse: ranking above actual relevant content from your site!

I found this explanation from Carter Bowles over at Northcutt helpful:

“Here’s something that happens more often than you might think. A tag page, archive page, or something similar ends up ranking for the query in question, and Google simply decides not to display the more relevant page in search results. It may be hiding in the index somewhere, but it never shows it on the front page, because Google’s algorithm tends to avoid placing multiple links from the same domain on the front page (unless you’re branding is very good). The end result is that the more relevant page never ends up ranking, and the less relevant archive page ends up taking its place.”

Tags

These are similar to archive pages in terms of indexing problems. Simply a list of tags isn’t likely to be relevant to any search query it would inadvertently rank for; you want to be ranking content.

Categories

I’ve seen people go both ways on this one; the consensus seems to be that once you have enough of a content backlog built up, categories can be a channel into a specific topic from your site. Since they’re so sparse when first starting a site, having a category page outrank your content could make your content harder to find. Plus, this tends to happen on category pages since they contain lots of internal links to your content! Unless you have your categories pages are stocked with unique content you’ll want to keep your categories noindexed.

Media Files

Does that GIF of a dog being surprised with a birthday party from a blog post you wrote 8 months ago really need to rank? Even worse, do you want those GIFs to rank in Google Image searches for your site?

The Authors Page

(if you're the only author)

If you’re the only author on your site, there’s really not much new information on this page. Best to leave it noindexed so it doesn’t overtake a more relevant page from your site.

How do I noindex a page?

There are several different ways to add noindex tags to your pages. One of the easiest ways I’ve found to manage the noindexing process on a WordPress site is the Yoast SEO plugin. The plugin has built-in controls for noindexing the author page, archives, tags, categories, and media; it also provides easy access to your site’s robot.txt file, allowing us to set noindex rules for individual pages.

Titles & Meta

To get started, hop into the Yoast SEO General page, and then head over to the Titles & Meta section. Under the “Archives” tab you’ll see options for “Author Archives Setting” as well as a short explanation from the Yoast team. For a one person blog, you’ll want to disable the Author archives; Yoast SEO will redirect the authors page to your site’s homepage. From this tab you can also set your Archives to noindex by setting the “Meta Robots” option in the Date archives settings to “noindex”. Next, select the “taxonomies” tab. From here you can set the Meta Robots to “noindex” for Categories + Tags. Finally, you can noindex your media process by setting the Meta Robots option for Media under “Post Types”.

robots.txt

Noindexing individual pages requires a more granular approach. Head to the “Tools” section of Yoast SEO and select the File Editor. This tool allows you to edit the robots.txt and .htaccess file for your site. What we want to do is edit the content of our robots.txt to disallow indexing of individual URLs for any agent.

BONUS: while you’re in your robots.txt, consider noindexing your entire WordPress plugin directory; no one else needs access to it and leaving it visible creates a possible weakpoint in your site security.

Insert the following lines into your robots.txt

User-Agent: Googlebot-Image

Disallow: /superawesomecontent/signup_success.html

Disallow: /wp-content/uploads/

The first line declares which users this rule applies to. In this case I’ve set the user agent to the Google Images crawler. You can also use an asterisk (*) to block any user agents from indexing according to your rule parameters. The second line specifies each URL you would like to noindex. You can also noindex directories; the third line demonstrates another way to prevent your media library from being indexed.

Overall there are several beneficial reasons for using noindex tags. Used properly they can make sure users are finding your most relevant content. Like sloths.