What are noindex and the robots meta tag?

The robots meta tag is a <meta name="robots" content="…"> line in a page's <head> that gives instructions to search engine bots. The value noindex means “don't show this page in search results”. The same instruction can be sent for non-HTML files with the X-Robots-Tag HTTP header. nofollow asks bots not to follow the links on the page.

#Why it matters

Used correctly, noindex keeps your index clean: thank-you pages, internal search results and empty categories stay out of the results. Left in by mistake, it quietly removes important pages from the index. That most often happens when a staging setting or a plugin option makes it to production.

#Where you see it in SEOAsist

Technical Audit reports pages with noindex in their robots meta tag as Pages marked noindex, at Serious severity. If a page is noindex on purpose, mark it Intentional in the issue panel; otherwise, if Site API, the WordPress plugin or Pixel is ready, Auto-fix can remove it. Details: Accept issues as intentional and auto-fix them. To see whether Google actually indexed a page, use Monitors → Index status.

#Example

<!-- Internal search results page: don't index, but follow the links -->
<meta name="robots" content="noindex, follow">

#Common mistakes

• Carrying a site-wide noindex from staging into production.
• Blocking a page in robots.txt and also adding noindex; Google can't crawl the page, so it never sees the noindex.
• Listing a noindex page in the sitemap or pointing a canonical at it; the signals contradict each other.