A small piece of HTML can change what a visitor sees or how the browser understands a page. In this lesson, you will explore robots directives: noindex, nofollow, and snippet controls using a short example that you can inspect and change.
The Idea in Plain English
Noindex asks cooperating crawlers not to index the page; nofollow discourages following links on the page. They are not security features.
Search engines are independent services. Valid markup helps them understand a page, but it never guarantees ranking, indexing or a particular search appearance.
A Working Example
Read the sample once before copying it. Notice the element names and attribute values, then compare them with the explanation below.
<head>
<title>Private Draft Preview</title>
<meta name="robots" content="noindex, nofollow">
</head>What Each Part Does
- The title gives the document a name in tabs and other browser surfaces.
- The meta element supplies machine-readable settings; its name or charset decides which setting.
- The head holds page metadata, not the visible lesson content.
- name identifies a field in submitted form data or a named metadata entry.
- content supplies the value of a meta declaration.
The browser does not hide or lock the page. The directive is meant for search crawlers that choose to obey it.
Try It Yourself
Open the page source to verify the robots tag. Remember that server access controls are a separate problem.
Common Beginner Mistakes
- Blocking crawling and assuming the crawler can read noindex at the same time.
- Changing several parts of the example at once, making it difficult to see which part caused a different result.
- Choosing a tag only because its default appearance looks right, without checking whether it describes the content correctly.
Key Takeaways
- Noindex asks cooperating crawlers not to index the page; nofollow discourages following links on the page. They are not security features.
- The browser does not hide or lock the page. The directive is meant for search crawlers that choose to obey it.
- Test the markup in a browser, and check accessibility and real-world behavior where relevant.