A webpage can tell browsers and other tools useful things about itself. The <meta name="robots"> entry provides crawling/indexing hints to compliant search engines.
This entry is best understood as document metadata. A meta tag does not automatically display words on the page. A browser, platform, search engine, or specialized tool must know the particular name before it has any useful effect.
Where this metadata belongs
Place metadata in the document <head>. The name attribute identifies the kind of information; content supplies its value. This is different from the visible page title and headings that appear in the body.
Example HTML document
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="robots" content="noindex, follow">
<title>Learning Club</title>
</head>
<body>
<main><h1>Welcome to Learning Club</h1></main>
</body>
</html>What the example means
The line <meta name="robots" content="noindex, follow"> declares a value of noindex, follow. The browser still uses the ordinary <title> for the tab, and the <h1> remains visible inside the page. This is a metadata example, not a guarantee that a platform will react to it.
Robots directives are requests to crawlers, not access control; the page remains publicly accessible.
How to decide whether to include it
Start with a real requirement. If you are preparing a search summary, platform integration, or app shortcut, identify which system reads this specific meta name. If you cannot name such a system, adding this tag may achieve nothing. Important content should remain visible and accessible without depending on metadata.
Try it yourself
Save the document as index.html. Open the browser developer tools and find the meta element under the head. Change its content, reload the page, and note whether the change affects a supported browser feature, an external preview, or neither. For platform-specific tags, the relevant device or application may be required.
Common Beginner Mistakes
- Robots directives are requests to crawlers, not access control; the page remains publicly accessible.
- Placing the meta element in the visible body instead of the head.
- Assuming an invented or outdated meta name changes search rankings or browser behavior.
Key Takeaways
- The exact metadata name is
robots. - Provides crawling/indexing hints to compliant search engines.
- Metadata is not a substitute for well-written visible content or the required server configuration.