{"id":953,"date":"2018-04-18T18:35:49","date_gmt":"2018-04-18T18:35:49","guid":{"rendered":"http:\/\/hlrnet.com\/sites\/?page_id=953"},"modified":"2025-12-24T10:58:23","modified_gmt":"2025-12-24T10:58:23","slug":"robots-txt","status":"publish","type":"page","link":"https:\/\/hlrnet.com\/sites\/robots-txt\/","title":{"rendered":"Robots.txt"},"content":{"rendered":"<span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\"> &lt; 1<\/span> <span class=\"rt-label rt-postfix\">minute<\/span><\/span><ul>\n<li>Met het bestand robots.txt (te plaatsen in de root van je website) kan je een deel van je website afschermen van spiders, of slechte robots (die bv. op zoek zijn naar e-mailadressen).<\/li>\n<li><a href=\"https:\/\/www.bjoernsworld.de\/suchmaschinen\/robots-txt.html\" target=\"_blank\" rel=\"noopener\">Bjoernsworld<\/a> (DE)<\/li>\n<li><a href=\"https:\/\/www.thesitewizard.com\/archive\/robotstxt.shtml\" target=\"_blank\" rel=\"noopener\">How to Set Up a robots.txt to Control Search Engine Spiders<\/a> (The Site Wizard)<\/li>\n<li><a href=\"https:\/\/www.jafsoft.com\/searchengines\/webbots.html\" target=\"_blank\" rel=\"noopener\">Jasoft: robots that visit your website<\/a><\/li>\n<li><a href=\"https:\/\/www.robotstxt.org\/\" target=\"_blank\" rel=\"noopener\">robotstxt.org<\/a><\/li>\n<\/ul>\n<span class=\"last-modified-timestamp\">Dec 24, 2025 @ 10:58 am<\/span>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\"> &lt; 1<\/span> <span class=\"rt-label rt-postfix\">minute<\/span><\/span>Met het bestand robots.txt (te plaatsen in de root van je website) kan je een deel van je website afschermen van spiders, of slechte robots (die bv. op zoek zijn naar e-mailadressen). Bjoernsworld (DE) How to Set Up a robots.txt to Control Search Engine Spiders (The Site Wizard) Jasoft: robots<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"iawp_total_views":1,"footnotes":"","_wp_rev_ctl_limit":""},"class_list":["post-953","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Robots.txt - hlrnet.com<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/hlrnet.com\/sites\/robots-txt\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Robots.txt - hlrnet.com\" \/>\n<meta property=\"og:description\" content=\"Reading Time:  &lt; 1 minuteMet het bestand robots.txt (te plaatsen in de root van je website) kan je een deel van je website afschermen van spiders, of slechte robots (die bv. op zoek zijn naar e-mailadressen). Bjoernsworld (DE) How to Set Up a robots.txt to Control Search Engine Spiders (The Site Wizard) Jasoft: robots\" \/>\n<meta property=\"og:url\" content=\"https:\/\/hlrnet.com\/sites\/robots-txt\/\" \/>\n<meta property=\"og:site_name\" content=\"hlrnet.com\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-24T10:58:23+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/robots-txt\\\/\",\"url\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/robots-txt\\\/\",\"name\":\"Robots.txt - hlrnet.com\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/#website\"},\"datePublished\":\"2018-04-18T18:35:49+00:00\",\"dateModified\":\"2025-12-24T10:58:23+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/robots-txt\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/hlrnet.com\\\/sites\\\/robots-txt\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/robots-txt\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Robots.txt\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/#website\",\"url\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/\",\"name\":\"hlrnet.com\",\"description\":\"Hans Le Roy\",\"publisher\":{\"@id\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/#organization\",\"name\":\"hlrnet.com\",\"url\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/wp-content\\\/uploads\\\/2018\\\/04\\\/cropped-default512x512-trans-1.png\",\"contentUrl\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/wp-content\\\/uploads\\\/2018\\\/04\\\/cropped-default512x512-trans-1.png\",\"width\":512,\"height\":160,\"caption\":\"hlrnet.com\"},\"image\":{\"@id\":\"https:\\\/\\\/hlrnet.com\\\/sites\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Robots.txt - hlrnet.com","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/hlrnet.com\/sites\/robots-txt\/","og_locale":"en_US","og_type":"article","og_title":"Robots.txt - hlrnet.com","og_description":"Reading Time:  &lt; 1 minuteMet het bestand robots.txt (te plaatsen in de root van je website) kan je een deel van je website afschermen van spiders, of slechte robots (die bv. op zoek zijn naar e-mailadressen). Bjoernsworld (DE) How to Set Up a robots.txt to Control Search Engine Spiders (The Site Wizard) Jasoft: robots","og_url":"https:\/\/hlrnet.com\/sites\/robots-txt\/","og_site_name":"hlrnet.com","article_modified_time":"2025-12-24T10:58:23+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/hlrnet.com\/sites\/robots-txt\/","url":"https:\/\/hlrnet.com\/sites\/robots-txt\/","name":"Robots.txt - hlrnet.com","isPartOf":{"@id":"https:\/\/hlrnet.com\/sites\/#website"},"datePublished":"2018-04-18T18:35:49+00:00","dateModified":"2025-12-24T10:58:23+00:00","breadcrumb":{"@id":"https:\/\/hlrnet.com\/sites\/robots-txt\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/hlrnet.com\/sites\/robots-txt\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/hlrnet.com\/sites\/robots-txt\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/hlrnet.com\/sites\/"},{"@type":"ListItem","position":2,"name":"Robots.txt"}]},{"@type":"WebSite","@id":"https:\/\/hlrnet.com\/sites\/#website","url":"https:\/\/hlrnet.com\/sites\/","name":"hlrnet.com","description":"Hans Le Roy","publisher":{"@id":"https:\/\/hlrnet.com\/sites\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/hlrnet.com\/sites\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/hlrnet.com\/sites\/#organization","name":"hlrnet.com","url":"https:\/\/hlrnet.com\/sites\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/hlrnet.com\/sites\/#\/schema\/logo\/image\/","url":"https:\/\/hlrnet.com\/sites\/wp-content\/uploads\/2018\/04\/cropped-default512x512-trans-1.png","contentUrl":"https:\/\/hlrnet.com\/sites\/wp-content\/uploads\/2018\/04\/cropped-default512x512-trans-1.png","width":512,"height":160,"caption":"hlrnet.com"},"image":{"@id":"https:\/\/hlrnet.com\/sites\/#\/schema\/logo\/image\/"}}]}},"publishpress_future_action":{"enabled":false,"date":"2026-05-01 20:17:01","action":"change-status","newStatus":"draft","terms":[],"taxonomy":"","extraData":[]},"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/hlrnet.com\/sites\/wp-json\/wp\/v2\/pages\/953","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hlrnet.com\/sites\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/hlrnet.com\/sites\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/hlrnet.com\/sites\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hlrnet.com\/sites\/wp-json\/wp\/v2\/comments?post=953"}],"version-history":[{"count":0,"href":"https:\/\/hlrnet.com\/sites\/wp-json\/wp\/v2\/pages\/953\/revisions"}],"wp:attachment":[{"href":"https:\/\/hlrnet.com\/sites\/wp-json\/wp\/v2\/media?parent=953"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}