{"id":11837,"date":"2018-08-17T15:47:35","date_gmt":"2018-08-17T18:47:35","guid":{"rendered":"https:\/\/blog.hostinger.io\/pt-tutoriais\/?p=11837"},"modified":"2023-07-21T17:09:30","modified_gmt":"2023-07-21T20:09:30","slug":"como-criar-robots-txt-bloquear-mecanismos-busca","status":"publish","type":"post","link":"\/pt\/tutoriais\/como-criar-robots-txt-bloquear-mecanismos-busca","title":{"rendered":"Como criar robots.txt e bloquear os mecanismos de busca"},"content":{"rendered":"<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Os bots dos <strong>mecanismos de busca<\/strong> est&atilde;o constantemente escaneando sites para serem indexados. Mas por alguma raz&atilde;o voc&ecirc; pode optar por esconder seus sites. Neste caso o <\/span><b>robots.txt<\/b><span style=\"font-weight: 400\"> pode ser utilizado para impedir que os bots acessem a sua p&aacute;gina.<\/span><\/p><p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Neste tutorial, voc&ecirc; vai aprender como criar&nbsp;<\/span><b>robots.txt <\/b>&ndash; chamado erroneamente por algumas pessoas de robot.txt<span style=\"font-weight: 400\">.<\/span><\/p><figure class=\"wp-block-image size-large\"><a href=\"https:\/\/assets.hostinger.com\/content\/tutorials\/pdf\/Speed-Up-Your-Website-BR.pdf\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" width=\"1024\" height=\"283\" src=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=1024,h=1024,fit=scale-down\" alt=\"\" class=\"wp-image-30339\" srcset=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=1024,fit=scale-down 1024w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=300,fit=scale-down 300w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=150,fit=scale-down 150w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=768,fit=scale-down 768w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=1536,fit=scale-down 1536w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure><p>\n\n\n\n\n\n\n<\/p><h2 class=\"wp-block-heading\" id=\"h-do-que-voce-vai-precisar\"><b>Do Que Voc&ecirc; Vai Precisar<\/b><\/h2><p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Antes de iniciar voc&ecirc; vai precisar:<\/span><\/p><ul class=\"wp-block-list\">\n<li>Acesso ao painel de controle de sua hospedagem ou FTP.<\/li>\n<\/ul><h2 class=\"wp-block-heading\" id=\"h-1-acesse-o-servidor-e-crie-o-arquivo-robots-txt\"><b>1. Acesse o Servidor e Crie o Arquivo robots.txt<\/b><\/h2><p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Antes de mais nada, voc&ecirc; precisa criar um arquivo para criar&nbsp;<\/span><b>robots.txt<\/b><span style=\"font-weight: 400\">. Para isso voc&ecirc; pode usar um cliente FTP ou o Gerenciador de Arquivos. O arquivo deve ser colocado na mesma pasta onde se encontra seu site (geralmente <\/span><b>public_html<\/b><span style=\"font-weight: 400\">). Neste tutorial vamos usar o Gerenciador de Arquivos para criar&nbsp;<\/span><b>robots.txt<\/b><span style=\"font-weight: 400\">:<\/span><\/p><div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"1024\" height=\"194\" src=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/public\" alt=\"Como Criar robots.txt e Bloquear os Mecanismos de Busca\" class=\"wp-image-36648\" srcset=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/w=1024,fit=scale-down 1024w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/w=300,fit=scale-down 300w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/w=150,fit=scale-down 150w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/w=768,fit=scale-down 768w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div><h2 class=\"wp-block-heading\" id=\"h-2-editar-o-arquivo-robots-txt\"><b>2.&nbsp;Editar o Arquivo robots.txt<\/b><\/h2><p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Cada mecanismo de busca usa seu <em>crawler<\/em> (bot ou user-agent) pr&oacute;prio. Ao criar&nbsp;<\/span><b>robots.txt<\/b><span style=\"font-weight: 400\"> voc&ecirc; pode especificar o <em>crawler<\/em> usando o User-agent. Existem centenas de crawlers, mas os mais comuns s&atilde;o<\/span>:<\/p><ul class=\"wp-block-list\">\n<li>Googlebot<\/li>\n\n\n\n<li>Yahoo! Slurp<\/li>\n\n\n\n<li>bingbot<\/li>\n\n\n\n<li>AhrefsBot<\/li>\n\n\n\n<li>Baiduspider<\/li>\n\n\n\n<li>Ezooms<\/li>\n\n\n\n<li>MJ12bot<\/li>\n\n\n\n<li>YandexBot<\/li>\n<\/ul><p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Por exemplo, se voc&ecirc; quer impedir o <em>crawler<\/em> do Google de acessar o seu site, basta editar o <\/span>arquivo<span style=\"font-weight: 400\">&nbsp;com a seguinte regra:<\/span><\/p><pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">User-agent: Googlebot \nDisallow: \/example-subfolder\/<\/pre><p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Caso voc&ecirc; queira bloquear p&aacute;gina de todos os <em>crawlers<\/em> &eacute; s&oacute; usar o *:<\/span><\/p><pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">User-agent: *\n\nDisallow: \/<\/pre><p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Se voc&ecirc; quer prevenir que o <em>crawler<\/em> acesse algum diret&oacute;rio ou arquivo espec&iacute;fico, a regra &eacute; semelhante, mas voc&ecirc; precisa especificar o nome do arquivo ou pasta. Digamos que voc&ecirc; n&atilde;o quer que os <em>crawlers<\/em> acessem o diret&oacute;rio <\/span><b>artigos<\/b><span style=\"font-weight: 400\"> e o arquivo <\/span><b>private.php<\/b><span style=\"font-weight: 400\"> apenas. Nesse caso o robots.txt vai ficar assim:<\/span><\/p><pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">User-agent: Bingbot\n\nDisallow: \/subpasta-exemplo\/pagina-bloqueada.html<\/pre><p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Quando terminar de editar o arquivo <\/span><b>robots.txt<\/b><span style=\"font-weight: 400\"> n&atilde;o esque&ccedil;a de salvar as altera&ccedil;&otilde;es.<\/span><\/p><figure class=\"wp-block-image size-large\"><a href=\"\/pt\/hospedagem-barata\" target=\"_blank\" rel=\"noreferrer noopener\"><img decoding=\"async\" width=\"1024\" height=\"300\" src=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=1024,h=1024,fit=scale-down\" alt=\"\" class=\"wp-image-32927\" srcset=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=1024,fit=scale-down 1024w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=300,fit=scale-down 300w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=150,fit=scale-down 150w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=768,fit=scale-down 768w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/26\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=1536,fit=scale-down 1536w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure><h2 class=\"wp-block-heading\" id=\"h-conclusao\"><b>Conclus&atilde;o<\/b><\/h2><p class=\"wp-block-paragraph\"><span style=\"font-weight: 400\">Voc&ecirc; acabou de aprender como impedir que <em>crawlers<\/em> (ou bots) de mecanismos de busca acessem e indexem o seu site. Isso &eacute; &uacute;til para prevenir que seu site apare&ccedil;a nos mecanismos de busca, claro, se &eacute; isso mesmo que voc&ecirc; quer que aconte&ccedil;a.<\/span><\/p><p>\n\n\n        <div class=\"protip\">\n            <div class=\"protip__heading\">\n                <svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\">\n                    <path d=\"M1.49234 23.5024C1.23229 23.5024 0.972242 23.4024 0.782206 23.2123C0.562165 22.9923 0.452144 22.6822 0.502153 22.3722C0.562165 21.9221 1.14227 17.9113 3.00262 16.351C3.63274 15.8209 4.43289 15.5509 5.26305 15.5609C6.09321 15.5909 6.87335 15.9109 7.47347 16.4911C8.6937 17.6913 8.76371 19.6717 7.6435 20.9919C6.0832 22.8523 2.08245 23.4324 1.63237 23.4924C1.59236 23.4924 1.54235 23.4924 1.50234 23.4924L1.49234 23.5024ZM5.16303 17.5613C4.84297 17.5613 4.53291 17.6713 4.29287 17.8813C3.60274 18.4614 3.07264 19.9317 2.75258 21.242C4.06282 20.9219 5.5331 20.3918 6.11321 19.7017C6.55329 19.1716 6.54329 18.3814 6.0832 17.9213C5.85316 17.7013 5.5431 17.5713 5.20304 17.5613C5.19304 17.5613 5.17303 17.5613 5.16303 17.5613ZM11.7243 21.8821C11.4942 21.8821 11.2642 21.8021 11.0841 21.652C10.8541 21.462 10.7241 21.1819 10.7241 20.8819V15.9109L8.08358 13.2705H3.11264C2.81259 13.2705 2.53254 13.1404 2.3425 12.9104C2.15246 12.6803 2.07245 12.3803 2.12246 12.0902C2.19247 11.7102 2.84259 8.36953 4.70294 7.12929C6.33325 6.04909 8.96375 6.49918 10.244 6.80923C11.5442 4.96889 13.2546 3.4286 15.2349 2.33839C17.4553 1.11816 19.9858 0.518051 22.4963 0.498047C23.0464 0.498047 23.4865 0.948132 23.4865 1.49824C23.4865 5.0389 22.3763 9.97983 17.1753 13.7605C17.4853 15.0408 17.9354 17.6613 16.8552 19.2816C15.615 21.1419 12.2744 21.7921 11.8943 21.8621C11.8343 21.8721 11.7743 21.8821 11.7143 21.8821H11.7243ZM12.7245 16.181V19.6016C13.7146 19.2916 14.7948 18.7915 15.2049 18.1814C15.675 17.4812 15.605 16.091 15.385 14.9008C14.5248 15.3808 13.6346 15.8109 12.7245 16.181ZM9.66388 12.0302L11.9643 14.3307C13.1845 13.8306 14.3648 13.2204 15.485 12.5103C19.9358 9.51974 21.2361 5.60901 21.4561 2.53843C19.6157 2.67846 17.8254 3.20856 16.2051 4.09872C14.2847 5.14892 12.6544 6.68921 11.4942 8.54956C10.7841 9.65977 10.174 10.82 9.66388 12.0302ZM4.39289 11.2701H7.81353C8.1936 10.3599 8.63368 9.46974 9.11377 8.60957C7.92355 8.38953 6.51329 8.31952 5.81315 8.78961C5.19304 9.19968 4.70294 10.3099 4.39289 11.2701Z\" fill=\"#673DE6\"\/>\n                <\/svg>\n                <p class=\"protip__title\">\n                    Aprenda mais sobre Otimiza&ccedil;&atilde;o de Sites                <\/p>\n            <\/div>\n            <p class=\"protip__content\"><a href=\"\/tutoriais\/otimizacao-de-sites\">O Que &eacute; Otimiza&ccedil;&atilde;o de Sites<\/a><br>\n<a href=\"\/tutoriais\/como-criar-sitemap-para-wordpress\">Como Criar um Sitemap no WordPress<\/a><\/p>\n                    <\/div>\n        \n<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Os bots dos mecanismos de busca est&atilde;o constantemente escaneando sites para serem indexados. Mas por alguma raz&atilde;o voc&ecirc; pode optar por esconder seus sites. Neste caso o robots.txt pode ser utilizado para impedir que os bots acessem a sua p&aacute;gina. Neste tutorial, voc&ecirc; vai aprender como criar&nbsp;robots.txt &ndash; chamado erroneamente por algumas pessoas de robot.txt. [&#8230;]<\/p>\n<p><a class=\"btn btn-secondary understrap-read-more-link\" href=\"\/pt\/tutoriais\/como-criar-robots-txt-bloquear-mecanismos-busca\">Read More&#8230;<\/a><\/p>\n","protected":false},"author":78,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"rank_math_title":"Como Criar robots.txt e Bloquear os Mecanismos de Busca","rank_math_description":"Para impedir que os buscadores encontrem alguma p\u00e1gina do seu site, veja como criar robots.txt e como us\u00e1-los do jeito certo.","rank_math_focus_keyword":"robots.txt","footnotes":""},"categories":[7701,7751],"tags":[5831],"class_list":["post-11837","post","type-post","status-publish","format-standard","hentry","category-marketing-digital","category-seo","tag-criar-robot-txt"],"hreflangs":[{"locale":"en-US","link":"https:\/\/www.hostinger.com\/tutorials\/how-to-block-search-engines-using-robotstxt\/","default":1},{"locale":"pt-BR","link":"https:\/\/www.hostinger.com\/br\/tutoriais\/como-criar-robots-txt-bloquear-mecanismos-busca\/","default":0},{"locale":"es-ES","link":"https:\/\/www.hostinger.com\/es\/tutoriales\/como-bloquear-los-motores-de-busqueda-utilizando-robotstxt\/","default":0},{"locale":"en-PH","link":"https:\/\/www.hostinger.com\/ph\/tutorials\/how-to-block-search-engines-using-robotstxt\/","default":0},{"locale":"en-MY","link":"https:\/\/www.hostinger.com\/my\/tutorials\/how-to-block-search-engines-using-robotstxt\/","default":0},{"locale":"en-GB","link":"https:\/\/www.hostinger.com\/uk\/tutorials\/how-to-block-search-engines-using-robotstxt\/","default":0},{"locale":"en-IN","link":"https:\/\/www.hostinger.com\/in\/tutorials\/how-to-block-search-engines-using-robotstxt\/","default":0},{"locale":"en-CA","link":"https:\/\/www.hostinger.com\/ca\/tutorials\/how-to-block-search-engines-using-robotstxt\/","default":0},{"locale":"en-AU","link":"https:\/\/www.hostinger.com\/au\/tutorials\/how-to-block-search-engines-using-robotstxt\/","default":0},{"locale":"en-NG","link":"https:\/\/www.hostinger.com\/ng\/tutorials\/how-to-block-search-engines-using-robotstxt\/","default":0},{"locale":"es-AR","link":"https:\/\/www.hostinger.com\/ar\/tutoriales\/como-bloquear-los-motores-de-busqueda-utilizando-robotstxt\/","default":0},{"locale":"es-MX","link":"https:\/\/www.hostinger.com\/mx\/tutoriales\/como-bloquear-los-motores-de-busqueda-utilizando-robotstxt\/","default":0},{"locale":"es-CO","link":"https:\/\/www.hostinger.com\/co\/tutoriales\/como-bloquear-los-motores-de-busqueda-utilizando-robotstxt\/","default":0},{"locale":"pt-PT","link":"https:\/\/www.hostinger.com\/pt\/tutoriais\/como-criar-robots-txt-bloquear-mecanismos-busca\/","default":0}],"acf":[],"_links":{"self":[{"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/posts\/11837","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/users\/78"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/comments?post=11837"}],"version-history":[{"count":12,"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/posts\/11837\/revisions"}],"predecessor-version":[{"id":36654,"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/posts\/11837\/revisions\/36654"}],"wp:attachment":[{"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/media?parent=11837"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/categories?post=11837"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hostinger.com\/pt\/tutoriais\/wp-json\/wp\/v2\/tags?post=11837"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}