{"id":11837,"date":"2018-08-17T15:47:35","date_gmt":"2018-08-17T18:47:35","guid":{"rendered":"https:\/\/blog.hostinger.io\/br-tutoriais\/?p=11837"},"modified":"2023-07-21T17:09:30","modified_gmt":"2023-07-21T20:09:30","slug":"como-criar-robots-txt-bloquear-mecanismos-busca","status":"publish","type":"post","link":"\/br\/tutoriais\/como-criar-robots-txt-bloquear-mecanismos-busca","title":{"rendered":"Como criar robots.txt e bloquear os mecanismos de busca"},"content":{"rendered":"<p><span style=\"font-weight: 400\">Os bots dos <strong>mecanismos de busca<\/strong> est&atilde;o constantemente escaneando sites para serem indexados. Mas por alguma raz&atilde;o voc&ecirc; pode optar por esconder seus sites. Neste caso o <\/span><b>robots.txt<\/b><span style=\"font-weight: 400\"> pode ser utilizado para impedir que os bots acessem a sua p&aacute;gina.<\/span><\/p><p><span style=\"font-weight: 400\">Neste tutorial, voc&ecirc; vai aprender como criar&nbsp;<\/span><b>robots.txt <\/b>&ndash; chamado erroneamente por algumas pessoas de robot.txt<span style=\"font-weight: 400\">.<\/span><\/p><figure class=\"wp-block-image size-large\"><a href=\"https:\/\/assets.hostinger.com\/content\/tutorials\/pdf\/Speed-Up-Your-Website-BR.pdf\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" width=\"1024\" height=\"283\" src=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/public\" alt=\"\" class=\"wp-image-30339\" srcset=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=1024,fit=scale-down 1024w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=300,fit=scale-down 300w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=150,fit=scale-down 150w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=768,fit=scale-down 768w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2022\/12\/eBook_-Speed-Up-your-website-BR.png\/w=1536,fit=scale-down 1536w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure><p>\n\n\n\n\n\n\n<\/p><h2 class=\"wp-block-heading\" id=\"h-do-que-voce-vai-precisar\"><b>Do Que Voc&ecirc; Vai Precisar<\/b><\/h2><p><span style=\"font-weight: 400\">Antes de iniciar voc&ecirc; vai precisar:<\/span><\/p><ul class=\"wp-block-list\">\n<li>Acesso ao painel de controle de sua hospedagem ou FTP.<\/li>\n<\/ul><h2 class=\"wp-block-heading\" id=\"h-1-acesse-o-servidor-e-crie-o-arquivo-robots-txt\"><b>1. Acesse o Servidor e Crie o Arquivo robots.txt<\/b><\/h2><p><span style=\"font-weight: 400\">Antes de mais nada, voc&ecirc; precisa criar um arquivo para criar&nbsp;<\/span><b>robots.txt<\/b><span style=\"font-weight: 400\">. Para isso voc&ecirc; pode usar um cliente FTP ou o Gerenciador de Arquivos. O arquivo deve ser colocado na mesma pasta onde se encontra seu site (geralmente <\/span><b>public_html<\/b><span style=\"font-weight: 400\">). Neste tutorial vamos usar o Gerenciador de Arquivos para criar&nbsp;<\/span><b>robots.txt<\/b><span style=\"font-weight: 400\">:<\/span><\/p><div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"1024\" height=\"194\" src=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/public\" alt=\"Como Criar robots.txt e Bloquear os Mecanismos de Busca\" class=\"wp-image-36648\" srcset=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/w=1024,fit=scale-down 1024w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/w=300,fit=scale-down 300w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/w=150,fit=scale-down 150w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/05\/hpanel-gerenciador-de-arquivos-public_html-robots.txt_.jpg\/w=768,fit=scale-down 768w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div><h2 class=\"wp-block-heading\" id=\"h-2-editar-o-arquivo-robots-txt\"><b>2.&nbsp;Editar o Arquivo robots.txt<\/b><\/h2><p><span style=\"font-weight: 400\">Cada mecanismo de busca usa seu <em>crawler<\/em> (bot ou user-agent) pr&oacute;prio. Ao criar&nbsp;<\/span><b>robots.txt<\/b><span style=\"font-weight: 400\"> voc&ecirc; pode especificar o <em>crawler<\/em> usando o User-agent. Existem centenas de crawlers, mas os mais comuns s&atilde;o<\/span>:<\/p><ul class=\"wp-block-list\">\n<li>Googlebot<\/li>\n\n\n\n<li>Yahoo! Slurp<\/li>\n\n\n\n<li>bingbot<\/li>\n\n\n\n<li>AhrefsBot<\/li>\n\n\n\n<li>Baiduspider<\/li>\n\n\n\n<li>Ezooms<\/li>\n\n\n\n<li>MJ12bot<\/li>\n\n\n\n<li>YandexBot<\/li>\n<\/ul><p><span style=\"font-weight: 400\">Por exemplo, se voc&ecirc; quer impedir o <em>crawler<\/em> do Google de acessar o seu site, basta editar o <\/span>arquivo<span style=\"font-weight: 400\">&nbsp;com a seguinte regra:<\/span><\/p><pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">User-agent: Googlebot \nDisallow: \/example-subfolder\/<\/pre><p><span style=\"font-weight: 400\">Caso voc&ecirc; queira bloquear p&aacute;gina de todos os <em>crawlers<\/em> &eacute; s&oacute; usar o *:<\/span><\/p><pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">User-agent: *\n\nDisallow: \/<\/pre><p><span style=\"font-weight: 400\">Se voc&ecirc; quer prevenir que o <em>crawler<\/em> acesse algum diret&oacute;rio ou arquivo espec&iacute;fico, a regra &eacute; semelhante, mas voc&ecirc; precisa especificar o nome do arquivo ou pasta. Digamos que voc&ecirc; n&atilde;o quer que os <em>crawlers<\/em> acessem o diret&oacute;rio <\/span><b>artigos<\/b><span style=\"font-weight: 400\"> e o arquivo <\/span><b>private.php<\/b><span style=\"font-weight: 400\"> apenas. Nesse caso o robots.txt vai ficar assim:<\/span><\/p><pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">User-agent: Bingbot\n\nDisallow: \/subpasta-exemplo\/pagina-bloqueada.html<\/pre><p><span style=\"font-weight: 400\">Quando terminar de editar o arquivo <\/span><b>robots.txt<\/b><span style=\"font-weight: 400\"> n&atilde;o esque&ccedil;a de salvar as altera&ccedil;&otilde;es.<\/span><\/p><figure class=\"wp-block-image size-large\"><a href=\"\/br\/\" target=\"_blank\" rel=\"noreferrer noopener\"><img decoding=\"async\" width=\"1024\" height=\"300\" src=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/02\/BR-Web-hosting_in-text-banner.png\/public\" alt=\"\" class=\"wp-image-32927\" srcset=\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=1024,fit=scale-down 1024w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=300,fit=scale-down 300w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=150,fit=scale-down 150w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=768,fit=scale-down 768w, https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/53\/2023\/02\/BR-Web-hosting_in-text-banner.png\/w=1536,fit=scale-down 1536w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure><h2 class=\"wp-block-heading\" id=\"h-conclusao\"><b>Conclus&atilde;o<\/b><\/h2><p><span style=\"font-weight: 400\">Voc&ecirc; acabou de aprender como impedir que <em>crawlers<\/em> (ou bots) de mecanismos de busca acessem e indexem o seu site. Isso &eacute; &uacute;til para prevenir que seu site apare&ccedil;a nos mecanismos de busca, claro, se &eacute; isso mesmo que voc&ecirc; quer que aconte&ccedil;a.<\/span><\/p><p>\n\n\n<div class=\"protip\">\n                    <h4 class=\"title\">Aprenda mais sobre Otimiza&ccedil;&atilde;o de Sites<\/h4>\n                    <p><a href=\"\/br\/tutoriais\/otimizacao-de-sites\">O Que &eacute; Otimiza&ccedil;&atilde;o de Sites<\/a><br>\n<a href=\"\/br\/tutoriais\/como-criar-sitemap-para-wordpress\">Como Criar um Sitemap no WordPress<\/a><\/p>\n                <\/div>\n<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Os bots dos mecanismos de busca est&atilde;o constantemente escaneando sites para serem indexados. Mas por alguma raz&atilde;o voc&ecirc; pode optar por esconder seus sites. Neste caso o robots.txt pode ser utilizado para impedir que os bots acessem a sua p&aacute;gina. Neste tutorial, voc&ecirc; vai aprender como criar&nbsp;robots.txt &ndash; chamado erroneamente por algumas pessoas de robot.txt. [&#8230;]<\/p>\n<p><a class=\"btn btn-secondary understrap-read-more-link\" href=\"\/br\/tutoriais\/como-criar-robots-txt-bloquear-mecanismos-busca\">Read More&#8230;<\/a><\/p>\n","protected":false},"author":78,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"rank_math_title":"Como Criar robots.txt e Bloquear os Mecanismos de Busca","rank_math_description":"Para impedir que os buscadores encontrem alguma p\u00e1gina do seu site, veja como criar robots.txt e como us\u00e1-los do jeito certo.","rank_math_focus_keyword":"robots.txt","footnotes":""},"categories":[4941],"tags":[5831],"class_list":["post-11837","post","type-post","status-publish","format-standard","hentry","category-desenvolvimento-web","tag-criar-robot-txt"],"hreflangs":[{"locale":"en-US","link":"https:\/\/www.hostinger.com\/tutorials\/how-to-block-search-engines-using-robotstxt","default":0},{"locale":"pt-BR","link":"https:\/\/www.hostinger.com\/br\/tutoriais\/como-criar-robots-txt-bloquear-mecanismos-busca","default":0},{"locale":"es-ES","link":"https:\/\/www.hostinger.com\/es\/tutoriales\/como-bloquear-los-motores-de-busqueda-utilizando-robotstxt","default":0},{"locale":"en-UK","link":"https:\/\/www.hostinger.com\/uk\/tutorials\/how-to-block-search-engines-using-robotstxt","default":0},{"locale":"es-MX","link":"https:\/\/www.hostinger.com\/mx\/tutoriales\/como-bloquear-los-motores-de-busqueda-utilizando-robotstxt","default":0},{"locale":"es-CO","link":"https:\/\/www.hostinger.com\/co\/tutoriales\/como-bloquear-los-motores-de-busqueda-utilizando-robotstxt","default":0},{"locale":"es-AR","link":"https:\/\/www.hostinger.com\/ar\/tutoriales\/como-bloquear-los-motores-de-busqueda-utilizando-robotstxt","default":0},{"locale":"pt-PT","link":"https:\/\/www.hostinger.com\/pt\/tutoriais\/como-criar-robots-txt-bloquear-mecanismos-busca","default":0},{"locale":"en-IN","link":"https:\/\/www.hostinger.com\/in\/tutorials\/how-to-block-search-engines-using-robotstxt","default":0},{"locale":"en-CA","link":"https:\/\/www.hostinger.com\/ca\/tutorials\/how-to-block-search-engines-using-robotstxt","default":0},{"locale":"en-PH","link":"https:\/\/www.hostinger.com\/ph\/tutorials\/how-to-block-search-engines-using-robotstxt","default":0},{"locale":"en-MY","link":"https:\/\/www.hostinger.com\/my\/tutorials\/how-to-block-search-engines-using-robotstxt","default":0},{"locale":"en-AU","link":"https:\/\/www.hostinger.com\/au\/tutorials\/how-to-block-search-engines-using-robotstxt","default":0},{"locale":"en-NG","link":"https:\/\/www.hostinger.com\/ng\/tutorials\/how-to-block-search-engines-using-robotstxt","default":0}],"acf":[],"_links":{"self":[{"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/posts\/11837","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/users\/78"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/comments?post=11837"}],"version-history":[{"count":12,"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/posts\/11837\/revisions"}],"predecessor-version":[{"id":36654,"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/posts\/11837\/revisions\/36654"}],"wp:attachment":[{"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/media?parent=11837"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/categories?post=11837"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hostinger.com\/br\/tutoriais\/wp-json\/wp\/v2\/tags?post=11837"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}