\n\n\n\n\n<\/p>
Why Would You Want to Stop Search Engines From Indexing Your Site?<\/h2>
There are some cases where people want to discourage search engines from indexing their sites:<\/p>
- \n
- Unfinished websites<\/strong> — at this stage of errors and trials, it’s best not to have your website available to the public eyes. We recommend using a WordPress staging environment<\/a> when performing tests on your site.<\/li>\n\n\n\n
- Restricted websites <\/strong>— if you plan to have an invite-only website, you do not want it to get listed on SERPs.<\/li>\n\n\n\n
- Test accounts <\/strong>— web owners create a site duplicate for testing and trial purposes. Since these sites are not designed for the public, don’t let it get indexed by search engines.<\/li>\n<\/ul>
So how do you block search engines from indexing your site? Well, take a look at several options below and try it yourself.<\/p>
1. Discouraging Search Engines From Indexing WordPress Sites<\/h2>
The simplest way to stop search engines from indexing your website is by preventing them from crawling it<\/a>. To do it, you need to edit your WordPress website’s robots.txt<\/strong> file<\/a>. Here are a few ways to achieve that:<\/p>
Using the WordPress Built-In Feature<\/h3>
Editing WordPress robots.txt is quite easy as you only need to use a WordPress built-in feature. Here’s how:<\/p>
- \n
- Login to the WordPress admin area and go to Settings<\/strong> →<\/strong> Reading<\/strong>.<\/li>\n\n\n\n
- Scroll down and locate the Search Engine Visibility <\/strong>option.<\/li>\n\n\n\n
- Check the option that says Discourage search engines from indexing this site.<\/strong>
![\"This](\"\/tutorials\/wp-content\/uploads\/sites\/2\/2017\/01\/search-engine-visibility-option-on-wordpress.png\")
<\/li>\n\n\n\n- Save Changes, <\/strong>and that’s it! WordPress will automatically edit its robots.txt<\/strong> file for you.<\/li>\n<\/ol>
Editing robots.txt File Manually<\/h3>
If you prefer the manual option, you can use File Manager <\/strong>or an FTP client<\/a> to edit the robots.txt<\/strong> file.<\/p>
In this article, we’ll show you how to do it through the hPanel’s File Manager:<\/p>
- \n
- Login to hPanel and locate File Manager <\/strong>in your dashboard.<\/li>\n<\/ol>
![\"hPanel](\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/2\/2023\/04\/hpanel-dashboard-filemanager-highlighted.png\/w=1024,h=1024,fit=scale-down\")
<\/figure><\/div>- \n
- Go to your WordPress root directory folder (in most cases, it’s public_html)<\/strong> and find the robots.txt<\/strong> file. If you can’t find it, create a new blank file.<\/li>\n\n\n\n
- Right-click on the file and select Edit<\/strong>.<\/li>\n<\/ol>
![\"Robots.txt](\"https:\/\/imagedelivery.net\/LqiWLm-3MGbYHtFuUbcBtA\/wp-content\/uploads\/sites\/2\/2023\/03\/filemanager-publichtml-robotstxt-edit.png\/w=1024,h=1024,fit=scale-down\")
<\/a><\/figure><\/div>Enter the following syntax:\n<\/p>
User-agent: *\nDisallow: \/<\/pre>
The code above will prevent search engines from indexing your whole site<\/strong>. If you want to apply the disallow rule to a specific page<\/a>, write the page’s subdirectory and slug. For example: Disallow \/blog\/food-review-2019.<\/strong><\/p>
\n\n - Right-click on the file and select Edit<\/strong>.<\/li>\n<\/ol>
- Go to your WordPress root directory folder (in most cases, it’s public_html)<\/strong> and find the robots.txt<\/strong> file. If you can’t find it, create a new blank file.<\/li>\n\n\n\n
- Scroll down and locate the Search Engine Visibility <\/strong>option.<\/li>\n\n\n\n
- Restricted websites <\/strong>— if you plan to have an invite-only website, you do not want it to get listed on SERPs.<\/li>\n\n\n\n