\n\n\n\n\n<\/p>
Why Would You Want to Stop Search Engines From Indexing Your Site?<\/h2>
There are some cases where people want to discourage search engines from indexing their sites:<\/p>
- \n
- Unfinished websites<\/strong> — at this stage of errors and trials, it’s best not to have your website available to the public eyes. We recommend using a WordPress staging environment<\/a> when performing tests on your site.<\/li>\n\n\n\n
- Restricted websites <\/strong>— if you plan to have an invite-only website, you do not want it to get listed on SERPs.<\/li>\n\n\n\n
- Test accounts <\/strong>— web owners create a site duplicate for testing and trial purposes. Since these sites are not designed for the public, don’t let it get indexed by search engines.<\/li>\n<\/ul>
So how do you block search engines from indexing your site? Well, take a look at several options below and try it yourself.<\/p>
1. Discouraging Search Engines From Indexing WordPress Sites<\/h2>
The simplest way to stop search engines from indexing your website is by preventing them from crawling it<\/a>. To do it, you need to edit your WordPress website’s robots.txt<\/strong> file<\/a>. Here are a few ways to achieve that:<\/p>
Using the WordPress Built-In Feature<\/h3>
Editing WordPress robots.txt is quite easy as you only need to use a WordPress built-in feature. Here’s how:<\/p>
- \n
- Login to the WordPress admin area and go to Settings<\/strong> →<\/strong> Reading<\/strong>.<\/li>\n\n\n\n
- Scroll down and locate the Search Engine Visibility <\/strong>option.<\/li>\n\n\n\n
- Check the option that says Discourage search engines from indexing this site.<\/strong>
![\"This](\"\/tutorials\/wp-content\/uploads\/sites\/2\/2017\/01\/search-engine-visibility-option-on-wordpress.png\")
<\/li>\n\n\n\n- Save Changes, <\/strong>and that’s it! WordPress will automatically edit its robots.txt<\/strong> file for you.<\/li>\n<\/ol>
Editing robots.txt File Manually<\/h3>
If you prefer the manual option, you can use File Manager <\/strong>or an FTP client<\/a> to edit the robots.txt<\/strong> file.<\/p>
In this article, we’ll show you how to do it through the hPanel’s File Manager:<\/p>
- \n
- Login to hPanel and locate File Manager <\/strong>in your dashboard.<\/li>\n<\/ol>
![\"hPanel](\"https:\/\/www.hostinger.com\/tutorials\/wp-content\/uploads\/sites\/2\/2023\/04\/hpanel-dashboard-filemanager-highlighted-1024x627.png\")
<\/figure><\/div>- \n
- Go to your WordPress root directory folder (in most cases, it’s public_html)<\/strong> and find the robots.txt<\/strong> file. If you can’t find it, create a new blank file.<\/li>\n\n\n\n
- Right-click on the file and select Edit<\/strong>.<\/li>\n<\/ol>
![\"Robots.txt](\"https:\/\/www.hostinger.com\/tutorials\/wp-content\/uploads\/sites\/2\/2023\/03\/filemanager-publichtml-robotstxt-edit-1024x730.png\")
<\/a><\/figure><\/div>Enter the following syntax:\n<\/p>
User-agent: *\nDisallow: \/<\/pre>
The code above will prevent search engines from indexing your whole site<\/strong>. If you want to apply the disallow rule to a specific page<\/a>, write the page’s subdirectory and slug. For example: Disallow \/blog\/food-review-2019.<\/strong><\/p>
\nPro Tip<\/h4>\n
The syntaxes in robots.txt<\/strong> files are case sensitive, so be careful when editing.<\/p>\n <\/div>\n\n\n\n<\/p>
2. Password Protecting Your WordPress Website<\/h2>
Search engines and web crawlers don’t have access to password-protected files. Here are a few methods to password protect your WordPress site:<\/p>
Using Hosting Control Panel<\/h3>
If you are a Hostinger client, you can password protect your website using hPanel’s Password Protect Directories<\/strong> tool:<\/p>
- \n
- Access hPanel and navigate to Password Protect Directories<\/strong>.<\/li>\n\n\n\n
- Enter your root directory into the first field.<\/li>\n\n\n\n
- Once the directory is selected, enter your username and password and click Protect<\/strong>.
![\"Password](\"\/tutorials\/wp-content\/uploads\/sites\/2\/2019\/11\/password-protect-directories.png\")
<\/a><\/li>\n<\/ol>\nPro Tip<\/h4>\n
If your root directory is public_html<\/strong>, leave the directory column blank.<\/p>\n <\/div>\n\n\n\n<\/p>
The process in cPanel is also quite similar:<\/p>
- \n
- Log in to your cPanel account and head to Directory Privacy<\/strong>.
![\"This](\"\/tutorials\/wp-content\/uploads\/sites\/2\/2017\/01\/directory-privacy-tool-on-cpanel.png\")
<\/li>\n\n\n\n- Select your root directory. In our case, it’s public_html<\/strong>.<\/li>\n\n\n\n
- Check the Password protect this directory<\/strong> option, and name the protected directory. Press Save<\/strong>.
![\"This](\"\/tutorials\/wp-content\/uploads\/sites\/2\/2017\/01\/directory-privacys-password-protecting-feature-on-hpanel.png\")
<\/li>\n\n\n\n- Create a new user to log in to the protected website, and that’s it!<\/li>\n<\/ol>
Using WordPress Plugins<\/h3>
There are tons of plugins that can help password protect your site. However, the Password Protected<\/strong><\/a> plugin might just be the best one out there. It’s been tested with the new WordPress update, and it’s pretty straightforward to use.<\/p>
After installing the plugin<\/a> and activating it, head to Settings → Password Protected <\/strong>and configure the settings to match your needs.<\/p>
3. Removing Indexed Page From Google<\/h2>
Don’t worry if Google has indexed your site. You can remove it from SERPs by following these steps:<\/p>
- \n
- Set up Google Search Console for your website.<\/li>\n\n\n\n
- Access Google Search Console of your newly added website and scroll down to Legacy tools and reports → Removals.<\/strong><\/li>\n\n\n\n
- Click the Temporarily hide button<\/strong> and enter the URL you want to remove from Google.<\/li>\n\n\n\n
- On a new window, choose Clear URL from cache and temporarily remove from search, <\/b>then Submit Request<\/strong>.
![\"This](\"\/tutorials\/wp-content\/uploads\/sites\/2\/2017\/01\/google-search-console-submit-removal-request.png\")
<\/li>\n<\/ol>And that’s it! Google will temporarily remove your site from search results. Make sure to apply the previous methods to prevent Google from indexing your site again.<\/p>
Conclusion<\/h2>
There you have it! Quick and easy ways to discourage search engines from indexing your sites. Here’s a quick recap of the methods we’ve learned today:<\/p>
- \n
- Edit the robots.txt<\/strong> file, which can be performed automatically or manually.<\/li>\n\n\n\n
- Password protect your website by using a plugin or your hosting control panel.<\/li>\n\n\n\n
- Remove indexed pages from Google via Google Search console.<\/li>\n<\/ul>
If you have any other methods, or if you have any questions, please do let us know in the comments. Good luck!<\/p>
\nLearn More About WordPress SEO<\/h4>\n
WordPress SEO Tips<\/a>
\nHow to Create a Sitemap in WordPress<\/a>
\nHow to Add WordPress Schema Markup<\/a>
\nHow to Add Meta Description and Meta Title in WordPress<\/a>
\nHow to Setup AMP (Accelerated Mobile Pages) in WordPress<\/a>
\nHow to Add Breadcrumbs in WordPress<\/a> <\/p>\n <\/div>\n<\/p>\n","protected":false},"excerpt":{"rendered":"WordPress site owners will do anything to get their websites indexed. However, you might not want search engines to crawl through your website if it’s still in development. In a case like this, it’s recommended to discourage search engines from indexing your site. Stick with us if you want to learn more about this topic! […]<\/p>\n
- Click the Temporarily hide button<\/strong> and enter the URL you want to remove from Google.<\/li>\n\n\n\n
- Select your root directory. In our case, it’s public_html<\/strong>.<\/li>\n\n\n\n
- Right-click on the file and select Edit<\/strong>.<\/li>\n<\/ol>
- Go to your WordPress root directory folder (in most cases, it’s public_html)<\/strong> and find the robots.txt<\/strong> file. If you can’t find it, create a new blank file.<\/li>\n\n\n\n
- Scroll down and locate the Search Engine Visibility <\/strong>option.<\/li>\n\n\n\n
- Restricted websites <\/strong>— if you plan to have an invite-only website, you do not want it to get listed on SERPs.<\/li>\n\n\n\n