Robots.txt search engine included banning method

The

Shanghai to track a particular link, love Shanghai also supports more precise control, please use this flag to write directly in a link:

search engine using spider program to automatically access the webpage on Internet and web information acquisition. Spider during a visit to a website, will be the first to check the website of the root domain of whether there is a robots.txt text file, this file is used to specify the spider on your site capture range. You can create a robots.txt on your website, in the document statement in the site do not want to be included in the search engine or part of the specified search engine only included a specific part.

>

robots.txt

2

? file?The

reading

to allow other search engines to track, but only to prevent love Shanghai track your web link, please send this page into Meta Tags:

 

If you don’t want to

4

What is the

please note that only if your site contains not to be included in the search engine content, only need to use the robots.txt file. If you want all the content included in the search engine website, do not create robots.txt files.

if the other site links to prohibit included web settings in your robots.txt file, then these pages may still appear in Shanghai love search results, but the content on your page will not be crawled, indexed and show the construction, love Shanghai in search results show only to other sites related to you the description of the page.

sign in

I set in the robots.txt forbidden love Shanghai included my site, why love Shanghai also appears in the search results of

robots.txt file in the

?

3

from search engine tracking web links, but only on the web pages indexed

robots.txt file should be placed in the root directory of the web site. For example, when the spider to access a web site (such as 贵族宝贝abc贵族宝贝), it will first check the 贵族宝贝abc贵族宝贝/robots.txt file exists in the site, if Spider finds the file, it will be based on the contents of this document, to determine the scope of the right to limit access to it.

if you don’t want to search engines to track the links on this page, and link weight transfer, please send this page into Meta Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *