How to Custom Robots.txt, Ads.txt and Search Console in your blogger

In this video, I will show how to Custom Robots.txt, Ads.txt and Search Console in your blogger



What is Robots.txt?

​Robots.txt is a text file which contains few lines of simple code.



It is saved on the website or blog's server which instruct the web crawlers on how to index and crawl your blog in the search results.



That means you can restrict any web page on your blog from web crawlers so that it can't get indexed in search engines like your blog labels page, your demo page or any other pages that are not as important to get indexed.


Always remember that search crawlers scan the robots.txt file before crawling any web page.

Each blog hosted on blogger has its default robots.txt file which is something look like this:



User-agent: Mediapartners-Google



Disallow:



User-agent: *



Disallow: /search



Allow: /



Sitemap: http://example.blogspot.com/feeds/posts/default?orderby=UPDATED



What is Ads.txt File?

The “Ads.txt” is simply a text file that can be placed on your blog's server and carries information about the advertising networks used by the blog's owner and the owner authorised these ads networks to show ads on the website. The main idea behind using this ads.txt file is to protect publishers from frauds and viruses as you will only authorise those ads network to display ads on your website which you're aware of and hence any malware script that showing ads on your blog without your permission from any unknown sources, cannot perform the task it ought to do.