The free robots.txt file generator allows you to easily create a robots.txt file for your website based on your input.

Robots.txt is a file you can place in the root folder of your website to help search engines index your site better. Search engines like Google use website crawlers, robots that crawl all the content on a website. There may be parts of your website, such as admin pages, that you don’t want to be crawled and included in users’ search results. You can explicitly ignore these pages by adding them to your file. The Robots.txt file uses something called the Robots Exclusion Protocol. This website will easily generate a file with the pages you want to delete.