الآن ، قم بإنشاء ملف "robots.txt" في دليلك الجذر. انسخ النص أعلاه والصقه في الملف النصي.
The robots. txt file, also known as the robots exclusion protocol or standard, is a text file that tells web robots (most often search engines) which pages on your site to crawl. It also tells web robots which pages not to crawl.
