# Arbyield - Robots.txt # https://arbyield.com User-agent: * Allow: / # Sitemap Sitemap: https://arbyield.com/sitemap-index.xml # Crawl-delay for polite crawling Crawl-delay: 1 # Disallow admin or private areas (if any) # Disallow: /admin/ # Disallow: /private/