User-agent: * Allow: / Disallow: /*.py$ Disallow: /*.sh$ Disallow: /*.log$ Sitemap: https://construction-recrea.com/sitemap.xml # Crawl-delay: 1