User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /cgi-bin/ Disallow: /private/ # Allow Googlebot to crawl everything User-agent: Googlebot Disallow: # Allow Bingbot to crawl everything User-agent: Bingbot Disallow: # Allow all bots to access specific directories and favicon User-agent: * Allow: /articles/ Allow: /website/ Allow: /src/ Allow: /favicon Disallow: /scripts/ Disallow: /tmp/ # Block specific bots (if necessary) User-agent: BadBot Disallow: / User-agent: SiteCheckerBotCrawler Allow: / # Sitemap location Sitemap: https://kolodych.com/sitemap.xml