User-agent: Scrapy Allow: / Sitemap: https://byvaniesk.sk/sitemap.xml # global User-agent: * Disallow: /xmlrpc.php