# ============================ # GOOGLE CRAWLERS (Allowed) # ============================ # Main Google Search User-agent: Googlebot Disallow: /search? Allow: / # Google Mobile User-agent: Googlebot-Mobile Disallow: /search? Allow: / # Google Image User-agent: Googlebot-Image Disallow: /search? Allow: / # Google News User-agent: Googlebot-News Disallow: /search? Allow: / # Google Video User-agent: Googlebot-Video Disallow: /search? Allow: / # Google AdsBot User-agent: AdsBot-Google Disallow: /search? Allow: / # Google Ads Mobile User-agent: AdsBot-Google-Mobile Disallow: /search? Allow: / # Google StoreBot User-agent: Storebot-Google Disallow: /search? Allow: / # Google Other Crawlers User-agent: APis-Google Disallow: /search? Allow: / # Google Favicon User-agent: Googlebot-Favicon Disallow: /search? Allow: / # Google Inspection Tool User-agent: Google-InspectionTool Disallow: /search? Allow: / # ============================ # BING / INDEXNOW (Allowed) # ============================ # Main Bing Bot User-agent: bingbot Disallow: /search? Allow: / # Microsoft Ad Bot User-agent: adidxbot Disallow: /search? Allow: / # Bing Webmaster Tools User-agent: BingPreview Disallow: /search? Allow: / # IndexNow protocol bot User-agent: IndexNow Disallow: /search? Allow: / # ============================ # OTHER IndexNow Partners # ============================ # Amazon Bot (IndexNow) User-agent: Amazonbot Disallow: /search? Allow: / # Naver User-agent: Yeti Disallow: /search? Allow: / # Yandex User-agent: Yandex Disallow: /search? Allow: / # Seznam.cz User-agent: SeznamBot Disallow: /search? Allow: / # Yep.com (Ahrefs Search Engine) User-agent: AhrefsSiteAudit Disallow: /search? Allow: / # ============================ # SEO TOOL BOTS (Allowed) # ============================ # Moz Pro Crawler User-agent: rogerbot Disallow: /search? Allow: / # Semrush Crawler User-agent: SemrushBot Disallow: /search? Allow: / # Ubersuggest Crawler User-agent: Ubersuggest Disallow: /search? Allow: / # ============================ # BLOCK ALL OTHER BOTS # ============================ User-agent: * Disallow: /