User-agent: * # AJAX/search endpoints produce no indexable content and should never be crawled. Disallow: /app/search_product_ajax Disallow: /app/part_search_autocomplete # Note: /app/search_parts is intentionally NOT blocked here. It sends an # "X-Robots-Tag: noindex" header, and crawlers must be able to fetch it to see that # header and drop already-indexed spam search URLs from their index. Once those URLs # are gone, a "Disallow: /app/search_parts" can be added to save crawl budget. Sitemap: https://flyparts.com/sitemap.xml