Sitemap: https://www.curbed.com/sitemap.xml Sitemap: https://www.curbed.com/_news.xml User-agent: Mediapartners-Google* Disallow: User-agent: Twitterbot Disallow: /amp/* User-agent: * Disallow: /contactus/ Disallow: /search.html Disallow: /search/ Disallow: *?origSession=* User-agent: GPTBot Disallow: / User-Agent: PerplexityBot Disallow: / User-agent: Google-Extended Disallow: User-agent: Amazonbot Disallow: / User-agent: Applebot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: Diffbot Disallow: / User-agent: FacebookBot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: PerplexityBot Disallow: / User-agent: YouBot Disallow: /