User-agent: CatchBot Disallow: / User-agent: exabot Disallow: / User-agent: psbot Disallow: / User-agent: twiceler Disallow: / User-agent: * Disallow: */N1-P*-M*-A*.html User-agent: * Crawl-delay: 10