User-agent: HTTrack 3.0x Disallow: / User-agent: psbot Disallow: / User-agent: * Disallow: /cgi-bin/ Disallow: /error/