# /robots.txt file for http://www.spidernet.net/ User-agent: psbot Disallow: / User-agent: * Disallow: /Main/Admin Disallow: /aspnet_client Disallow: /Main/bin Disallow: /Main/client_scripts Disallow: /Main/CUSTOM Disallow: /Main/IntranetModules Disallow: /Main/Modules Disallow: /Main/Templates Disallow: /Main/Utilities Disallow: /webctrl_client