# https://www.robotstxt.org/robotstxt.html User-agent: * Disallow: /*.aspx$ Disallow: /404/ Disallow: /admin/ Disallow: /helpertest/ Disallow: /cron_job/ Disallow: /resume_image/ Disallow: /app_connect/ Disallow: /android_api/ Disallow: /agency/ Disallow: /api/*.php$