Below is the code to use: User-agent: * Disallow: /xyz/ Disallow: /xyz Disallow: /feeds/ Disallow: /feeds Disallow: /auction.php Disallow: /auctions.php Disallow: /auctions.php Disallow: /get.php Disallow: /wp-content/ Disallow: /wp-content Disallow: /wp-content/plugins/ Disallow: /wp-content/plugins