What ever Google giveth, Google can take away.  That’s t he case for Google Offers.  If you did a search for

[offers] in Google, you’ll notice that the Google Offers page is the first thing to be shown in the search listing.  But, upon closer inspection, you’ll see that Google has decided to block the page from spiders with their robots.txt file.  The page also says “This site is no longer being supported.”

Notice the screen shot for the search “offers” found currently in Google:

Image from Search Engine Land

Image from Search Engine Land

Although Google Offers is on the top of the results, the snippet reads, “A description for this result is not available because of this site’s robots.txt – learn more.”  Because you get a page that would return a 404 status code when you click through, shouldn’t that mean that Google should remove the page from the search results?

Here is what the Google Offers page looks like:

Image from Search Engine Land

Image from Search Engine Land

This is where old links and a site’s reputation (after all, Google has a pretty big reputation) can continue to keep content showing up after any and all content has been changed or deleted from the page.  It’s funny how even with the page being blocked by the robots.txt, Google’s crawlers are still trying to access the URL to see if the disavaow and 404 status has changed.

Original Source by