It seems that whether Google planned for this to happen or not, Google has been withholding data from publishers who would buy ads on their service. For these publishers looking to get a broader visible marketing scheme, this kind of sucks. As I said, I don’t think Google has meant to do this, but even if this is true or not, it seems to work. This is something that could be called the “Not Provided” scheme.
For the last couple of years, beginning all the way back in October of 2011, Google began to hold back information from publishers even though they originally offered it for free. This information was “referrer data.” This particular information was what allowed publishers to know what terms that users would use when they did a search on Google, and then finally heading away from Google to the site of the publisher.
Before all of this happened, you could tell exactly what people typed in to find your site, but not anymore. The mechanism that was used by Google was standard fare across the entire industry that allowed analytics tools to tap into.
At first, not many terms were being withheld. Some of these terms would also show up as “not provided.” But, as time went on, more and more sites have been witness to the increasing “not provided” and withheld terms world of hurt. Some may even wonder if, eventually, we’ll be seeing a 100% not provided world.
Many might be wondering why would Google do this? How could they break what isn’t broken? This system has been around since before Google was Google. Google stated that it is for the good of the searchers. To be specific, it’s for their privacy. There’s the chance that somebody malicious could be searching for sensitive information. Google believes that if they can withhold search timers, they can effectively stop any potential information leaks as well as eavesdropping.
Even if you don’t appreciate what Google’s doing, you have to agree that they’re doing it for the good of their search user base. But, even though they are trying to stop leaks from happening, it can still happen in three different ways:
1) Search terms that get suggested by Google Instant autocomplete
2) Search terms that Google provides to publishers through its Google Webmaster Central service
3) Search terms that Google continues to transmit across the open web to its advertisers
Number three is especially important to look at. Advertisers can still receive referrer information the traditional way. Google went out of it’s way to make this happen. Advertisers can still get this information without logging in to some closed Google system.
What does this mean? By shifting to this new system, this loophole could be putting a price on privacy. There’s only so much money that Google’s willing to spend before it gets just too darn expensive for itself.
The only course of action that Google can take that would keep them in everybody’s good graces is to increase ad revenues. This is the second loophole. By using Google Webmaster Central, Publishers can, after logging in, see the terms that drive traffic to their sites for free.
But, here are the stipulations. Publishers will only be able to see the top 2,000 queries for any particular day that are sending traffic to your site, but only for the last 90 days. Google expanded their previously small number of search terms in early 2012. It’s possible that publishers can see thousands and thousands of different queries due to the fact that the exact terms change everyday.
Because of the 90 day window, the historical data for many sites have been lost, and will always be lost, even though seeing the top 100 or 200 queries are great. If you want to see how your top terms are comparing to 12 months ago, or want to compare your traffic, you can’t. Certainly not with Google Webmaster Central. They aren’t going to be storing it for longer.
Why won’t Google expand the period of time that this data is retained? Here is a recent discussion with Matt Cutts, the head of Google’s web spam. If the video doesn’t start at the correct spot, you can find it at 38:15:
Last month, Google quietly announced a new “Paid and Organic” report for those with AdWords accounts. By linking to your Google Webmaster Central account, you can store those search terms that Google has been withholding out of Google Webmaster Central after 90 days. It automatically pulls the search terms constantly. This means no Python script is required.
If you want to see your top terms after that, just:
1. Select “All Online Campaigns.
2. Make an empty campaign, assuming you don’t already have one set up.
3. Click on the “Dimensions” tab, change “View” to “Paid & organic.”
Boom. There’s all the info you wanted. Top terms, sortable by clicks, queries and all that good stuff will now be available to you.
A good bit of news to come out of this is that you won’t be required to be a paying AdWords customer to do this. All you need is an AdWords account. The downside is how Google forces publishers to obtain information about their “non-paid” listings.
In a way, it doesn’t really appear that Google did this with any real intention, but with helding terms to push AdWords. All in all, the road to Hell is paved with good intentions. This system was pitched as if it was protecting user privacy, although there were some pretty big loopholes. One of these loopholes, as mentioned earlier, included an explicit one for its advertisers. The system that is in place now only encourages people to use AdWords.
It makes Google appear that they don’t want publishers to be fully appraised unless there’s a way to get a little bit of ad revenue from them first. This doesn’t really make Google look like the good guy.
According to Google, they stated,
“We plan to provide a year’s worth of data in the Top Search Queries report in Webmaster Tools.”
So far, there isn’t a timetable for this to happen, but it could happen soon.