I’ve written about competitive tools before, but my last post was more of a general tool roundup. This post addresses a huge issue with competitive search marketing tools: accuracy, or lack thereof.
There are several tools I’ve been keeping an eye on for the past year or so: SpyFu, SEMrush, and KeywordCompetitor. I check these out from time to time, because aside from the known issues with spy tools in general (including impression jacking and clickfraud), I would love to have one I could trust for competitive information. Last week I decided to dive into SpyFu and take a closer look, in hopes that it had improved over the last year or so.
The verdict is mixed. While SpyFu provides some interesting information, I still have serious concerns about the accuracy of their data. They are vague about their data collection and manipulation methodologies, and what they do share is sketchy. SpyFu’s accuracy may improve somewhat with large accounts that have more impressions to scrape, but for ordinary sites, it’s not impressive.
A quick sanity check against my client’s Adwords data showed that:
- SpyFu states that the average ad position is 3 times lower than it actually is.
- They estimate my client’s daily ad budget at 50% of actual, but quietly acknowledge a margin of error close to 100%.
- Among the Ten Best Paid Keywords, SpyFu includes a keyword that performs below 200 other keywords in their account, no matter which metrics you look at. I have no idea what SpyFu’s criteria for “best” are, and they don’t elaborate.
- The number of daily paid clicks listed in SpyFu was less than half of the real number.
- An ad spend comparison over time showed that SpyFu captured some trends in very general terms, but highs and lows were greatly exaggerated, and they were way off on dollar amounts.
All of that said, there is some value to relative analysis. If you know how much they are off on your site, you can make a general guess as to how far they’re off on your competitors’ site. While there’s no way of knowing which numbers are accurate and which aren’t, having some kind of scale to go by is better than nothing. It may also be good for sparking new ideas that you hadn’t considered, whether it’s a long-tail keyword or a different way of wording your ad copy.
Everyone is probably sick of hearing me say this, and I understand that directional information can be useful, but if my experience as a research project manager taught me nothing else, it’s this: Don’t make business decisions based on data you know to be significantly inaccurate. If you find yourself drawn to use SEM spy tools (and I’ll be the first to admit that it’s tempting), I’d suggest doing some good, solid research on findings before making any big changes, just to be sure you aren’t charging headfirst down a rat hole with your client’s cash. ROI is more important now than ever before, so there’s little room for error when it comes to your search marketing budget.
It’s disappointing, because a good competitive tool could be incredibly valuable for optimizing your SEM campaign effectiveness. But at least you know that if your competitors are using these tools, they won’t get good intelligence about you.
