Hi and thank you for listening to SEO Fight Club. I’m Ted Kubaitis and I have 22 years of web development and SEO experience. I have patented web technologies and started online businesses. I am both an engineer and a marketer. My goal is to help you win your SEO fights.
This episode’s FREEBIE!
With every episode I love to give something away of high value. This episode’s SEO freebie is my own personal 40 point negative SEO checklist. If you are concerned about negative SEO then this checklist will help you conduct periodic audits for suspicious activity. This free download will save you hours on checking your websites for attacks.
You can download the freebie at: http://seofightclub.org/episode4
Negative SEO refers to the practice of using techniques to ruin a competitor’s rankings in search engines, but really just in Google.
Most people think of Negative SEO as unwanted spammy backlinks. To put it simply, those people have a failure of imagination. To understand the dozens of tactics of negative SEO you first have to understand its numerous motivations and intentions.
Here are some of those intentions:
- Get a site banned from search results.
- Get a page demoted in the rankings.
- Steal a website’s customers.
- Steal a website’s content.
- Steal a website’s viewers or contributors.
- Change the topics a webpage ranks for.
- To hurt the reputation of the website.
- To hurt the reputation of the author.
- To hurt the reputation of a product.
- To disrupt a website’s ad revenue.
- To squander a website’s ad spend.
- To ruin a website’s content or data.
- To disrupt the operation of the website.
- To cause financial harm to a website.
- To cause confusion about products and services or to blur the lines
- differentiating them.
- To slander or make false claims about a website, business, or person.
- To bully, harass, or otherwise intimidate a person, website, or business.
I am certain there are more I’m not thinking of. Very quickly people can start to sense how they or people they know have already been hurt by negative SEO. I’m sure you are already sensing the kind of uneasy ground we all stand on now. When a business gets hit by these it can be crippling. It is absolutely devastating when you are hit by multiple of these at the same time.
There are an estimated 40 kinds of negative SEO tactics that I know of and the list seems to grow every year as Google adds new things to punish that can be weaponized by negative SEO practitioners.
- spammy links
- false ratings and reviews (worst product ever)
- rating and review spam (this is ok but my product over here is better)
- content spam
- content theft
- GoogleBot interruption
- False canonicalization
- False authorship
- toxic domain redirection
- denial of service
- crippling the site’s speed
- fraudulent DMCA takedown
- Cross site scripting
- Hacking the site
- Keyword bombing posts and comments
- Website or Social Identity Theft
- Fake complaints
- Injecting Malware
- Click-spamming ads
- CTR and other false signal bots
- faking Email spam to get competitor publicly blacklisted
- Fake bots that claim to be competitor and behave badly (again to get publicly
- Submitting alternative URLs or hosts to exploit missing canonical tags
- Link building a page into a new keyword context
- Link building incorrect pages into the keyword context for bad experience
- Flooding a web database with bogus data
- Posting adult or damaging content
- Linking from adult sites or other toxic locations
- Disavow misuse to declare a competitor as spam to google
- Misuse of Google’s spam report form
- Inserting grammar, spelling, and content encoding errors
- Unwanted bot and directory submission
- Redirecting many domains bot activity onto a target site all at once.
- Negative and false Press
- Inclusion in blog networks, link wheels, other linking schemes
- Content overflow… keep posting to a one page thread until the page is too big
- Topic Flooding… flood the forum with so many crappy posts the forum becomes unusable
- Keep replying to bad or outdated posts so it keeps fresher or better content off the main indexes
- Pretend to be a competitor and ask for link removals
- Flooding junk traffic to a site so Google gets the wrong idea about the site’s
- audience location or demographics
- Domain squatting and hijacking
I’m not sure how many of these are still effective but I want to tell you about my experience to one of them that was extremely devastating. The GoogleBot interruption attack.
I used to say “negative SEO isn’t real”. My desk is in the engineering bullpen. There are no cubes or offices. This allows everyone to overhear of all of the issues of the day. I heard the network admin complaining about very weak denial of service attacks on our websites.
The specific type of denial of service attack my network administrator was battling is called “slow loris”.
Slow Loris Defined
Slowloris is a piece of software written by Robert “RSnake” Hansen which allows a single machine to take down another machine’s web server with minimal bandwidth and side effects on unrelated services and ports.
Slowloris tries to keep many connections to the target web server open and hold them open as long as possible. It accomplishes this by opening connections to the target web server and sending a partial request. Periodically, it will send subsequent HTTP headers, adding to—but never completing—the request. Affected servers will keep these connections open, filling their maximum concurrent connection pool, eventually denying additional connection attempts from clients.
The attacks didn’t make much sense. We could detect them and block them within an minutes, and they would keep appearing every two to three weeks. This went on and on months and possibly years. We are not entirely sure when the attacks started.
We didn’t understand the motivation behind these attacks. We could easily counter them. Why was someone working so hard to do this when we could stop it so quickly when it happens?
Several months went by and I was in a meeting trying to explain the unusual volatility in SEO revenue. While in that meeting I got chills down my spine. I had a thought I just couldn’t shake. Later, I put the chart of the slow loris attacks on top of the chart for SEO revenue, and every drop in SEO followed a slow loris attack. From then on I knew negative SEO was very real and very different from what everyone thought negative SEO was. This was a very effective negative SEO attack. It had absolutely nothing to do with backlinks.
I spent the next few weeks learning what I could about this attack. I learned how it worked. Basically, the attacker was waiting for an indication that googlebot was crawling our site, then they would launch the attack so our web server would return 500 errors to Googlebot. Googlebot would remove the pages that returned a 500 error from the search results. Googlebot would not retest the pages for days This is not the case today. Google will retests pages within hours now, but it was the case at the time. To make things even worse, once googlebot found working pages again they would reappear several places lower in the results for about 2 or 3 weeks before recovering to their original positions.
These attacks that we assumed were unsuccessful and weak were totally the opposite. They were both successful and devastating. They had lasting effects and the timing of the attacks was keeping us pinned down in the rankings.
If you were only watching rankings then this would just look like normal everyday Google dance. No one cares that one week a page went down 6 spots and then two weeks later comes back up. We have thousands of pages across 20 websites, and most of those websites are on shared servers. Google Search Console tools doesn’t let you see combined impact across sites. If I wasn’t reporting on SEO revenue, which many SEOs object to, this would have continued undetected.
So now I knew the attack was real, and I knew how it worked. So how do I stop them?
For the interruption attack to be effective the attacker needs to time his attack to coincide with Googlebot’s visit to the website. How can they do this? There are five ways I can think of:
Monitor the google cache and when a cache date is updated you know googlebot is crawling.
Analyze the cache dates and estimate when Googlebot will come back
Cross-site scripting to see visiting User agents
Attack often and hope for the best
Hack the system and access the logs
I believed the attacker was probably doing #1.
I put a NOARCHIVE tag on all of our pages. This prevents Google from showing the Cached link for a page. This would stop the attacker from easily monitoring our cache dates.
The attacks stopped for about 4 months following that change. I thought we had won, but I was wrong.
Late in the third quarter of 2014 we were hit hard by an extremely-precise attack. Our attacker then went dormant. The attacker had his attack capability back and the attacker knew we were on to them. We suspected he was picking his timing more carefully now. It was the second time I got chills down my spine. Like most online stores we do most of our sales in the fourth quarter. My suspicion was that the attacker was lying in wait for Black Friday. One of these attacks the week before Black Friday would cripple our top performing weeks of the year.
We scrambled to figure out how they were timing the attacks with GoogleBot. We failed. The week before Black Friday we were hit harder than we were ever hit before. We lost seventy percent of our SEO revenue for the month. I was devastated.
The company accepted that the attacks were amounting to significant losses. We knew the attacks were going to continue. We invested hundreds of thousands of dollars in security appliances that detect and block hundreds of different attacks.
It took six months to get all the websites protected by the new firewalls and to get all the URLs properly remapped. We had finally stopped the onslaught but at a pretty heavy cost in time and money. This year we have seen high double-digit growth in SEO. It is due to stopping the negative SEO attacks.
The attack I just described I call a “GoogleBot Interruption Attack”. Negative SEO is so new these attacks probably don’t have official names yet.
I have seen a number of other attacks too, but none we as crippling as the GoogleBot interruption attack.
Another attack I have encountered is when a black hat takes a toxic domain name that has been penalized into the ground and then points the DNS to your website. Some of those penalties appear to carry over at least for a short while. The worst is when a lot of these toxic domains are pointed all at once at your website.
Another similar attack to that is when an attacker redirects the URLs from the toxic sites to your URLs. This has the effect of giving your website a surplus of bad backlinks. What is scary about this is the attack can recycle those toxic backlinks and change targets over and over again.
Another attack is the attacker targets a page that is missing a canonical tag by submitting a version of the URL that works but Google has never seen before. This is done by adding things like a bogus URL parameter or anchor text. Then they link build to the bogus URL until it outranks the original. The original will fall out of the results as a lower PR duplicate. Then they pull the backlinks to the bogus URL, and they have effectively taken a page out of the Google index until Google recalculates PR again. Just put a canonical tag on every page and you can protect yourself from this one.
Another attack just requires a lot of domains and they don’t have to be toxic. The requirement is that they are indexed and visited by a LOT of bots. The attacker in many cases will point hundreds of these domains at a single website and use the collective of bot activity as a denial of service against the website.
I’m certain there are more kinds of attacks out there. It is limited to the creativity of the attackers, and the bad guys can be pretty creative. I am constantly afraid of what I might find next having run the gauntlet and paid the price already.
Your only hope is to accurately attribute SEO revenue and monitor it regularly. Conversions are good, but if you’re looking for strong signal on the health of your SEO then revenue is better. Revenue implies good indexation, rankings, traffic, and conversions all in one very sensitive gage. Conversions are good too, but the needle doesn’t move as much, making it harder to see the signals.
Secondly… sit next to your engineers. The issues they encounter are directly relevant to the effectiveness of your SEO. The frantic firefighting of the network administrator is one of the best indicators. Log serious events and plot them with revenue and other KPIs
Third… logs. The crawl error logs in Google Search Console and your web server logs tell you about the issues googlebot encounters and the attempts made on your server.
- Lots of 500 errors might signal a GoogleBot interruption attack.
- Lots of 404 errors might be toxic domain redirection.
- Lots of URLs in errors or duplicate content that make no sense for your website might signal canonical misuse.
Following each and every soul-crushing SEO revenue event I had to pour through the logs and testimony of everything to try and make sense of things. Not every SEO event was an attack. In many cases the events were caused by errors deployed on our websites. Or the marketing team installed a new problematic tracking pixel service. Several times the owners bought domains and pointed them at our sites not knowing that the previous owners had made them permanently toxic. As an SEO, you need to detect and address these as well:Revenue, Logs, and general awareness of daily events was critical to early detection.
when I went to the reader base of many popular SEO forums and blogs, I was ridiculed and called a liar for asking for help with a problem most SEOs had never seen or heard of before. It was all too common that the peanut gallery of SEO professionals would criticize me for not having links and kept saying I had the burden of proof. These were supposedly members of the professional SEO community, but it was just a political flame war. The black hat community actually helped me research the kinds of attacks I was facing, explained how they worked and suggested ideas for countering them. Google and the SEO community in general were very unsupportive. I’m going to remember that for a very long time.
For some reason we are a big target. It is probably because so many of our products compete with similar products that are often affiliate offerings. If you are an online retailer that does a lot of sales seasonally, you need to be on the look out. The big threat is solved for my sites for now, but the vast majority of retail sites are unprotected, and many of them aren’t in a position to solve the issue the way we did.
Over the years I’d say we’ve been attacked hundreds of times but it wasn’t until 2014 that we became aware of it, and there were a lot of random events that helped that happen. There is “security by obscurity” for most websites. You have to be a worthy enough target to get this kind of attention.
Detection is paramount. You can’t mitigate problems if you are unaware of them. For false parameters specifically there are several options… you can use canonical tags on every page, which I highly recommend. You can also use URL rewriting to enforce very strict URL formatting. But if you aren’t looking at the logs and if you aren’t looking at your search result URLs closely then you wont even know about the issue.
Detailed revenue attribution is the big one. Seeing that the losses only come from Google is an important signal. For me, SEO revenue comes from dozens of sources. Search Engines, like Google, Bing, Excite, AOL, Yahoo, etc… Syndicated Search like laptop and ISP start pages and meta search engines, Safe Search AVG, McAfee, etc… and finally my SEO experiments.
Having the revenue attribution lets me know the revenue loss only occurred on Google this time so it can’t be consumer behavior like Spring Break because the drop would have been across the board if consumers just went on holiday.
Also keep an eye on your errors, search results, and logs. Also keep an eye on your network administrator’s “Frustration Meter”.
Here are a few specific things to check when looking for negative SEO attacks:In
Google Search Console:
- Check GSC Messages for penalties.
- Check GSC Messages for outages.
- Check GSC Messages for crawl errors.
- Check Server Errors Tab for Desktop and Mobile
- Check Not Found Errors for Desktop and Mobile
- If errors look suspicious then Download and archive them.
- If errors look minor and are no longer reproducible then mark them as fixed so you only see new errors next time.
- Check the Index Status page and make sure your total number of pages looks correct.
- Check your content keywords and make sure nothing looks spammy or out of place there.
- Check Who Links to your site the most under search traffic
- Make sure your link count hasn’t abnormally grown since last check. Update your link count spreadsheet
- Check your Manual Actions
- In search analytics check countries and make sure your not suddenly popular in Russia
- In search analytics check CTR and Position and make sure the chart looks ok… no drastic events
- In Search Appearance investigate your duplicate title and meta description pages. Check the URLs to make sure they aren’t bogus
- Check Security Issues
In your web server logs:
- Check Server Logs: SQL injection
- Check Server Logs: Vulnerability Testing
- Check Server Logs: 500-503 Errors
- Check Server Logs: Outrageous requests per second
- Check Server Logs: Bad Bots
- Check Server Logs: Large volume 404 errors from 1 referring domain
In the Google Search Results:
- Check For Bizarre URLs
- Check For Domains matching your content
- Check For Unusual Sub-domains
- Check For Odd URL parameters and URL anchors
- Check For Negative mentions of domain or products
On your website and servers:
- Check Ratings and Reviews
- Check For Comment or Post Spam
- Check Content Indexes For Over-abundance of old or bad topics
- Check for profile spam
- Actively run anti-virus on server
- Routinely back up server
- Periodically run vulnerability testing on your site to close security vulnerabilities
- Patch your server regularly
- Update your web platform and plugins regularly
- Double check your WordPress Security plugin for any loose ends if applicable.
- Periodically change your admin passwords and account names
- Use strong passwords
- Don’t share accounts or email credentials
- Use a version control system and check the update status of deployed code for
- changes regularly.
- Check your domain name expiration date
There is a lot to consider in this episode. Please download the FREEBIE which is my own personal 40 point negative SEO checklist. If you are concerned about negative SEO then this checklist will help you conduct periodic audits for suspicious activity. This free download will save you hours on checking your websites for attacks.
Please subscribe and come back for our next episode where we will be “Setting SEO Guidelines For A Single Page” and I will continue the lesson by teaching my most powerful methods and secrets to content tuning a single page that targets a single keyword.
Thanks again, see you next time and always remember the first rule of SEO Fight Club: Subscribe to SEO Fight Club!