Rand Fishkin from SEOMOZ talking about negative SEO tactics and how these practices function. Negative SEO is definitely not something we condone but education around these techniques can be a helpful, precautionary method that could prevent you from being the subject of malicious intent.
Archive for month: April, 2012
How to Diagnose a Google Ranking Ban, Penalty, or Filter
If you undertake black or gray hat techniques, you run a fair chance of having your site penalized in the search results. But even if you are not engaged in these techniques yourself, your site may be punished for associating with black hat purveyors. Hosting on a shared server or sharing domain registration information with bad neighborhoods can lead to to ranking problems, if not punishment. Certainly linking to a bad neighborhood can lead to discipline. If you purchase a domain, you’ll inherit any penalties or bans imposed on the prior version of the website.
There are a wide range of penalties and ranking filters that search engines impose and a still-wider range of effects that those penalties produce. In diagnosing and correcting ranking problems, more than half the battle is figuring which penalty, if any, is imposed and for what violations. Ranking problems are easy to fix but arduous to diagnose with precision. Sudden drops in rankings might lead you to suspect that you’ve received a penalty, but it might not be a penalty at all.
In the following section we’ll look at some specific penalties, filters, conditions, and false conditions, and how to diagnose ranking problems.
The worst punishment that Google serves upon webmasters in a total ban. This means the removal of all pages on a given domain from Google’s index. A ban is not always a punishment: Google “may temporarily or permanently remove sites from its index and search results if it believes it is obligated to do so by law.” Google warns that punishment bans can be meted out for “certain actions such as cloaking, writing text in such a way that it can be seen by search engines but not by users, or setting up pages/links with the sole purpose of fooling search engines may result in removal from our index.”
One of the most newsworthy instances of a total ban was when Google, in 2006, issued a total ban to the German website of carmaker BMW (http://www.bmw.de). The offense? Cloaked doorway pages stuffed with keywords that were shown only to search engines, and not to human visitors. The incident became international news, ignited at least partially by the SEO blogging community. BMW immediately removed the offending pages and within a few weeks, Google rescinded the ban.
How to Diagnose a Total or Partial Ban
To diagnose a full or partial ban penalty, run the following tests and exercises:
- Check Google’s index. In the Google search field, enter the following specialized search query: “site:yourdomain.com.” Google then returns a list of all of your site’s pages that appear in Google’s index. If your site was formerly indexed and now the pages are removed, there is at least a possibility that your site has been banned from Google.
- Check if Google has blacklisted your site as unsafe for browsing (type http://www.google.com/safebrowsing/diagnostic?site=mysite.com with your domain at the end).
Check for Nofollow/Noindex settings. It might seem obvious, but check to make sure you haven’t accidentally set your WordPress site to Noindex. To check, go to your WordPress Dashboard and click the “Privacy” option under “Settings.” If the second setting, “I would like to block search engines, but allow normal visitors” is set, then your site will promptly fall out of the index. A stray entry in a robots.txt file or in your WordPress template file can instruct search engines not to index your entire site.
Check Google Webmaster Tools. Sometimes, but not always, Google will notify you through your Webmaster Tools account that your site has been penalized. But you won’t always receive this message, so you can still be penalized even if you don’t receive it. See the image below for an example message.
PageRank Adjustment/PageRank Penalty
An alternative penalty short of an outright ban is a PageRank adjustment. The adjustment can be partial (a drop from a PR4 to a PR2) or can be full (a drop to PR0). With a PageRank adjustment, Google simply adjusts or removes the PageRank value for a site. Google often imposes this punishment upon low-value general directories that sell links. Part of the difficulty with diagnosing and repairing a PageRank penalty is that the PageRank that Google shows to users is historical, sometimes six months pass between PageRank updates.
How to Diagnose a PageRank Penalty
To diagnose a Google PageRank penalty, run the following tests and exercises:
- Check your inbound links. Whenever your PageRank drops, the most likely reason is that you’ve lost valuable links. Check your link profile in Yahoo Site Explorer. Have you lost any premium, high-PR links you had formerly? Use the reliability of the PageRank algorithm to help diagnose: if you have a PR4 link pointing into one of your pages, and that PR4 link has only one outbound link, that one link alone will be strong enough to make the destination page a PR1 or a PR2. If despite such a link your page remains a PR0, that raises the likelihood of a PageRank penalty.
Check all pages. Be sure to check every page on your site, you might just have your PageRank shifting around within your site. It is true, however, that generally your home page will have the highest PageRank value of any page of your site. So, if you’ve got a PR0 on all pages including the homepage, a PageRank penalty is suspect.
Check canonicalization. Recall the “www” and “non-www” distinction and that search engines see these as separate domains in some cases. WordPress handles this automatically, but some online tools don’t check this for you so you have to be sure your are checking both the www and non-www versions of your domain.
Compare PageRank. Compare Google’s reported PageRank score for your pages with SEOmoz’ mozRank. Typically, these two scores will correlate loosely (within about 10%). If the Google score is much lower than the SEOmoz mozRank score, it’s likely that Google is trimming some PageRank. You can see the SEOmoz Page Rank score with the free SEO Site Tools plugin or by visiting https://www.opensiteexplorer.org/.
- Check internal links. In Google Webmaster Tools, Google reveals its profile of internal links on your site. See the figures below for examples of an unhealthy internal link profile, and a healthy link profile. If your site has 100 indexed pages, but Webmaster Tools references only a handful of links, it means that Google is not properly processing your internal links. We need to be careful here because a range of conditions can cause this. It can potentially arise from a PageRank penalty but also from poor internal navigation structure.
The -950 Ranking Penalty
Google sometimes employs a -950 ranking penalty to individual pages (but not to entire sites) for particular search queries. The -950 penalty means that for a particular search, your page would have 950 positions added to it. So, a term for which you ranked on page one of Google’s search results in position three, you’d now rank on page ninety-five of the search results at position 953. Sound harsh? It is, and Google has made faint references to it as a penalty for over-optimization. Some SEO professionals contend that they have seen the penalty imposed for shady link building practices.
How to Diagnose a -950 Ranking Penalty
Diagnosing a -950 ranking penalty is easy: try search terms for which you formerly ranked (hopefully you noted their exact former position) and follow the search results out to page 95 or 96. Remember that you can always set Google to display 100 results instead of ten by using the advanced search option at Google.com, which is convenient for checking ranking position in the 100s and above.
The -30/-40 Ranking Penalty
Google often serves up another variety of penalty: it’s the -30 or -40 position penalty. This is an often-imposed penalty, and is applied by Google to entire sites, not just particular pages and not just for particular search queries. This penalty is common enough to trip up legitimate webmasters for very minor oversights or offenses. Most signs point to the -30 penalty being applied algorithmically and is “forgivable,” so changing the condition that led to the penalty automatically reverses the penalty. This penalty has historically been imposed upon sites for serving up poor quality content. For example, the penalty has been imposed upon sites that display thin content. Thin content is content that is partially generic, as with an affiliate site repeating common descriptions of products it sells. Low-value directories have also been served this penalty.
How to Diagnose a -30/-40 Penalty
If you suspect that your site has been been hit with a -30/-40 penalty, there is one sure-fire test to determine if you tripped the penalty. Perform a Google search for your domain name, with out the “www” and without the “.com” or “.net” part of the domain. This search, in normal circumstances, should return your site at or near the first position (depending a bit on the competition of that term). If this test yields your site showing up in a position dropped to the 40s or 50s, it is almost certainly is a -30/-40 penalty.
False Positives That Aren’t Penalties
Don’t assume you’ve been penalized by Google just because your rankings drop or because your rankings remain poor for a new site. Ranking positions can jump around naturally, especially just before algorithm updates, when Google updates its search engine rules. You may also have lost one or more valuable inbound links, that can lead to a drop in rankings. You may also be alternating between Google’s personalized search modes. Personalized search is a Google feature that returns results based on your personal browsing habits. So, if you’ve visited your own website in the past few days, Google will return your website near the top of the results, figuring that it’s one of your personal favorites. Personal search is a convenience tool, but it doesn’t return true rankings. To see actual ranking results you need to make sure personalized search is off. To do this, look on any Google search results page in the upper left hand corner for “Personalize Search On.” Click on the link just under it that reads, “Turn it off.”
Google penalties are almost never imposed for no reason at all. Yes, Google imposes penalties on light offenders while more egregious violations go unpunished. While that might not seem fair, it doesn’t change the fact that if you have perfectly complied with Google’s Webmaster Guidelines, you are extremely unlikely to be penalized. If you’ve been penalized, there’s a reason.
During the last few weeks, Google sent many webmaster notification messages about unnatural links. If you have received such a message, here’s what you have to do to make sure that your website doesn’t get penalized.
What is this Google unnatural links message?
Google has sent the following message to many webmasters:
“Dear site owner or webmaster of example.com.
We’ve detected that some of your site’s pages may be using techniques that are outside Google’s Webmaster Guidelines.
Specifically, look for possibly artificial or unnatural links pointing to your site that could be intended to manipulate PageRank. Examples of unnatural linking could include buying links to pass PageRank or participating in link schemes.
We encourage you to make changes to your site so that it meets our quality guidelines. Once you’ve made these changes, please submit your site for reconsideration in Google’s search results.
If you find unnatural links to your site that you are unable to control or remove, please provide the details in your reconsideration request.
If you have any questions about how to resolve this issue, please see our Webmaster Help Forum for support.
Google Search Quality Team”
What will happen when you get such a message?
Webmasters who received such a message observed that many sites were penalized 3-4 weeks after the message. The penalty is for the keywords that are included in the unnatural links.
If most of the links that point to the website are unnatural links then the whole website might be penalized.
What are unnatural links?
There are several backlink types that Google finds unnatural:
- Public backlink networks:Google doesn’t like fully automated or paid backlink networks. If you participate in a backlink network that can be joined by anyone (even for a fee) then it is very likely that Google has already penalized the network or that the network is a target for the near future.If you can find the backlink network, Google’s engineers can find it, too.
- Private backlink networks: some SEO agencies have private backlink networks. Google probably has the technology to detect these networks without creating accounts.
- Paid sidebar links: if your website has too many backlinks from the sidebars of other websites, Google might find them unnatural.
- Over-optimized anchor text: if the links to your website all use exactly the same anchor text, it is likely that Google might reconsider the rankings of the linked pages.
- Fake forum and social media links: some tools create fake forum and social media site accounts to get backlinks to your website. Chances are that Google can detect that type of link.
What should you do?
If you used one of the methods above to get backlinks to your website, try to get rid of these links as quickly as possible.
The formula to high rankings on Google is very easy:
Good content + good backlinks + no spam = high rankings
Optimize the content of your web pages to make sure that Google and other search engines know what your website is about. Then get good backlinks to show search engines that your website can be trusted.
Google has become more aggressive regarding spammy backlinks. In the past, a website might get high rankings for several months until Google detected the site. Now it seems that the it takes a maximum of 3 months until Google detects the spammers.
Do not fall for SEO solutions that promise quick and easy backlinks. It doesn’t make sense to get high rankings for 2-3 months just to get penalized after that time. If you are serious about your business, you have to use strategies that deliver high rankings that will stay, even if it takes longer to get these rankings in the short term.
Here’s the list for March:
- Autocomplete with math symbols.
[launch codename “Blackboard”, project codename “Suggest”]
When Google process queries to return predictions in autocomplete, Google generally normalize them to match more relevant predictions in Google’s database. This change incorporates several characters that were previously normalized: “+”, “-”, “*”, “/”, “^”, “(“, “)”, and “=”. This should make it easier to search for popular equations, for example [e = mc2] or [y = mx+b].
- Improvements to handling of symbols for indexing.
[launch codename “Deep Maroon”]
Google generally ignore punctuation symbols in queries. Based on analysis of Google’s query stream, Google now started to index the following heavily used symbols: “%”, “$”, “\”, “.”, [email protected], “#”, and “+”. Google continue to index more symbols as usage warrants.
- Better scoring of news groupings.
[launch codename “avenger_2”]
News results on Google are organized into groups that are about the same story. Google have scoring systems to determine the ordering of these groups for a given query. This subtle change slightly improves Google’s scoring system, leading to better ranking of news clusters.
- Sitelinks data refresh.
[launch codename “Saralee-76”]
Sitelinks (the links that appear beneath some search results and link deeper into the respective site) are generated in part by an offline process that analyzes site structure and other data to determine the most relevant links to show users. Google recently updated the data through Google’s offline process. These updates happen frequently (on the order of weeks).
- Improvements to autocomplete backends, coverage.
[launch codename “sovereign”, project codename “Suggest”]
Google consolidated systems and reduced the number of backend calls required to prepare autocomplete predictions for your query. The result is more efficient CPU usage and more comprehensive predictions.
- Better handling of password changes.
Google general approach is that when you change passwords, you’ll be signed out from your account on all machines. This change ensures that changing your password more consistently signs your account out of Search, everywhere.
- Better indexing of profile pages.
[launch codename “Prof-2”]
This change improves the comprehensiveness of public profile pages in Google’s index from more than two-hundred social sites.
- UI refresh for News Universal.
[launch codename “Cosmos Newsy”, project codename “Cosmos”]
Google refreshed the design of News Universal results by providing more results from the top cluster, unifying the UI treatment of clusters of different sizes, adding a larger font for the top article, adding larger images (from licensed sources), and adding author information.
- Improvements to results for navigational queries.
[launch codename “IceMan5”]
A “navigational query” is a search where it looks like the user is looking to navigate to a particular website, such as [New York Times] or [wikipedia.org]. While these searches may seem straightforward, there are still challenges to serving the best results. For example, what if the user doesn’t actually know the right URL? What if the URL they’re searching for seems to be a parked domain (with no content)? This change improves results for this kind of search.
- High-quality sites algorithm data update and freshness improvements.
[launch codename “mm”, project codename “Panda”]
Like many of the changes Google make, aspects of Google’s high-quality sites algorithm depend on processing that’s done offline and pushed on a periodic cycle. In the past month, Google pushed updated data for “Panda,” as Google mentioned in a recent tweet. Google also made improvements to keep Google’s database fresher overall.
- Live results for UEFA Champions League and KHL.
Google added live-updating snippets in Google’s search results for the KHL (Russian Hockey League) and UEFA Champions League, including scores and schedules. Now you can find live results from a variety of sports leagues, including the NFL, NBA, NHL and others.
- Tennis search feature.
[launch codename “DoubleFault”]
- More relevant image search results.
[launch codename “Lice”]
This change tunes signals Google use related to landing page quality for images. This makes it more likely that you’ll find highly relevant images, even if those images are on pages that are lower quality.
- Fresher image predictions in all languages.
[launch codename “imagine2”, project codename “Suggest”]
Google recently rolled out a change to surface more relevant image search predictions in autocomplete in English. This improvement extends the update to all languages.
- SafeSearch algorithm tuning.
[launch codenames “Fiorentini”, “SuperDyn”; project codename “SafeSearch”]
This month Google rolled out a couple of changes to Google’s SafeSearch algorithm. Google updated Google’s classifier to make it smarter and more precise, and Google found new ways to make adult content less likely to appear when a user isn’t looking for it
- Tweaks to handling of anchor text.
[launch codename “PC”]
This month Google turned off a classifier related to anchor text (the visible text appearing in links). Google’s experimental data suggested that other methods of anchor processing had greater success, so turning off this component made Google’s scoring cleaner and more robust.
- Simplification to Images Universal codebase.
[launch codename “Galactic Center”]
Google made some improvements to simplify Google’s codebase for Images Universal and to better utilize improvements in Google’s general web ranking to also provide better image results.
- Better application ranking and UI on mobile.
When you search for apps on your phone, you’ll now see richer results with app icons, star ratings, prices, and download buttons arranged to fit well on smaller screens. You’ll also see more relevant ranking of mobile applications based on your device platform, for example Android or iOS.
- Improvements to freshness in Video Universal.
[launch codename “graphite”, project codename “Freshness”]
Google improved the freshness of video results to better detect stale videos and return fresh content.
- Fewer undesired synonyms.
[project codename “Synonyms”]
When you search on Google, Google often identify other search terms that might have the same meaning as what you entered in the box (synonyms) and surface results for those terms as well when it might be helpful. This month Google tweaked a classifier to prevent unhelpful synonyms from being introduced as content in the results set.
- Better handling of queries with both navigational and local intent.
[launch codename “ShieldsUp”]
Some queries have both local intent and are very navigational (directed towards a particular website). This change improves the balance of results Google show, and helps ensure you’ll find highly relevant navigational results or local results towards the top of the page as appropriate for your query.
- Improvements to freshness.
[launch codename “Abacus”, project codename “Freshness”]
Google launched an improvement to freshness late last year that was very helpful, but it cost significant machine resources. At the time Google decided to roll out the change only for news-related traffic. This month Google rolled it out for all queries.
- Improvements to processing for detection of site quality.
[launch codename “Curlup”]
Google made some improvements to a longstanding system Google have to detect site quality. This improvement allows us to get greater confidence in Google’s classifications.
- Better interpretation and use of anchor text.
Google improved systems Google use to interpret and use anchor text, and determine how relevant a given anchor might be for a given query and website.
- Better local results and sources in Google News.
[launch codename “barefoot”, project codename “news search”]
Google deprecating a signal Google had to help people find content from their local country, and Google building similar logic into other signals Google use. The result is more locally relevant Google News results and higher quality sources.
- Deprecating signal related to ranking in a news cluster.
[launch codename “decaffeination”, project codename “news search”]
Google deprecating a signal that’s no longer improving relevance in Google News. The signal was originally developed to help people find higher quality articles on Google News. (Note: Despite the launch codename, this project has nothing to do with Caffeine, Google’s update to indexing in 2010).
- Fewer “sibling” synonyms.
[launch codename “Gemini”, project codename “Synonyms”]
One of the main signals Google look at to identify synonyms is context. For example, if the word “cat” often appears next to the term “pet” and “furry,” and so does the word “kitten”, Google’s algorithms may guess that “cat” and “kitten” have similar meanings. The problem is that sometimes this method will introduce “synonyms” that actually are different entities in the same category. To continue the example, dogs are also “furry pets” — so sometimes “dog” may be incorrectly introduced as a synonym for “cat”. Google been working for some time to appropriately ferret out these “sibling” synonyms, and Google’s latest system is more maintainable, updatable, debuggable, and extensible to other systems.
- Better synonym accuracy and performance.
[project codename “Synonyms”]
Google made further improvements to Google’s synonyms system by eliminating duplicate logic. Google also found ways to more accurately identify appropriate synonyms in cases where there are multiple synonym candidates with different contexts.
- Retrieval system tuning.
[launch codename “emonga”, project codename “Optionalization”]
Google improved systems that identify terms in a query which are not necessarily required to retrieve relevant documents. This will make results more faithful to the original query.
- Less aggressive synonyms.
[launch codename “zilong”, project codename “Synonyms”]
Google heard feedback from users that sometimes Google’s algorithms are too aggressive at incorporating search results for other terms. The underlying cause is often Google’s synonym system, which will include results for other terms in many cases. This change makes Google’s synonym system less aggressive in the way it incorporates results for other query terms, putting greater weight on the original user query.
- Update to systems relying on geographic data.
[launch codename “Maestro, Maitre”]
Google have a number of signals that rely on geographic data (similar to the data Google surface in Google Earth and Maps). This change updates some of the geographic data Google using.
- Improvements to name detection.
[launch codename “edge”, project codename “NameDetector”]
Google improved a system for detecting names, particularly for celebrity names.
- Updates to personalization signals.
[project codename “PSearch”]
This change updates signals used to personalize search results.
- Improvements to Image Search relevance.
[launch codename “sib”]
Google updated signals to better promote reasonably sized images on high-quality landing pages.
- Remove deprecated signal from site relevance signals.
[launch codename “Freedom”]
Google removed a deprecated product-focused signal from a site-understanding algorithm.
- More precise detection of old pages.
[launch codename “oldn23″, project codename “Freshness”]
This change improves detection of stale pages in Google’s index by relying on more relevant signals. As a result, fewer stale pages are shown to users.
- Tweaks to language detection in autocomplete.
[launch codename “Dejavu”, project codename “Suggest”]
In general, autocomplete relies on the display language to determine what language predictions to show. For most languages, Google also try to detect the user query language by analyzing the script, and this change extends that behavior to Chinese (Simplified and Traditional), Japanese and Korean. The net effect is that when users forget to turn off their IMEs, they’ll still get English predictions if they start typing English terms.
- Improvements in date detection for blog/forum pages.
[launch codename “fibyen”, project codename “Dates”]
This change improves the algorithm that determines dates for blog and forum pages.
- More predictions in autocomplete by live rewriting of query prefixes.
[launch codename “Lombart”, project codename “Suggest”]
In this change Google rewriting partial queries on the fly to retrieve more potential matching predictions for the user query. Google use synonyms and other features to get the best overall match. Rewritten prefixes can include term re-orderings, term additions, term removals and more.
- Expanded sitelinks on mobile.
Google launched Google’s expanded sitelinks feature for mobile browsers, providing better organization and presentation of sitelinks in search results.
- More accurate short answers.
[project codename “Porky Pig”]
- Migration of video advanced search backends.
Google migrated some backends used in video advanced search to Google’s main search infrastructure.
- +1 button in search for more countries and domains.
This month Google internationalized the +1 button on the search results page to additional languages and domains. The +1 button in search makes it easy to share recommendations with the world right from your search results. As Google said in Google’s initial blog post, the beauty of +1’s is their relevance—you get the right recommendations (because they come from people who matter to you), at the right time (when you are actually looking for information about that topic) and in the right format (your search results).
- Local result UI refresh on tablet.
Google updated the user interface of local results on tablets to make them more compact and easier to scan.