SEO tooling: Backlink analysis, disavow, link strategy review

Nowadays it is essential for any business to be online and attempt to gain visibility through social media and search engines. Although over the last few years SEO (search engine optimisation) has become less important as Google, Yahoo and others have become smarter about how to search and index content, one can not overlook how some SEO and SEM “experts” apply grey- and black-hat strategies to gamble search engine rankings.

About a decade ago it was perfectly acceptable to have your website listed in web-directories and place links through paid content or link exchange programmes. All of this has come to an end and search engines are now awarding websites for relevance and good content. You will find that most SEO and SEM “experts” engage in some dubious link building schemes (such as content farms, link farms, paid link exchange) but equally worrying is that your competitors can craft “poison links” (links from websites with questionable content) to attempt to damage your reputation.

Once in a while it is a good thing to review your inbound links to judge if you need to work on your web-reputation and conduct a back-link cleanup.

Google Webmaster Toolkit

GWMT is your first port of call. If you have not configured GWMT for your website, I urge you to do this right away via this link. GWMT will provide you with a “technical” view of the health and performance of your website and provides insight how Google sees your site. It is obvious that you should have also signed up for Google Analytics to manage your website analytics.

In GWMT go to “Search Traffic > Links to your site” and review which domains link to you and if you have inbound links of questionable origin.

Disavowing links

Google provides a “Disavow Links” feature which allows you to tell Google about low-quality links to your site and by submitting a disavow file hinting to Google to remove those links. It is important to understand that the ideal solution is to contact the webmaster of the linking domain and ask for link removal or annotating the link with a rel=”nofollow”. In most cases webmasters refuse the removal or ask for payment in exchange of the removal:

I find businesses like SubmitYourArticle.com despicable (they had their place a decade ago) as their article syndication network now extorts money from webmasters who fell victim to some shady SEO practises by hired SEO consultants or content teams. It is still fascinating though, that Google, despite fighting web-spam still indexes content farms such as Articlecity.com (check the Google index for Articlecity.com)

The disavow process is quite simple and I find SEO consultants make it sound more complex than it actually is:

1. Identify bad / low-quality links to your site. There are a number of tools out there and the most basic is GWMT. My preferred paid tool is Moz.com
2. List the bad links / root domains in your disavow file
3. Submit the disavow file via Google Webmaster Tools
4. Wait…. wait some more… and some more …. until Google decides to recrawl all the listed domains/pages and eventually disregards those low-quality links

In most cases webmasters will only look at disavow once they have been hit by a Google penalty. I consider a disavow/link-quality review part and parcel of a webmaster’s housekeeping and it is generally better to frequently review your website rather than trying to put out fires when it is too late. Also remember that your competitors (or upset customers, or a disgruntled SEO-/SEM agency) might use negative SEO against you (in essence they will build spammy backlinks against your site and hope that Google will penalise you).

I strongly suggest not to pay for any link removal as this will financial ruin you and from my experience Google already disregards any ranking factors from link networks, content farms, article directories or blog networks. I do recommend Pitchbox.com if you need to manage outreach campaigns (especially if you use tools such as LinkResearchTools as a SEO tool, since it has direct integration into Pitchbox). In some of the work I did, both Pitchbox and LinkResearchTools have been valuable tools to audit and manage backlinks and both aided in the removal of low-quality links.

One of the coolest website tools I came across some time back is UrlToDomain – if you ever had the requirement where you need to extract just the domains from a long list (like a CSV) of URLs, UrlToDomain is the tool for you. I used it in conjunction with LinkResearchTools (I tag links as “disavowed” in LRT and then export all links as a CSV) where in most cases I wanted to disavow a whole domain rather than individual URLs.

Once you are ready to disavow low quality links, I suggest that your disavow file contains comments (a line starting with a “#” is a comment and ignored by Google) about the nature of the disavow request and any actions you have taken for the manual removal. The disavow file will look similar to this:

# Google Disavow tool: https://www.google.com/webmasters/tools/disavow-links-main
#
# disavows all inbound links from a domain
# domain:example.com 
# disavows a specific source URL
# http://www.example.com/spammy-page.html

# 2015-01-06: Removed domains asking for money for link removal
domain:articlecity.com
domain:submityourarticle.com

The file contains of a disavow record per line and can be either a domain which then disavows all inbound links or it can be an URL. Once you have submitted your disavow request it can take weeks and sometimes several months for Google to process the change (and sometimes they will never do it). There is no feedback from Google about the disavow and the only indicator is to monitor the links to your site in GWMT via “Search Traffic > Links to your site”.

I did find that if your site is healthy (i.e. no crawl errors, accessible robots.txt, valid sitemap, no manual actions or penalties) Google takes about 1 month for inbound links to drop.

LinkResearchTools

I use LinkResearchTools (LRT) and Moz for SEO, keyword and backlink analysis – all are paid tools, but if you know what you are doing can really streamline your work. GWMT will provide you basic insight and other tools provide you more depth.

One of the many features of LRT is the “DTOX” report which highlights low-quality/spammy links which webmasters can action. The tooling is straightforward, where you capture your domain and then feed backlink files (exported from GWMT, Moz, Majestic, SEMRush or your own URL list). LRT will then go off and explore inbound links and grade them based on a number of algorithms. The final result is a “DTOX” score and the general sentiment is that anything above 1000 is going to get you into trouble with Google.

Generally a link-analysis and backlink cleanup exercise is (despite all the available tooling) a laborious job and requires manual vetting of every (!) inbound link. LRT provides powerful filtering capabilities which helps you in scanning through links very quickly and where a regular link analysis would take several days (if not weeks), LRT will allow you to scan and vet links within a few hours.

A few thoughts & comments about Google disavow

Google does have some basic documentation about the disavow tool and it’s content – it’s good to read it to get some more clarity. A while back I listened to a GWMT hangout with Google’s John Mueller where it was mentioned that disavowed links stay in webmaster tools. I am not sure that I fully agree with this (and perhaps there is another explanation), as I have evidence that close to 20,000 spammy links to my blog are not visible with GWMT although they are still present on the spammy website (I disavowed a number of domains) – perhaps this is coincidence, but the source links still point to content on my blog and do not carry a rel=”nofollow”.

Google processes the disavow file automatically and in a GWMT forum post it was mentioned that it is irrelevant what comments you place in the disavow file. The comments are not used by Google’s webspam team and are purely for your own understanding and keeping a history. I time-stamp my comments and often include references or ticket-numbers received from webmasters.

You are able to reavow a previously disavowed link. This is helpful if you accidentally disavowed a link. I honestly can not think of any other case where it would be relevant to reavow a previously disavowed spammy link – in most cases the whole domain linking into your site would be spammy. I did read an article, where a webmaster reavowed a link, Google crawled it and then penalised the website with a manual action – I do believe that there was more to that story, as a manual action against a whole domain would have been flagged by a number of bad practises by that webmaster.

For me it was puzzling to hear that Google made the statement that disavow data is not “mined” in order to profile spammy websites and Google does not use disavow data against the sites being disavowed. I am not convinced that is true – surely the disavow files submitted by webmasters show some hints that inbound links/domains are of poor quality – irrespective of the background why a webmaster wants to disavow links, one can assume that some time and effort was spent in creating the file. I do think that Google uses this to a certain degree. On the other hand I am a bit puzzled about the double-standard where Google continues to index obvious link farms and is very quick to penalise the linked sites and hardly ever takes action against the link farm itself (all article directories are a prime example – they should really be removed from the index altogether).