Lee Odden

4 Tips For Clean Link Checking

Lee Odden     Online Marketing, SEO, SEO Tips

clean linksIt’s time for another SEO Basics post and this time getting more out of link building efforts is the topic. 

Link building continues to be an important part of marketing and optimizing web sites and web marketers can often get distracted by quantity goals rather than quality.  Link building efforts for search engine optimization purposes rely on clean links that can be crawled by search engine bots.  But what’s a “clean crawlable link”?  It’s one that is not blocked with Robots NoIndex meta tag, JavaScript redirect, blocked with robots.txt or a NoFollow tag.

There are hundreds of ways to attract and acquire links. If link requests, article submissions or other high labor, low impact tactics are used, then it’s important to make sure the links acquired are good for both users and search engines.

Here are a few things to check for:

1. To check the robots meta tag, look at the page source code. If there is no robots tag, that’s fine.

If there is a robots meta tag and it looks like this,
meta name = “robots” content = “index, follow”
that’s good – although it’s not really necessary on the part of the webmaster.

If the robots meta tag looks like this:

“robots” content = “noindex, nofollow” 

“robots” content = “index, nofollow” 

that’s not good as far as links for SEO benefit.

2. To see if there is a JavaScript redirect of links from a desired page, put your cursor over the link and look at the url that appears in the status bar at the bottom of the browser. If it shows the correct link url, then in most cases, it’s ok.  

However, this can be faked within the JavaScript so if you can copy the displayed URL you may want to run the link through a tool like Rex Swain’s HTTP Viewer which will show you if the link redirects, what type and where.

3. To see if robots.txt is blocking search engine spiders from crawling links on a page, then add the text “/robots.txt” to the end of the URL in question.

For example:

Here you will see:

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /editor/
Disallow: /help/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /mambots/
Disallow: /media/
Disallow: /modules/
Disallow: /templates/
Disallow: /installation/

What the “disallow” instruction does above is to tell search engine spiders not to crawl the designated directories. In the case of the article sharing site above, if any articles are located in one of these directories, then the links within those articles to client sites are no good for SEO benefit. However, readers can still click on the links and arrive at the indicated destination.

4. To see if there is a no follow tag, right click on the link URL using a browser like MSIE or Firefox and click on “Properties”. See if there is an attribute called:

rel = nofollow.
If that’s there, it’s no good for SEO.

If it’s not there at all or has another value besides “nofollow” then it’s probably ok.

You only have to do this once in most cases on any page of a particular site. The reason sites will do any of these 4 things is to hoard or “sculpt” their site’s PageRank. In the case of blogs, it’s to discourage comment spam.

Now here’s the rub: An exeperienced SEO professional will never do the above 4 steps manually. In almost all cases, they will either develop tools to automatically check for “clean links” or they won’t bother at all. It’s an important question to ask when working with a SEO consultant. Just because a client site’s inbound link count goes from 500 to 5,000 links doesn’t mean all of those links are created equal. In other words, quantity is nowhere near the complete measurement of a link building effort to improve search engine rankings.

Link building that is based on forms or submissions is very difficult to scale and the links are often nofollowed after it is discovered SEOs are populating the site with their content. Link building as a result of creating and promoting content worth linking to is high value, high impact and very scalable.  However, if it is important to check link sources to determine their value for SEO benefit you can use the 4 steps above, create your own script to check them automatically or work with a SEO company that has their own.

PoorSo SoOKGoodAwesome (4 votes, average: 4.75 out of 5)

Lee Odden About Lee Odden

@LeeOdden is the CEO of TopRank Marketing and editor of Online Marketing Blog. Cited for his expertise by The Economist, Forbes and the Wall Street Journal, he's the author of the book Optimize and presents internationally on B2B marketing topics including content, search, social media and influencer marketing. When not at conferences, consulting, or working with his talented team, he's likely running, traveling or cooking up something new.


  1. Avatar Frank Reed says

    If the SEO professional doesn’t check at all should he / she then be called a professional? Like you pointed out quality trumps quantity so just trying to impress a client with numbers is what has given the industry a bad rap at times. Like the traffic v. conversion argument.

    Thanks for the post, Lee.

  2. Thanks Frank. Most companies don’t know any better. They “know enough to be dangerous” about SEO and are aware of the value of links but may not know that all links are not created equally. 🙂

  3. Avatar Wollongong says

    Well done mate, this is a very confusing topic for many people / businesses. Most people don’t know this robot.txt even exits!

  4. You know what turned out to be a great backlink checking tool? Trackback comments in your WP settings.

    The problem with all backlink checking services is that they have to outsource the info from Google and Yahoo. Google deliberately is not showing the true number of backlinks. And Yahoo is better, but as many fight for Google – then measuring Google with Yahoo makes little sense.

    The members of my service are getting one way links from inside blog posts, and for this type of backlinks trackbacks are a perfect tool to backlink checking: you get to your comments section and see all pages that gave you a backlink.

    P.S. But, of course, one shouldn’t approve these trackbacks as comments on the blog, otherwise you give a link back and this is not wise in terms of SEO.

  5. I’m uncertain why something as relatively simple as links are a “confusing topic for many people/business” as per one of the comments on this blog. If you spent 20 minutes researching links on Google I’d think you would quickly find enough information to have a reasonable understanding. Its hardly rocket science or quantum physics now is it~?

  6. Hey Darren, nothing is rocket science except rocket science.

  7. Lee, I agree and I applaud your blog for informing people, its just sometimes people seem to tie themselves up in unneccessary knots over something quite simple… maybe I’ve been SEO’ing for too long (7 years)

  8. Lee:

    I believe it was Google who introduced this technique of NoFollow/DoFollow tags to adjust for the nature of their ranking algorithm.

    Is there anything we can do about the 800 pound Google gorilla??

    This NoFollow tag as it applies to blogs is the worst thing to happen to blogging ever.

    It has discouraged active commenting on blogs that is often insightful and helpful.

    So we are stuck with the never ending hunt for DoFollow
    blogs and forums.

    The value of provoking targeted traffic through blog comments has been lost with all the emphasis on creating
    backlinks with maximum SEO benefits.

    Why can’t Google fix their ranking algorithm so that NoFollow/DoFollow is irrelevant?

    Google has inflicted punishment on those who want to buy text links for targeted traffic by insisting these links be NoFollow links.

    Google is pretending that they can distinguish links bought for link popularity versus links bought for targted traffic.

    It’s a sorry mess ww’re in thanks to the big G.

    Please, forgive the rant.

    Best of success,


  9. Great article but I had one question. You sai to check if the link had:

    rel = nofollow.
    If that

  10. thanks frank for the tips its look like simple can be folow

  11. Lee, This is good informative article, it covers everything about how to check your links are worth or not? I don’t know javascript much, but you have given some good ways to check is it dofollow or not. I need to install the tool and check my all inbound links.

  12. Thanks for the great tip. You would be surprised how many paid dead links Google has been bring up lately, depending on the key word or key phrase used.

  13. There’s a plugin that works with WordPress 2.7 called “Nofollow Case by Case”. It can be accessed within the “install plugins” page in WP 2.7. Just be sure to have your spam blocker enabled…

  14. I saw some sites with:

    will a space after the “index,” make any difference?

  15. about 3. it’s the standard joomla/mambo robots.txt, in order to avoid spiders (legit or not!) go trought directory that does not contain content items, which are always reachable by a url string that does not contains any of those path/ (urls are reproduced by site.com/index.php?&params.. or by something more sef friendly)

    also, in robots.txt you can include SITEMAP directive, like ASK.COM requires


  16. about 4.
    “nofollow” tag is a must in some scenarios, in order to avoid content duplication.
    if your web content management system allows frontend PDF or PRINTABLE page creation through a link, inserting nofollow will stop spiders linking to those pages, which actually are only duplicated content formatted in a different way (= ba for SEO) .

  17. Hi,

    Very useful information for budding SEO professionals. Can you please suggest any free open source tool that can check the sites for these 3 things?

    Kind regards,