Hello Link Diagnosis Users,

As always, we appreciate your loyalty, and wanted you to know that we are busy working on an updated free version and paid version of Link Diagnosis. We are not going to take any functionality away from the free tool version.

We need your input though. What else could we be doing? How can we make the results better?

Please provide your insights. Keep in mind that speed is always an issue based on the sheer volume of scrapes we perform. We will soon allow you to modify the results sets so that data can be more quickly reached.

Best,
LinkDiagnosis.com Team

Link Diagnosis has been redesigned in PHP and moved to a new server. The tool is already performing about 20% faster. We are working hard to offer some additional services, which will allow you to control the speed of the tool. Every computer is different and different browser versions react differently so right now we have to keep the speed slower so it works for everyone.

We will also be incorporating a service which allow you to create and store projects online, setup multiple URL’s per project, combine multiple URL results into one big report with sorting, and even private-labeled report exports. Stay tuned folks - more good stuff to come!

I have just released a new feature to Link Diagnosis Firefox Extension that will allow easy diagnostics of the indexed pages on your website.

Couple of weeks ago I was facing a tedious task of finding out which pages out of 100k on the site are indexed and which are not. I knew that some of them could have been marked as duplicate content or that Google simply didn’t indexed them because of the size of the website.

First I installed Google Webmaster Tools hoping that Google will tell me that. Unfortunately, the Indexed Pages tab just points me to use site: command.

I don’t trust site: command. Especially, the count of number of pages is very inaccurate. I know I have 100k pages and Google tells me I have 150k pages indexed.

Also, there is no easy way to see more than 1000 pages (you can play with inurl: commands but it takes ages and you can get banned).

Because of these problems I decided to code a tool which would automate it - Site Diagnosis.

The internal algorithm of the tool works as follows:

1. Go through every URL in XML Sitemap file and do a simple check inurl:http://www.samplesite.com/dir/url1
2. For every url that does not appear on inurl: command there is still a chance that page is indexed but does not appear with inurl
3. For every url in XML Sitemaps I get a title and perform this check site:http://www.samplesite.com sample title
4. If the page does not rank in top 10 for its title within the site then probably something is wrong.

This check is suprisingly accurate and most of the pages that don’t survive this check have some problems like duplicate content, missing titles, missing content or not enough content. These troubled pages usualy don’t appear in the search results if you search for any text on the page - not even when you enclose sentences in quotes.

Obviously, the goal of search engine optimization is to fix these pages so Site Diagnosis will hopefully be essential in identifying them.

For anyone who doesn’t know there is a date filter on Google which brings you only results that were created within the last xx period. See screenshot:

When you use that filter and see the search result pages then you will see that under every listing there is a date which shows you when Google saw that page for the first time. We can then use this information to find how old the page is if we craft the query to include the url of the page we are interested in.

inurl:http://blog.linkdiagnosis.com/?p=16

For that query it returns the date when my blog post about Amazon was created - 15 February 2008 - which is spot on.

Now to make it even easier to get the age of the page I have created a new feature to Link Diagnosis Firefox Extension.

Install the extension and you will be able to find out the age of the page with one click.

See the screenshot below:

Enjoy!

P.S Don’t do many queries at once otherwise Google will ban you (for me after about 70 inurl commands with 1 sec delay). If anybody has any tips how to prevent that from happening, then please let me know and I will code the age of the page to the main report so you can see both metrics - pagerank and age together.

Over the last couple of years the trust of the website and page has been primary factor when evaluating the value of the link. Most of the people for simplicity use pagerank for evaluating how much benefit will give the link to their site. Bill from SEO by the SEA has just written an article how search engines are evaluating the links and content importance based on on-page factors.

The most important bits from Yahoo patent (and also from Microsoft research paper) are that search engines can and do create a visual model of the page to find out which is the “Most significant element”. Some of the factors that Yahoo confirms in the paper:

• formatting of the text - bold , h1 etc. - GOOD
• tables with data and other grid-like structures - GOOD
• distance from the top and center. The most important ones are near the horizontal center of the page and also above the fold.
• content / links that are at the absolute header/footer of the page - BAD

Microsoft research paper talks more about the fact that Pagerank model is extended where the atomic entity is not the page but page-block. One page can have multiple blocks with each having different semantics and importance.

This information can have serious impact on how the link juice is flowing from the pages and especially the part from Microsoft paper affects that. Most of the bought links are not placed in the “Most Significant Element” of the page - which is the main content bit. The bought links are placed in side bars, footers etc. These research papers say that these links are much less important than the links which are within the content block. This then just rises the value of bought content links of PayPerPost style where they are within the context of the blog post.

Another implication of these research papers confirms that search engines are able to reconstruct the HTML Document Object Model and know which content is important regardless of the position in the source code of the page. This contradicts with many SEO Experts which have been claiming in the past that you should put the most important content first just after the BODY element.

As always, your comments are welcome.

I have just released 1.0.5 version which should fix the most annonying bug - report goes to 100% and then freezes. If you get this sometimes then please update to the new version (FF should automatically pickup the new version anyway). If you have any other problems, please fill the bug report with as much information as possible so we can make the tool more stable together.

More than 500 SEOs use this tool every day and  I am glad that numbers are raising. Please don’t be afraid to share this tool with others  - more people use it - more I will want to include better features and spend time maintaining it.

Cheers!

Its not Link Diagnosis related but one of my sites went down because of Amazon S3. Anyone else sees that ?

I can see that Colin Schlueter on Jaiku just ask if its down so it may be a start of major outage?

UPDATE: This thread on Amazon forums discusses over 35 minute down time.

Just wanted to share the cheer when I see SEOs all around the world enjoying the extension.

Well, at least I hope they are enjoying it as I don’t understand a word in spanish, german or russian .

There was a serious bug that prevented from running a diagnosis on a subdomain. I have just released a 1.0.4 version which fixes that. Thanks to all who have reported that bug.

Finally, I got around to do some bug fixing. All the new users that I got after Patrick gave me traffic bump from Sphinn have made that lots of bugs started to appear and I got much more bug reports.

One of the biggest problems was that some people experienced sometimes that website didn’t detect that extension was installed and was only producing the basic data. Hopefully, that is fixed now. Please let me know if you get this error.

Also, I have tried to add support for Firefox 3 BETA. However, folks at Mozilla don’t make it very easy and there are some serious problems with that BETA. I have decided to wait with the Firefox 3 support until they sort out a more compatible release with the older code. Sorry