What is pretty to human eye may not be true for a search engine bot or spider. You want your website to be nice looking to your visitors but at the same time you would want to rank well in search engines. For SEO (Search Engine Optimization) purpose it’s vital that you know how a spider or bot sees your site. Visually appealing doesn’t always mean it is crawler friendly. Certain elements or objects cannot be read well by spiders. For example, if your website mainly consists of flash objects and very few text  then chances are it will not rank as high as you would like in SERP (Search Engine Results Page). That’s why Google recommends checking your website with a text browser. A text browser can display your site exactly the way a spider would see.  Once you view your site in a text browser you will have an idea what actions you might need to take.

There are lots of browsers like that out there. You can pick one that fits your need. But if you don’t want to spend time looking for one, downloading and installing one then there are still other options for you. You can use web based tools. One of my favorites is   http://www.seo-browser.com

Once you’re there just type in your URL and hit PARSE URL. Then it will display a text version of your site.

Type your URL

If you want to view more in-depth information about your site or page then switch to ADVANCED mode by clicking on top right-hand side corner of the site. You can also select agents from the drop down list. You can then see which spider sees what. Advanced option also gives you lots of useful information at a glance.  It will display page load time, page size, number of links etc. It also will show if you have any redirects placed on the site.

Switch to advanced mode

View detailed page information

This website also has some other handy tools for webmasters. As a webmaster you cannot ignore any crawler accessibility issue.

© 2009, Ricepan.com. All rights reserved. You may not republish this full article without permission under any circumstance.