TopRank Marketing Editor

How to create Google & Yahoo Site Maps – Easily

TopRank Marketing Editor     Google, RSS, SEO, SEO Tools, Yahoo

At TopRank we’ve been building Google & Yahoo site maps ever since each search engine debuted the service. However, recently I’ve been focusing on how to better automate the process and optimize my time spent creating site maps. I’ve researched and tried tool after tool until I came up with one that worked well for us.

It all started at Google. Google actually lists out a bunch of third party resources. How convenient! You can see them here:

Ones listed under Programs/Code Snippets appear to be code that you add to your website to create your own sitemaps. What I was looking for is a program to create site maps for client sites, so those were out.

CMS and Other Plugins is a nice list of add-ons for your blogging software or content management system. These will create the sitemap automatically and are probably the easiest way to create a sitemap. However, this is only for the one site again, not what I’m looking for.

Next comes the Downloadable Tools. This is where I was hoping to find the best software, however I was disappointed. Since I have a nice G5 iMac, I stared with the two Mac applications. Both cost money, but if they work well, it would be worth it.

Rage Google Sitemap Automator crashed before doing anything. I’m not sure if it was a fluke, or if it had startup issues or if it didn’t like my OS version, but I didn’t care; I tried another.

Second was Map-IT Sitemapper. Again it costs, but I wanted to try it to see if it would be worth it. It seemed to only want to index files on my harddrive and not link out to the web. Since my willingness to ‘figure out’ software today is short, I moved on.

I also tried GSiteCrawler for the PC and it seemsed to work well and it was free. However it seemed kind of complex and I didn’t finish exploring its options.

Next came the best category, Online Generators. I tried SitemapsPal but didn’t like the little text box that I had to copy the code out of. Tried Pingoat, but I could only seem to get a few links. Out of more than a thousand pages, it only found about 12. I reported the issues to their support form and tried again. Next on my testing list was and I was finally very happy.

Not only does it create the sitemaps, but it creates them in xml, gz and txt formats. It works beautifully and it’s free! One downside is that it can only do up to 500 pages, but if you have more than 500 pages, hopefully you can find a plugin for your system to create the map for you. I’m very happy with and have since bookmarked it for future use.

There are plenty of more choices on the Google site and probably many more that Google doesn’t list. My question to you is, what do you use for creating Yahoo and Google sitemaps?

– Thomas

PoorSo SoOKGoodAwesome (1 votes, average: 5.00 out of 5)


  1. Thomas…

    thank you very much for this post… I am going to try out xml sitemaps right now.

    Greetings from Germany

  2. Avatar says

    Great post and thanks for the recommendation. We do have a php script available which you run on your own server which will add all pages of your site to the sitemaps. A demo version is online at

    Another usual feature on this is if you have any links to 404 pages on your site it will tell you which pages they are on and what the links are.

  3. I have been experimenting with the GSite Crawler and am very impressed. It takes a little tinker with to get the options right…especially if you have a site with a lot of URL parameters. And there is no predefined page limit (500 pages is too limiting for my current assignment).

    It also creates the XML and GZIP versions for export along with a lot of other good data including error reports, stats, exportable URL lists, download stats (speed and size) and much more.

    It’s free too so you can’t really beat that. Plus, the developer is very good at providing support through a public forum. I’d recommend it for anyone who is interested.


  4. I have tried G Site Crawler and an online service whose name escaped me. Of the two I found G Site Crawler to be the best.

  5. Avatar Thomas McMahon says

    I’ll be testing out GSite Crawler more in the future as it seems like the best downloadable option for the PC. But, for connivence on my iMac, I like the online options. 🙂

  6. GSiteCrawler is a great tool, it creates URL lists for Yahoo too BTW. Others generate even HTML site maps and RSS site feeds.

    To determine what tool is the best to create the site maps and mass submission feeds, one should first ask for what type of site and for what type of update frequency the tool should support what business process.

    For example a blog or a forum should implement a plug-in updating the sitemaps on every content change. A huge dynamic site probably needs an individual solution, generating sitemaps from the database(s). For all kind of dynamic sites you cannot make use of (most of) the tools listed on Google’s links page, because they produce static sitemaps, that is you need to rerun the generator frequently to cover your updates, that’s a PITA.

    However, 7 months after the launch there are tools and design patterns out there which should fit any site’s needs. Just digg a little deeper 😉

    BTW you should try to submit your XML sitemap to MSN too. They don’t officially support the sitemaps protocol, but their spider eats those submissions and crawls URLs harvested from Google XML Sitemaps:

  7. Avatar Mary Stevens says

    I have just found your blog today…(referred by a friend) and am going to be studying the huge amount of content that you have here! Thanks for the effort. We are just about to launch a new site…and have been fighting with the site map software that we have…so I was happy to see this information.


    Mary Stevens

  8. Avatar Thomas McMahon says

    Glad to help Mary.

  9. Avatar Thomas McMahon says

    Thanks for the MSN tip. I didn’t realize they accepted submissions via XML.

  10. Great post, Thomas! I will try xml-sitemaps right away. Thanks for the links.

  11. I’ve just interviewed Google’s Sitemaps team:
    They provide a lot of information on Sitemaps, and finally give the conclusive answer to the “404 vs. 410” debate.

  12. Virtual Hand-Holding: Aids in Making Sitemaps
    In virtual hand-holding, it is an important tip to remember that people visit the site in order to look for some information. Internet surfers are an unforgiving lot. They only visit sites that are useful to them. With thousands of websites out there that offer the same features as your own site, how do you stand out? This is where sitemaps come in.

  13. One of the easiest way to generate sitemap is at: address. Just put your domain and the complette sitemap will be done. I have used it for my site : Digital Cameras

  14. For large sites ( >10,000 pages) on shared hosting, many sitemap programs exhaust the server memory, which is usually limited to 32MB on each site by default. You can increase the memory limit with your htaccess file, but don’t get your site closed down.

    I tried all of the sitemap generators listed above. Most of them failed with large sites. Finally, I settled on GSitemapCrawler. It stores all the URLs in a local database, so it does not exhaust the server memory. It can upload the sitemap as .xml or compressed as .xml.gz

    I found GSitemapCrawler to be easy to learn, use, and it handled the larger sites with no problem.

  15. This is a well explained awesome post on sitemaps. Saw this blog while searching for sitemap details. Cant go without commenting on your blog. Really an useful piece of information and a helpful post. Thanks.

  16. This is a well explained awesome post on sitemaps. Saw this blog while searching for sitemap details. Cant go without commenting on your blog. Really an useful piece of information and a helpful post. Thanks.

  17. This is a well explained awesome post on sitemaps. Saw this blog while searching for sitemap details. Cant go without commenting on your blog. Really an useful piece of information and a helpful post. Thanks.

  18. This is a well explained awesome post on sitemaps. Saw this blog while searching for sitemap details. Cant go without commenting on your blog. Really an useful piece of information and a helpful post. Thanks.


  1. » How to create Google & Yahoo Site Maps - Easily Affiliate & Blog Tips at ZillionBits make money online says:

    […] Originally by Thomas McMahon from Online Marketing Blog on January 5, 2006, 6:04pm […]

  2. […] This is also a good time to point out the XML site map post that I made a while back. […]