On this blog, I've talked about adding metadata to your Drupal site and tracking your Drupal site with Google analytics. The next step on the way to search engine happiness is to begin spoon-feeding Google all the latest content on your site.

The way to do that is through a sitemap that you register with Google. Right away we have to make a clear distinction between HTML sitemaps and XML sitemaps.

HTML Sitemaps

The kind of sitemap you may be familiar with is the HTML sitemap, the kind of map you might look at if you're truly desperate to find something on a large site. There is an example of such a map available on the left column menu; I have cleverly called it "Site map".

Some sitemaps are schematic in appearance, showing paths and connections branching from the home page.This is nonsense. An effective website has many points of entry, and has sophisticated linkages, navigation, and search capabilities. There is no golden path - users go where they want, guided by interest and relevance, not by any kind of hierarchical structure.

The HTML sitemap on this site is powered by the Drupal Site Map module. There are other modules you can add, but this one will do for now. If you click on the link, you'll see all kinds of stuff - the front page, the blogs (there's only the one), primary links (the ones across the very top of the page, including my resume), and tags. This is the sitemap from a content point of view, allowing visitors to quickly find what they want according to author, category, and topic.

One quick note on configuration. The Site Map module allows you to control all kinds of behaviour, but the most important thing to set is which content and menus to display. There is a section on Site Map administration page marked "Menus to include in the site map:". I chose to select Primary links and Secondary links, and the reason I did that is to make sure important pages like my resume page appear. I did not select Navigation, because that created and unwieldy and redundant sitemap. If visitors could find the content based on the navigation, they wouldn't need the sitemap.

The other important section is marked "Categories to include in the site map:". Here I selected both items that appeared for me, Blog topic and Tags. Tags are very important - so important I will devote another blog post to it. Just remember to select it here.

So that's what you need to know about HTML sitemaps, except for one final, vitally important point: Google has no use for your HTML sitemap.

XML Sitemaps

Google's crawler robots will follow all the links on your site, but that doesn't mean they will find all the content. For instance, older blog posts may not still be listed on your Recent Blog Posts menu, and they may not be linked to by other posts, but they're still there and they're still important. Site search may uncover them, and HTML sitemaps may link to them, but these methods aren't reliable enough for Google. So Google puts the responsibility on you, the webmaster, to feed it what it craves. What it needs is well structured, reliable data. And that's where XML sitemaps come into play.

XML sitemaps are based on an industry standard protocol that is explained and defined at sitemaps.org. Don't follow this link - I include it merely out of politeness. I can also show you the XML sitemap for this site. Don't follow that link either, it won't leave you any smarter. The thing you want right now is the Drupal XML Sitemap module, which will give you the tools you need to give Google what it needs.

Where XML sitemaps are actually useful is with Google Webmaster Tools, which is the place to find an enormous amount of SEO information. It's also the place to peek behind the curtain and see what Google thinks of your site. 

Note how this is different from Google analytics. Analytics will tell you what kind of attention your site is actually getting, including what keyword searches have brought people to your site. Google Webmaster Tools tells you what your site looks like from Google's perspective, and includes a lot of information that determines how you rank on Google searches.

There's buckets more to say about Webmaster Tools, but we're focused on sitemaps right now. Once you log in (you can use the same account you used for Google Analytics) you need to add and then verify your site. Google gives you two options to verify - you can either add a metatag to your homepage or you can upload a specific HTML page to your site.

If you go the metatag route, you can use functionality provided by the Nodewords module, which you can find on your Drupal site navigation under Administer -> Content management -> Meta tags. Once there, you'll have to click "Default and specific meta tags", and then under that "Front page". There you will find, toward the bottom of the page, "Google Webmaster Tools verification code:". Switch back to your Google Webmaster Tools and select Meta tag from the drop down menu next to Verification method. You'll see the full metatag displayed there, looking something like this:

<meta name="google-site-verification" content="G9NnNcgx1FcMCmgzSgAbQSFko2dnISoOs0_S4Ob5RvA" />

What you want is the code, the big ugly string of text next to content=". Copy that, switch back to your Drupal site, paste it into the "Google Webmaster Tools verification code:" field, save, and you're done.

Well, maybe. Truth be told, when I tried this method, I got nothing. No metatag was added to my front page, so I went for Plan B, which was to upload the HTML file that Google provided. Seems there was a bug in Nodewords. The good news is, now I can see the metatag, so the latest version has fixed the problem. It's good to have options.

Once you've done this, completed the verification process with Google, you're golden. I'll cover the information you get from Google Webmaster Tools in another post, but for now, you're more or less done with sitemaps.