Create Sitemaps with SiteOrbiter

  Janet Fouts       February 6, 2008      Tutorials Mac Developer

Websites are getting larger and larger, and keeping track of them as they evolve can be a hassle. If you’re a web developer, you’ll want to create what’s called a site map, a special file that lists all of the pages of your website in an hierarchical order. This file can be submitted to search engines to help them index your website. But how the heck are you supposed to create one on your Mac?

Enter a native Mac application called SiteOrbiter. SiteOrbiter is freeware (registration required) and requires Mac OS 10.1 or later. This application will index your files, create an XML site map for Google and other search engines, and also create a visual sitemap that can be saved as a PDF.

Here’s how to do it:

1. Download SiteOrbiter from http://www.siteorbiter.com. To install the application, open the disk image and drag SiteOrbiter to your Applications folder.

2. When you open SiteOrbiter for the first time, you’ll be greeted by the SiteOrbiter Assistant. Enter the URL of the website you’d like to index or, if it’s located on your Mac’s hard drive, click the Browse button and select the top level directory.

   

3. SiteOrbiter will start indexing your website. This could take a while, especially if you have a large website. You’ll see files flashing by in the status display in the upper-left corner of the screen (below).

   

4. Once SiteOrbiter has finished indexing your website, you’ll see a graphical representation of your website in the main window.

   

5. Save your sitemap for search engines by selecting Save from the File menu. You can also export the graphical representation of your website by selecting “Extract to PDF” from the File menu.

You can also export the data as a sitemap for Google by selecting “Generate Site Map” from the Reports and Maps menu.

Taking Another Look at the SiteOrbiter Assistant

Now that you’ve created your very own sitemap with SiteOrbiter, it’s time to take a look under SiteOrbiter’s hood. We’ll start by reexamining the SiteOrbiter Assistant. What do all of those options actually do?

• User Agent: This menu is especially interesting. You can appear to the server to be using any agent you like. If, for example, your site is designed to deliver different dynamic data to different browsers, you can generate a site map of what each of those browsers are being delivered and check for accuracy of your sniffer scripts.
• Maximum Recursive Depth: This option allows you decide how deep to descend into the bowels of the website. If you’re not sure, start with 8 or 10 and see if a lot of pages are missing. If so, you can adjust as necessary.
• Copy Remote Site: You can also use SiteOrbiter to download a remote website to your Mac’s hard drive. Use this cautiously. It can take a while to download, and if the site isn’t yours please don’t violate any copyrights!

Configuring SiteOrbiter’s Preferences

• **Download Folder:**Select the folder you want to use as an archive. Note: If you download a site, and you move the files, SiteOrbiter will download the entire site again when you next request a download. If not, SiteOrbiter will only download files that have been modified.
• **Only Include:**You can set this to include, for example, only HTML, PHP or maybe just SWF files.
• Treat Links with Arguments: Each argument will be recorded as a seperate page. (I.e.: http://www.mysite.com/index.pl?today and http://www.mysite.com/index.pl?tomorrow would each be separate pages.)

Understanding SiteOrbiter’s Toolbar

• Index/Stop: Clicking this will re-index your website or, if SiteOrbiter is currently re-indexing, stop the re-indexing process.
• Browser: Highlight a page and click Browser to display that page in a browser window.
• Up, Left, Down, Right: Moves the map around in the window. You can also use the scrollbars.
• Hide Map: For a large site, redrawing the map can be a processor hog, so you can turn it off to improve performance.
• Download: Again, please be judicious. This can be a boon when you are taking over a website, and you’ll have the site map as well. Once you’ve downloaded the site you can use this feature to keep your disk copy up to date. SiteOrbiter will only download differences after the initial download.
• Transcript: This shows header information. Useful with dynamic pages and scripts.
• Only HTML: Ignores non-HTML files in the map view.
• Select Lock: You can lock a node to keep it open even as you click on other nodes. Think of it as a placeholder.
• Names: Turned off by default, this will show the names associated with the nodes.
• Selected: Shows just the info for the selected node.
• Zoom: Zoom out for a macro view or in on a particular node.

Using the Map Window

• Subset mode: Clicking on a link in the map window will select that link and you can then view any links related to that “node.” This makes it simple to quickly see the relationships between objects in the site, and you’ll see outbound and inbound links.
• All Objects: This display mode can be quite daunting on a larger site, but it makes it easy to spot orphaned pages and get a very high level view of the site. The home page is always identified as the starting object, but you can use “show from” mode to select a particular node and show the site from that viewpoint.

Conclusion

That’s pretty much it. All in all, SiteOrbiter is a useful tool for getting a grip on the loose ends that may be lolling around in your website, and it can also give you a lot of insight into who’s linking to your site and what pages they are linking to. Combine that with your traffic stats and you’ll have a much better picture of what your users are up to!