What Can We Learn While Crawling The Web?

What's the best way to learn about a client's website besides simply browsing around the pages and checking things out?  For an SEO who needs to know as much as possible about a site, they have to crawl it.  By crawling a site, they can learn about the structure of the URLs.  This allows the [...]