Smart Start Hotel Executive Report: A New Type of Benchmark
By Roger Lopez, Performance Analyst, Gomez
Mr. Roger Lopez
It was announced earlier this year that Netscape, a pioneer of the World Wide Web, would no longer be supported as of February 1st 2008. Back in the mid 90s, Netscape was the dominant Internet browser, the popular choice of 90% of web users. Today, this once-prevalent browser is now barely used by 0.6% of web users.
Fast forward to Internet Explorer (IE), which until the last year or so, enjoyed similar 'exclusivity' as Netscape did in the 90s. This got me thinking about browsers and websites and how some organizations still create their websites solely for viewing in IE. However, the days of IE being THE browser of choice have come and gone. Today there are many more browsers being used to access the Internet, including mobile browsers too.
According to Wikipedia, the top three browsers are IE, Firefox and Safari (Figure 1). This graph shows how browser usage as of February 2008.
So I thought I would explore how the hotel websites in the Gomez Hotel Room Search Benchmark appear when viewed across a variety of browser and operating system (OS) combinations. How do they look when viewed at the smallest of the 'standard' resolutions, 800x600?
The first problem was deciding which browser/OS combinations to test. There are substantially more possibilities today than even a few years ago, as you can see in figure 2. I had three OSs to choose from (Linux, Mac OSX and Windows), and seven different browsers (IE, Mozilla, Firefox, Safari, Konqueror, Opera, Netscape and Camino.) I then had to figure out how to rate the websites. To keep things simple, I decided to award one point for every browser/OS combination where the site looked good and was usable, i.e. no chopped menus or overlapping text, all menus and text boxes present.
With the basics figured out, I captured screen shots of these websites in the myriad of combinations. I'm not sure what I expected to find: I assumed that most, if not all, websites would work just fine with the odd one behaving in a slightly strange manner, but nothing of consequence. The results were a little surprising: some websites had minor but very noticeable issues.
For example, Best Western's website excludes non-flash enabled browsers. Because of this, visitors with no flash see an empty box advising them to download a missing plug-in. While not technically an error, (most people would just download the plug-in and get on with it) an empty box looks bad. Marriott's way of dealing with the lack of flash is far more elegant. They display an image in place of the flash movie, allowing the visitor to have the same experience on the website without having to download something. They followed a simple rule: your brand is on display, make it look good no matter what.
A number of websites had problems with Netscape. To be fair, web designers shouldn't really be held accountable for not supporting Netscape 4.0, which came out circa 1997. However, in my opinion there's no reason for a properly built site not to render in any browser. For example, in figure 5 Expedia simply tells the user to upgrade.
All of the things I've pointed out so far are pretty harmless, either because they are minor blemishes or because they happen in a browser that is largely unused. However I did run across one website that did not do well at all.
To verify my results, I opened Firefox on my Windows XP laptop, and typed www.starwoodhotels.com. The result at a large resolution was the Starwood website as expected. But changing the browser to an 800x600 resolution made a drastic difference (figure 6). Practically nothing appeared, just a large and empty box.
To calculate the averages, I added up the total points gained and divided by the total possible points. I then had to decide how to group the scoring. Grouping by OS meant adding up the total for all browsers running on Windows, MAC OSX and Linux. A quick glance revealed that the only website that got full marks or 100% for browsers on Windows was Hotel-Guides.com.
The worst was Starwood with a staggering 46.88%. On Mac OSX, I found several websites that did well across all browsers, the most notable being InterContinental Hotels and Choice Hotels. Starwood was again the worst performer at 40%. Linux was the OS where almost everyone scored 100% except for two sites, LasVegas.com and Starwood.
A more interesting way to add up the scores was to do it by browser regardless of OS. I came up with two different scores, one as an average for all browsers and one as an average of the top browsers.
The two categories I used are an average of all browsers and an average of the top three browsers. As expected all sites performed better when only the top three browsers were taken into account. Even still, there were only a couple of websites that reached the 100% mark. The combined average for all sites and all browsers is about 80%. That is surprisingly low by today's standards. Even when you weed out the browsers with less than 1% usage you still only get to a combined average of 89.2%.
Maybe the variety of browser/OS choices is a bit too much for designers to deal with. Web designers have to contend with Firefox 1.5, 2.x and now 3.x, as well as IE 6, 7 and now the 8 beta, not to mention the greater need to think about a mobile version of the website for viewing on all those smart-phones people are using.
In the end, it all depends on knowing who your customers are and how they are accessing your website so that you can ensure they are receiving a quality web experience. But it's also useful - and relatively easy - to know how your site looks in all the browsers your customers use, just in case one of today's 'unpopular' browsers turns out to be the Firefox or Safari of tomorrow. To conduct the tests referenced in this article, I used Gomez's Reality View(tm) XF solution.
Roger Lopez is a Performance Analyst/Consultant in Gomez's Professional Services Group. His role includes analyzing benchmark data reported by the Gomez® ExperienceFirst(tm) network and working directly with customers to optimize and improve their web experience testing and performance. Prior to joining Gomez two years ago, Roger held positions as a technology support executive at TTG, Inc., a business territory design software company and in search engine optimization for BizX, a bartering service for online marketplaces. Roger has a degree in Computer System Engineering from the Universidad de las Americas. Roger can be reached at rlopez@gomez.com
Gomez® and Gomez.com® are registered service marks, and ExperienceFirstTM and Reality View(tm) XF are service mark of Gomez, Inc. All other trademarks and service marks are the property of their respective owners.
