Mr. Roger Lopez Up to this point we’ve been writing about the month to month results and the changes experienced by the sites monitored in the benchmark. This month we will focus on trending. Much like the benchmark gives perspective on how sites are doing compared to the industry, trending gives perspective on how a site has performed over an extended period of time.
Going back a year to July 2006 we will compare the benchmark average with this month’s top performer, Radisson; the bottom of the chart, Hotel-Guides; and two special cases, Hyatt and MGM Mirage.
The benchmark average as expected has had little variation in the past twelve month, staying around the 12 second mark. This tells us that as a group the sites have been pretty steady.
Having that as a baseline we’ll look at some of the sites and see how they have or have not changed over the past year. The ideal behavior would be that as changes have been made the sites have gotten progressively faster or at least remained steady. Comparing Radisson to Hotel-Guides we can see a big difference in response times for this month with 3.68 and 21.32 seconds respectively. Sure Radisson is much faster but of the two Hotel-Guides has made the biggest gain in performance. Hotel-Guides has gone from a high of 36.45 seconds in November of last year to 21.32 seconds for July. They’ve shaved 15 seconds from their response time in about six months. Not bad.
Radisson who back in August of 2006 was averaging 12.50 seconds seems to have made some changes that brought the average response times down to 5.92 seconds in October. They’ve made further progress and their current average is 3.68 placing them at the top this month.
Some sites’ behavior is quite the opposite where the response time over time has increased. An example of this is Hyatt who back in August of 2006 had an average response time of 8.81 seconds. Their performance has degraded over time and has increased by over ten seconds to the current average of 19.13 seconds.
Trending can also show how an out of control site has become consistent. Ideally the steps to improvement begin with availability, the second phase is to achieve a lowered response time and after you’ve achieved those two you want to make the site consistent. A perfect example is MGM Mirage. They have gone from having peaks and valleys from month to month that ranged anywhere from 8.63 seconds to 28.01 seconds to a consistent 19.5 seconds response time in the last few months.
As new technologies become available and adopted by the different sites, response times should improve. It’s important to have this baseline to make sure that changes made to the application help improve performance or at the very least maintain it.
Gomez Hotel Performance Index
June Results (7/1/07 – 8/1/07)
The Gomez Hotel Performance Index helps hoteliers keep an eye on the competition, measure customer experiences, and accurately understand the connection between web application performance, customer behavior and business results. Covering the top online hoteliers and aggregators, it measures the responsiveness; availability and consistency of the benchmark participants along these metrics:
Success Rate Summary
The majority of the hoteliers had a success rate above the benchmark average of 99.01%, which was higher than last months average of 98.82%. Gomez considers a rate above 98% as excellent performance. In the month of July, 17 of the 25 hoteliers performed above the benchmark average. Those performing above the average include Harrahs, Days Inn, Hotel-Guides, Hotels.com and Ramada. Notables that performed below the benchmark average were MGM Mirage and One Travel, with Hotels-and-Discounts rounding out the chart at 96.13% availability.
Response Time Summary
The benchmark average of 12.48 seconds is a little slower than the previous months 12.31 seconds. Radisson and Marriott continued to lead the pack for the eighth consecutive month with sub 6 second response times. Expedia rounds out the top three with an average response time of 6.18 seconds. Hotel-Guides is once again at the bottom of the chart with 21.32 seconds average response time.
Consistency Summary
Consistency is a measure of the disparity of the response time from visit to visit for completed transactions. The average consistency for the benchmark in July was 6.83 seconds, which is down from last month’s average of 6.53 seconds. 14 of the 25 hoteliers performed above the benchmark average. Radisson tops the chart with a consistency of 2.27 seconds. Notables below the benchmark average include Hyatt and One Travel.
Roger Lopez is a Benchmark Analyst in Gomez’s Professional Services Group. His role includes analyzing benchmark data reported by Gomez ExperienceFirstTM network and working directly with customers to optimize their testing. Prior to joining Gomez just over a year ago, Roger held positions as a technology support executive at TTG, Inc., a business territory design software company and in search engine optimization for BizX, a bartering service for online marketplaces. Roger has a degree in Systems Engineering from the Universidad de las Americas. Roger can be reached at rlopez@gomez.com
