Mr. Roger Lopez

Comparing the data for the last two months I noticed there is generally not much difference in the way websites perform from month to month. Significant changes tend to occur around site updates or modifications, or if there is a surge in the number of visitors and the site is not ready to handle the traffic.

While reviewing the response time, consistency and success rate for each of the benchmarked sites, I realized how complex websites have become. Gone are the days where your site gave people basic information about your company, i.e. contact information, directions to your location, company history, etc. Those sites were, for the most part, made up of a single HTML file and maybe a Cascading Style Sheet (CSS) to make it look nice.

Now websites have all sorts of bells and whistles that do all kinds of fancy things like interactive mapping, dynamic calendars and live inventory checks. But what effect does all this ‘stuff’ have on the site’s performance?

The way most websites add all this functionality is by writing code that will perform an action when the user clicks, hovers or checks something on the page. The code can be either part of the HTML for the page or it can be a separate file. With more functionality more files of code are needed.

Since most sites add functionality via JavaScript (JS) files I decided to look at the top performing two sites and compare them against the three worst performers. I analyzed the number of external JS files used by the site for the transaction, the combined weight of the files, total weight for the transaction and the response time. The idea is to see what percentage of the transaction’s weight (total bytes) is made up of JS files and see if there is a relation to response time.

Here’s what I found:

The first thing that stands out is One Travel’s percentage. Two thirds of the site is JavaScript. Considering their response time for this example was 25.327 seconds and their monthly average is 24.79 seconds I would say they could benefit from either less JavaScript or compressing the files to help their response times.

While looking at the JS percentages it became obvious that there are more factors that effect response time than simply how many JavaScript files are needed by the site. Marriot is only about 200 KB lighter than LasVegas.com yet the difference in response time is four times as much for LasVegas.com. While Marriot JS files make up only 13% of the content, LasVegas.com’s content is made up of mostly JS files. It is likely that the difference in response time is directly related to the amount of JavaScript. Next month we’ll measure the Efficiency of these two sites to see if we can find out more about why their performance is so different.

Gomez Hotel Performance Index

September Results (9/1/07 – 10/1/07)

The Gomez Hotel Performance Index helps hoteliers keep an eye on the competition, measure customer experiences, and accurately understand the connection between web application performance, customer behavior and business results. Covering the top online hoteliers and aggregators, it measures the responsiveness; availability and consistency of the benchmark participants along these metrics (see full methodology):

Success Rate Summary

The majority of the hoteliers had a success rate above the benchmark average of 98.58% in August and 98.74% in September. Gomez considers a rate above 98% as excellent performance. In the months of August and September, at least 16 of the 25 hoteliers performed above the benchmark average. Notables that performed below the benchmark average were Hilton, Hyatt and Expedia.

Response Time Summary

The benchmark average for August was 12.54 seconds which is a little faster than the September’s 12.57 seconds. Radisson and Marriott continued to lead the pack with sub six second response times. At the bottom of the chart we find LasVegas.com for the month of August and One Travel rounds out the charts for the month of September.

Consistency Summary

Consistency is a measure of the disparity of the response time from visit to visit for completed transactions. The average consistency for the benchmark in August was 6.49 seconds, which is down compared to September’s average of 6.17 seconds. At least 13 of the 25 hoteliers performed above the benchmark average. Radisson tops the chart during both months. Notables below the benchmark average include Hyatt and One Travel.

Roger Lopez is a Benchmark Analyst in Gomez’s Professional Services Group. His role includes analyzing benchmark data reported by Gomez ExperienceFirstTM network and working directly with customers to optimize their testing. Prior to joining Gomez just over a year ago, Roger held positions as a technology support executive at TTG, Inc., a business territory design software company and in search engine optimization for BizX, a bartering service for online marketplaces. Roger has a degree in Systems Engineering from the Universidad de las Americas. Roger can be reached at rlopez@gomez.com