What Really Went Over The Wire?

The point is, some days you have to drop down into tcpdump and pcap files to get the truth.

- Steve Souders while tracking down how much HTML5 video data iOS actually receives.

Turns out server logs were giving an incorrect picture of the amount of data that was being transmitted. It took watching individual packets to track down what was really happening. Go read the post for the odd twist at the end.

Reading this reminded me of using Wireshark to track down multiple packet issues in HTTP POST requests from XMLHttpRequest.

Chrome 19 Beta Crashes on stevesouders.com

Chrome Logo

I’m using Chrome 19.0.1084.36 beta as my primary browser, and I noticed something really odd recently. Chrome 19 beta crashes any time I visit stevesouders.com (Steve is the web performance guru at Google).

This isn’t an isolated single tab crash, the whole Chrome browser comes tumbling down. I tried it with Firefox 12 and Safari 5.1.5 and both worked fine.

I think this makes Chrome 19 the worst performing browser when viewing stevesouders.com :-)

Performance Trends For Top Sites On The Web

Steve Souders posted an update on the HTTP performance trends for top sites, based on data gathered via http://httparchive.org/. Here are the bottom line numbers:

Here’s a recap of the performance indicators from Nov 15 2010 to Aug 15 2011 for the top ~13K websites:

  • total transfer size grew from 640 kB to 735 kB
  • requests per page increased from 69 to 76
  • sites with redirects went up from 58% to 64%
  • sites with errors is up from 14% to 25%
  • the use of Google Libraries API increased from 10% to 14%
  • Flash usage dropped from 47% to 45%
  • resources that are cached grew from 39% to 42%

I was surprised by the total transfer size increase. If you followed that trend on a weekly basis, every Friday for the last 9 months you added another 2.6 kB to the total transfer size of your site. Not much for any given week, but it adds up fast.

Performance Analysis: beta-newsroom.lds.org

A new version of newsroom.lds.org.org has recently been launched at http://beta-newsroom.lds.org/. This seemed like a good opportunity to give it a once over Steve Souders style, looking at the performance of the new site.

The first thing I did was run it through webpagetest.org; with the Dulles, VA – IE 8 – FIOS settings. You can see the results at http://www.webpagetest.org/result/100924_5TGG/. A quick glance at the category scores (top right) shows that there is likely plenty of room for performance improvements.

Compression

The first thing that stood out was the lack of compression for resources. Starting with the very first request, the HTML for http://beta-newsroom.lds.org/ there is no compressed version made available. A simple way to confirm this is with curl:

curl -v --compressed http://beta-newsroom.lds.org/ > /dev/null

Which makes an HTTP request that looks like:


> GET / HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: beta-newsroom.lds.org
> Accept: */*
> Accept-Encoding: deflate, gzip

and gets back a response of:


< HTTP/1.1 200 OK
< Content-type: text/html; charset=utf-8
< Server: MarkLogic
< Content-Length: 33232
< Proxy-Connection: Keep-Alive
< Connection: Keep-Alive
< Age: 9064
< Date: Fri, 24 Sep 2010 05:07:27 GMT

The raw HTML is 33,232 bytes. The compressed version of the HTML is 8,549 bytes. An easy way to trim the page size by 24,683 bytes. I'd be surprised if their web server (MarkLogic) doesn't support compression, so this is likely a simple configuration change some where.

Going done the list of easily compressed resources:

  • http://beta-newsroom.lds.org/assets/styles/screen.css - 105,478 bytes, 16,689 compressed, saves 88,789 bytes
  • http://beta-newsroom.lds.org/assets/scripts/jquery-1.4.min.js - 69,838 bytes, 23,666 compressed, saves 46,172 bytes
  • http://beta-newsroom.lds.org/assets/scripts/common.js - 19,574 bytes, 5,080 compressed, saves 14,494 bytes
  • http://beta-newsroom.lds.org/assets/scripts/jquery.cycle.all.min.js - 23,729 bytes, 7,122 compressed, saves 16,607 bytes
  • http://beta-newsroom.lds.org/assets/videos/uvp/scripts/swfobject.js - 21,543 bytes, 4,728 compressed, saves 16,815 bytes
  • https://secure.lds.org/stats/s_code_ldsall.js - 31,543 bytes, 12,415 compressed, saves 19,128 bytes

On a fully loaded page that weighs over 600KB turning on compression support for these 7 items would reduce it by 226,688 bytes.

The https://secure.lds.org/stats/s_code_ldsall.js file is servered by a different web server, Netscape-Enterprise/4.1, which is fairly old. I'm not sure if it even properly supports compression. If not throwing a caching server (nginx, varnish, squid, etc.) in front would do the trick.

Another method for reducing the file sizes is minifying the CSS and Javascript (in some cases this is being done already).

Blocking

Loading all of that Javascript at the top of the page is causing other downloads to block. As a result the site doesn't start to render until we are more than a second into the page. This is a good place to remember the rule of thumb: load CSS as early as possible and Javascript as late as possible.

There is one block of inline Javascript in the page, but it is at the very bottom.

For the most part there doesn't appear to be too many places where parallel downloads are blocked. One thing that could be done to improve parallel downloads though is to spread out more of the resources across different host names.

Images

The page loads 51 images, totaling 248KB. The Page Speed results indicates that these could be optimized to reduce their size by 104KB.

Serving images (and other static content) from another domain would also cut down on cookie data sent back for each request.

Caching

Here is another area that I was really surprised by, the bulk of those images don't provide cache headers. As a result browsers will re-download those images for each page load through out the site. The repeat view waterfall chart should be much smaller, no need to fetch all those images on every page view.

Throwing in some Last-Modified and ETag headers will clear that up.

Conclusion

There are some other techniques that could be employed to help speed things up, but they depend heavily on how the site is developed and deployed. There is already enough low hanging fruit to make a big difference.

I think you could conservatively target the total size for the page at less than 300KB. This would reduce the amount of data transmitted to browsers by more than 50%. Another benefit is the time it takes for the page to fully load. Currently it is right at 2.5 seconds. Something closer to 1.5 seconds seems reasonable. With proper caching headers return visitors could see times under 1 second.

All of this was just for the front page of the site. I haven't looked at any other pages on the site, but I suspect that they'd benefit from the same items listed above.

Recommended Reading: Steve Souders, High Performance Web Sites

Although I link to various sites when something interesting comes up, I haven’t done a good job of recommending people and blogs to follow. Time to fix that.

Kicking off my recommended reading list is the High Performance Web Sites blog by Steve Souders. I’ll make this very simple, if you do web development at all this blog is REQUIRED reading. When it comes to front end web performance Steve has literally written the book on the subject (High Performance Web Sites), twice (Even Faster Web Sites).

I was at his WordCamp SF 2009 session where he made changes to a test site during the session and showed how each change improved the page load time. If you aren’t able to make it to an event where Steve is speaking several of his session videos are online, like this 2009 Google I/O session – Even Faster Websites. It’s worth the time to see what his research has turned up on how browsers work.