Decrypting WebPageTest HTTPS Packets With Wireshark

The first thing I reach for to find out what is happening when a page loads is WebPageTest. There are times where even the detail that it provides is not enough, where I need to know what is happening at the network packet level. The problem is that more often than not the network packets are encrypted, in large part because of the increased use of HTTP/2. Fortunately WebPageTest makes it easy to both capture the packets for a test and provide the data needed to decrypt them.

Below are my notes on how to capture WebPageTest test packets and decrypt them in Wireshark.

You can use the WebPageTest results I used for this post at https://www.webpagetest.org/result/221122_BiDc56_3TF/. I’ve also got copies of the tcpdump file and TLS Key Log for use after the WebPageTest archive expires.

WebPageTest

There is one checkbox that needs to be set to tell WebPageTest to save the network packets for the test: Capture network packet trace (tcpdump) - under the Advanced Configuration section, on the Advanced tab.

When the test is complete there are two download links under the Individual Runs section that we need: tcpdump and TLS Key Log.

After those are have been downloaded we move over to Wireshark.

Wireshark

In Wireshark open the tcpdump file, in the example test I linked to above it is 1.cap. Scrolling through the list of packets you will see several with a Protocol label of TLS1.3 and an Info label of Application Data. Wireshark knows that these are TLS encrypted packets, but can’t see what is inside them - that is the whole point after all.

Next we will tell Wireshark about the TLS Key Log, this provides the details that it needs in order to decrypt the packets. Here is what that looks like in Wireshark 4.0.1. Select one of the TLS1.3 - Application Data packets and right click on it. In the popup menu goto Protocol Preferences → Transport Layer Security → (Pre)-Master-Secret log filename.

That will open up a small section above the packet list where you can point it to the TLS Key Log that you downloaded from WebPageTest.

With that in place Wireshark can now decrypt the TLS packets and parse the HTTP/2 level data. Scrolling through the packet list you will now see green highlighted HTTP2 protocol packets.

In my example packet 37 is the one that contains the initial GET request. Wireshark is able to show the HTTP/2 frame, decrypted TLS, and decompressed versions of that packet. Because it understands HTTP it also parses that for easy reference.

That is all it takes to get access to the unencrypted network packets of a WebPageTest result. From here you can explore each step from the network point of view.

Here are a few examples, with packet numbers based on the example WebPageTest result I linked to at the top.

• Packet 1: first network packet is a DNS request
• Packet 34: contains the first SETTINGS and WINDOW_UPDATE data
• Packet 40: has the HTTP response headers for the first request
• Packet 61: includes the HTML text

If timing is something you are trying to pin down, you can go from millisecond to microsecond level precision. In Wireshark go to View → Time Display Format → Microseconds.

Being able to step through the whole process packet by packet is a powerful option. The combination of WebPageTest and Wireshark makes it easy to collect and browse through the decrypted packet stream.

When in doubt, look at what went over the network, in detail.