Timing Details With cURL

Jon’s recent Find the Time to First Byte Using Curl post reminded me about the additional timing details that cURL can provide.

cURL supports formatted output for the details of the request ( see the cURL manpage for details, under “-w, –write-out <format>” ). For our purposes we’ll focus just on the timing details that are provided.

Step one: create a new file, curl-format.txt, and paste in:

\n
            time_namelookup:  %{time_namelookup}\n
               time_connect:  %{time_connect}\n
            time_appconnect:  %{time_appconnect}\n
           time_pretransfer:  %{time_pretransfer}\n
              time_redirect:  %{time_redirect}\n
         time_starttransfer:  %{time_starttransfer}\n
                            ----------\n
                 time_total:  %{time_total}\n
\n

Step two, make a request:

curl -w "@curl-format.txt" -o /dev/null -s http://wordpress.com/

What this does:

  • -w "@curl-format.txt" tells cURL to use our format file
  • -o /dev/null redirects the output of the request to /dev/null
  • -s tells cURL not to show a progress meter
  • http://wordpress.com/ is the URL we are requesting

And here is what you get back:

            time_namelookup:  0.001
               time_connect:  0.037
            time_appconnect:  0.000
           time_pretransfer:  0.037
              time_redirect:  0.000
         time_starttransfer:  0.092
                            ----------
                 time_total:  0.164

Jon was looking specifically at time to first byte, which is the time_starttransfer line. The other timing details include DNS lookup, TCP connect, pre-transfer negotiations, redirects (in this case there were none), and of course the total time.

The format file for this output provides a reasonable level of flexibility, for instance you could make it CSV formatted for easy parsing. You might want to do that if you were running this as a cron job to track timing details of a specific URL.

For details on the other information that cURL can provide using -w check out the cURL manpage.

28 thoughts on “Timing Details With cURL”

  1. @Joseph,

    That’s what I did. But I was wondering if there was better way using just the curl command.

    Thanks! :)

  2. Can anybody tell me when ever I am sending a curl request (curl http://x.x.x.x/file_name) if I want to parse the size of the file then how to do?
    I already trace the size from header.(i dump the header bu “-D” and redirect it).but it is taking so much time.

  3. If the server includes the ‘Content-Length’ header in the response then that will give you the size of the file. If it is a large file and you don’t want to download the whole thing just to get the size, try making a HEAD request. Here is an example:


    curl -v --compressed -I https://s1.wp.com/home.logged-out/css/homepage-one-field-long.css > /dev/null

    The response includes “Content-Length: 35326″.

  4. thank you sir,
    but the response still contain 6-7 line of information including “200 OK”,”last modification”,”accept-range” ,”content-length” etc i just want 1 line information i.e “content-length : 23456 “.can it be possible?

    regards
    swa

  5. If in curl I want to extract an exact size data let curl http://x.x.x.x/ 1.doc 1234
    where 1.doc is of size 30000 bytes and i want 1234 bytes from that file 1.doc. Whenever I will send the request the curl request will extract for 1234 byte data only. Is there any way to do this.

  6. OK still thank you. Then is there any way to send multiple curl request in parallel . Multiple means not 10 or 100 or 1000 its like 100,000 request. I tried the method
    curl http://x.x.x.x/{1.doc,2.doc,3.doc………} but it limits upto 10500.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>