Is That Web Scale?

James Hamilton on Zynga shutting down data centers and moving back to AWS ( emphasis mine ):

These days, there is more data available and it’s fairly easy to show that “being at scale” requires order 10^6 servers, deep investments in server and networking equipment and the software stacks above, and massive investments in the software stack that manages these resources.

You read that correctly, if you have fewer than 1,000,000 servers you are not “at scale”.

Tracking TCP Round Trip Time in Nginx

The TCP_INFO option provides information about round trip times ( RTT ) for a TCP connection. Turns out that Nginx can expose this information for you as embedded variables:

$tcpinfo_rtt, $tcpinfo_rttvar, $tcpinfo_snd_cwnd, $tcpinfo_rcv_space

information about the client TCP connection; available on systems that support the TCP_INFO socket option

Easy to log this information. From there: periodically parse Nginx logs looking for client IP and RTT data, GEO IP lookup on the client, compute the median RTT value for each country/state, color code countries from highest to lowest RTT, then use Google Maps for the display.

You’ll have yourself a nice page showing the RTT values across the world for visitors to your site.

Do You Keep Your Promises?

Asynchronous JavaScript is not easy. There has been a strong push towards promises ( hello ES6! ). Unfortunately they also introduce some new side effects, which may not be immediately obvious. Nolan Lawson covers a number of these in We have a problem with promises.

In the end:

That being said, promises aren’t perfect. It’s true that they’re better than callbacks, but that’s a lot like saying that a punch in the gut is better than a kick in the teeth. Sure, one is preferable to the other, but if you had a choice, you’d probably avoid them both.

While superior to callbacks, promises are still difficult to understand and error-prone, as evidenced by the fact that I felt compelled to write this blog post. Novices and experts alike will frequently mess this stuff up, and really, it’s not their fault. The problem is that promises, while similar to the patterns we use in synchronous code, are a decent substitute but not quite the same.

If you are in the world of JavaScript callbacks and promises, you’ll really want to check out Taming the asynchronous beast with ES7 ( also by Nolan Lawson ).

Going to ES7 brings async and await ( hopefully ). Being able to write code that conceptually looks synchronous is very appealing:

let db = new PouchDB('mydb');
try {
  let result = await{});
  let doc = await db.get(;
} catch (err) {

This of course introduces yet another set of issues that you have to be aware of. That said, this feels like the right direction.


From the ‘you can never have too many ways to test your network’ department: TurboBytes Pulse:

TurboBytes Pulse enables you to quickly collect DNS, HTTP and Traceroute responses from many computers around the world. Most of these ‘agents’ are connected to consumer ISP networks.

HTTP results from around the world for


The source code is available at


If you are doing lots of work with dates and times in Python you may want to look at Arrow:

Arrow is a Python library that offers a sensible, human-friendly approach to creating, manipulating, formatting and converting dates, times, and timestamps. It implements and updates the datetime type, plugging gaps in functionality, and provides an intelligent module API that supports many common creation scenarios. Simply put, it helps you work with dates and times with fewer imports and a lot less code.

Every language needs something like strtotime.

FamilySearch Indexing, The Numbers Game

This is one of those times where the numbers seem overwhelming, FamilySearch Indexing Not Keeping Up:

“Do the math,” Jake said. FamilySearch captures about 150 million images in the field each year. FamilySearch is also scanning the microfilm out of the Granite Mountain Record Vault. This year FamilySearch expects to scan about 300 million images. On average there are four to five records per image. That amounts to about 2 billion records digitized just this year. And Jake expects to have the same amount next year. But we are only indexing about 250 million per year. That’s only 12% of the records “brought in the door.”

At the current rate it will take 8 years to index the images generated this year.

If you want to help close that gap the indexing program is open to anyone.

JavaScript Off

Tom Morris on Why I’m turning JavaScript off by default:

I snark a lot about JavaScript, but I’m of the opinion that most of the web would be improved if there were a lot less JavaScript running on it.

Even two years ago when this was originally written, this is considered heretical in many circles.

That issue aside, there is value in trying out the web with JavaScript turned off. It simulates what happens when your JavaScript has a fatal error, or takes 3 minutes to load. On a global Internet those are conditions that your users will run into.