Python EXecutable

What happens when you combine Python executing and zip files? You get pex ( Python EXecutable ):

pex is a library for generating .pex (Python EXecutable) files which are executable Python environments in the spirit of virtualenvs. pex is an expansion upon the ideas outlined in PEP 441 and makes the deployment of Python applications as simple as cp. pex files may even include multiple platform-specific Python distributions, meaning that a single pex file can be portable across Linux and OS X.

Brian Wickman covers the basics of how this works in a 17 minute video. The video is definitely worth watching if you are curious about this.


If you are doing lots of work with dates and times in Python you may want to look at Arrow:

Arrow is a Python library that offers a sensible, human-friendly approach to creating, manipulating, formatting and converting dates, times, and timestamps. It implements and updates the datetime type, plugging gaps in functionality, and provides an intelligent module API that supports many common creation scenarios. Simply put, it helps you work with dates and times with fewer imports and a lot less code.

Every language needs something like strtotime.

Python Version Usage

It has been more than 5 years since Python 3.0 was released. Last week I mentioned a Python 2 vs. 3 article that pitched a reviving of Python 2.

With all that in mind, here is the 2015 Python scientists survey from Thomas Robitaille. The main results:

  • 83% reported using Python 2.x ( 2.6 & 2.7 )
  • 17% reported using Python 3.x ( 3.4 )

This might not be as bad as it sounds. He did a similar survey in November 2012 ( nearly 3 years after Python 3.0 ) that reported Python 3.x usage in the 1% range.

That is an average increase of 8% across the roughly two years between the surveys. If that remains constant it will be another 9 years before 2.x usage drops below 10%.

Python Uphill

Major upgrades to programming languages often run into difficult uphill battles. From Python 3 is killing Python:

People either continue to write software in Python 2 or they pick another language that did not shoot itself in the face.

I have been building applications and services in Python 3. I’m not blown away by it. It is very similar to writing software in Python 2, except that there is a much smaller pool of 3rd party libraries available.

There are other solutions, but reviving Python 2 is so obviously the correct thing to do, that other solutions are not worth mentioning.

Clearly not everyone agrees that re-focusing on Python 2 is a good idea.

See also PHP 6 and Perl 6.


Brython, Python in your browser:

Brython is designed to replace Javascript as the scripting language for the Web. As such, it is a Python 3 implementation, adapted to the HTML5 environment, that is to say with an interface to the DOM objects and events.

That is a trip. Practical? Probably not. Definitely neat though.

The brython.js code that does all the heavy lifting is only 175,936 bytes.

I am Speaking at the OpenWest Conference

The OpenWest Conference is happening May 2-4, 2013 (formerly the Utah Open Source Conference) at Utah Valley University in Orem, Utah.

This year the keynote speakers are Rasmus Lerdorf, creator of PHP, and Mark Callaghan, lead of the MySQL engineering team at Facebook.

For my part I’ll be giving three different presentations this time around. First up is “Simple Filesystems with Python and FUSE”, where I’ll cover the basics of getting a simple filesystem up and running written in Python using the FUSE library. Next up is “Site Testing with CasperJS”, which is an intro to using CasperJS to run user tests against your site. Last, but not least, is “Scaling WordPress”, where I’ll talk about some of the methods that (the largest WordPress install in the world) uses to host tens of millions of sites that add up to billions of page views per month.

I tried to keep my session titles direct and to the point. At times there will up to ten sessions running at once ( OpenWest session schedule ), so I wanted people to be able to tell at a glance what my sessions are about.

Tickets for OpenWest are available at $80. Every open source group in the area has been given a discount code though, so you can bring that down significantly.

If you’ll be at the OpenWest conference be sure to say hi.

wpcomfs – A Filesystem

Back in April announced a new REST style API. That got me thinking about writing a filesystem layer to expose that data, along similar lines to the pressfs code I’d written last year.

It is still rough, and only supports read-only public data, but in the spirit of release early (and often) I’m sharing the code for wpcomfs at

Assuming you have FUSE with Python bindings already working on your system you can download this and start trying it out in three easy steps:

  1. mkdir /tmp/wpcomfs
  2. python /tmp/wpcomfs/
  3. mkdir /tmp/wpcomfs/sites/

After those steps you’ll have read-only data for available at /tmp/wpcomfs/sites/

The mount point /tmp/wpcomfs and the site are just examples. You mount wpcomfs where ever you’d like and you can expose public data for any public site hosted on Since there are millions of sites hosted at wpcomfs will only load data for sites that you mkdir.

This also works for sites that are using mapped domain names. For instance mkdir /tmp/wpcomfs/sites/ will provide you with a filesystem layer for


When you mkdir a site you get site data, recent posts, and recent comments. In order to keep interactions with the filesystem responsive this data is only loaded once, when mkdir is run. Running rmdir /tmp/wpcomfs/sites/ will remove the site data.

The top level directory for a site looks like:

$ ls -la /tmp/wpcomfs/sites/
total 14
-r-------- 1 root root    7 Jun  6 14:09 ID.txt
-r-------- 1 root root   28 Jun  6 14:09 URL.txt
dr-------- 2 root root 4096 Jun  6 14:09 comments
-r-------- 1 root root   61 Jun  6 14:09 description.txt
dr-------- 2 root root 4096 Jun  6 14:09 meta
-r-------- 1 root root   18 Jun  6 14:09 name.txt
dr-------- 2 root root 4096 Jun  6 14:09 posts

The contents of each file are available in read-only mode:

$ more /tmp/wpcomfs/sites/

All of the dates exposed for files and directories in wpcomfs are based on when the site data was loaded. Basically when mkdir was run for the site.

You can mkdir as many sites as you want. I haven’t tested an upper limit, but I imagine if you load enough of them your system will run out of memory and die.


I have noticed a few bugs. Specifically, in some cases post content runs into some encoding issues and doesn’t get exposed properly at the filesystem level. Another one that I’ve seen is that the comment count number doesn’t show up correctly. Both of these are issues in the Python code that I need to take the time to work out.

While not really a bug, there are also some patterns in this code that I’m not entirely happy with. I’m hoping with a bit more Python experience I’ll be able to simplify those.

Kick The Tires

The code is available at, please give a try and let me know what you think.

GPlusFS: Google+ Data as a Filesystem

With the basics of a Google+ API announced last week I started poking around. I liked that there was an option to simply sign up for an API key, allowing me to quickly try it out from the command line using cURL. The JSON formatted results were easy enough to understand.

That led to hooking up FUSE and Python to provide the ability to mount the people.get data as a filesystem. A bit more hacking and gplusfs was born. It is still basic, but has been working fine so far with my profile. This is all still read-only, since the Google+ API is read-only.

Here is what a directory listing of my profile looks like:

> ls -la *
-r-------- 1 root root   57 1970-01-01 00:00 aboutMe.txt
-r-------- 1 root root   12 1970-01-01 00:00 displayName.txt
-r-------- 1 root root    4 1970-01-01 00:00 gender.txt
-r-------- 1 root root   21 1970-01-01 00:00 id.txt
-r-------- 1 root root 5117 1970-01-01 00:00 image.jpg
-r-------- 1 root root   11 1970-01-01 00:00 kind.txt
-r-------- 1 root root   45 1970-01-01 00:00 url.txt

total 1
dr-------- 2 root root  0 1970-01-01 00:00 ./
dr-------- 2 root root  0 1970-01-01 00:00 ../
-r-------- 1 root root 10 1970-01-01 00:00 work.txt

total 1
dr-------- 2 root root  0 1970-01-01 00:00 ./
dr-------- 2 root root  0 1970-01-01 00:00 ../
-r-------- 1 root root 63 1970-01-01 00:00 json.txt
-r-------- 1 root root 45 1970-01-01 00:00 profile.txt

For JSON values that are strings I used the key name for the file, with an added .txt extension. The size of the file is determined by the length of the string. It works the way you’d expect:

> more displayName.txt 
Joseph Scott

For the profile image the API provides the URL of the image, gplusfs grabs a copy of the image and exposes that to the filesystem instead of the URL string.

The other special case is list values, which are exposed as directories. The JSON data provides multiple values for these, so breaking these out as separate directories with their own files seemed like the path of least surprise.

The source code is available at and configuration wise it just needs your Google+ user id and an API key. Give it a spin and let me know if you run into any issues with your profile data.