Joseph Scott

Silverlight Only at Utah Code Camp

Joseph Scott — Tue, 09 Mar 2010 06:12:34 +0000

I got an email announcing an upcoming Utah Code Camp, scheduled for the last Saturday of this month. So I headed over to utahcodecamp.com and got this:

A giant black screen with small icon at the top center that said ‘Install Microsoft Silverlight’. Obviously I don’t have Silverlight installed and don’t really have an interest in installing it any time soon. Not one part of the site worked unless Silverlight was installed.

Through out the 1990s and early 2000s I worked hard to convince every designer who had discovered Flash, that no, they really don’t want to start creating every site in all Flash. It is disappointing to see this trend starting over again with Silverlight.

mod_ping, Maybe I Should Have Called It PubSubHubbub

Joseph Scott — Fri, 05 Mar 2010 06:03:01 +0000

ReadWriteWeb has an article about Google potentially using PuSH to get updates into the search index. I knew this idea sounded familiar so I went hunting through some of my old posts and found – Apache Module Idea mod_ping from 2004.

Back then I had thought a lot about searching blog feeds. There were some services that offered blog feed searching but they were all pretty bad. I wrote a review of the situation at the time in Why Hasn’t Anyone Figured Out How To Do Feed Searches?. Keep in mind that Google’s blog search feature wouldn’t be announced for another year.

It seemed odd to me that blog feed search would be so bad given how strongly the blogging software community had embraced the idea of pinging updates. This led me to the idea of some sort of mod_ping for Apache that would do similar pings for any type of website updates, it didn’t have to be limited to just blogs. Obviously I never took this idea any where (besides writing that post) and to be honest I hadn’t really revisited the idea much since. Search engines took a different route for update frequency, with features like sitemaps in 2006.

Fast forward to 2010 and we’ve got discussion of Google potentially using PubSubHubbub (PuSH) to subscribe to updates from every single web site out there. This brings up an interesting question though. Since PuSH focuses on feed formats (RSS & Atom) for pings, what format will pings from sites that don’t have feeds look like? Will the ping just contain the entire HTML output of the updated page? What about a diff (unified format of course!) between the new HTML and the previous HTML for a given page?

Folks like Brett Slatkin have been thinking about this sort of thing on a deeper level than I ever did, so I’m curious to see where this goes.

PuSHPress: A PubSubHubbub Plugin For WordPress

Joseph Scott — Wed, 03 Mar 2010 16:49:50 +0000

PubSubHubbub, or PuSH, is now supported on all WordPress.com blogs. For those running sites using the WordPress.org software the PuSHPress plugin is available to do the same thing. The big difference in this plugin compared to the other PuSH WordPress plugins is that it includes a PuSH hub built in.

A little bit of PubSubHubbub background

Like rssCloud, PuSH adds a line to your feed to let clients know where they can send PuSH subscription requests. In the RSS2 feed this looks like:

and in Atom looks like:

The href attribute contains the URL that subscribers can send requests to. Details on what that request looks like are in the PubSubHubbub Spec. The short version, it’s a simple HTTP POST call with some specific field names.

Here’s an example PHP subscription request asking that updates to http://example.com/feed/ be sent to http://example.com/push/ via the hub at http://example.com/?pushpress=hub

$curl = curl_init( );

curl_setopt( $curl, CURLOPT_URL, 'http://example.com/?pushpress=hub' );
curl_setopt( $curl, CURLOPT_POSTFIELDS, 'hub.callback=http://example.com/push/&hub.mode=subscribe&hub.secret=not-telling-you&hub.verify=sync&hub.topic=http://example.com/feed/&hub.verify_token=133t-$7r1n9' );
curl_setopt( $curl, CURLOPT_VERBOSE, 1 );
curl_setopt( $curl, CURLOPT_POST, 1 );

curl_exec( $curl );
print_r( curl_getinfo( $curl ) );
curl_close( $curl );

The hub at example.com will then send new content (in feed format) right after it’s published to URL provided in hub.callback.

Why the PuSHPress plugin?

I love that there are large, open PuSH hubs available for anyone to use. There’s a list at http://code.google.com/p/pubsubhubbub/wiki/Hubs. In PuSH since most of the hard work (figuring out what’s new in a feed and sending that out to subscribers) is done via the hub this is very handy and really great for testing.

I also think there’s a role for a readily available simple hub that anyone can put up and use, in this case built on top of WordPress. And by leveraging WordPress the hub part can be greatly simplified. The real bonus for users is choice, they can relay the updates through 3rd party hubs, or use the built in hub in PuSHPress.

It’s worth noting that PuSHPress has a WordPress filter on the array of hubs; pushpress_hubs. This allows other plugins to easily add to or replace the hubs mentioned in the feeds by PuSHPress.

To help keep things simple and limit potential abuse the PuSHPress plugin will only allow subscriptions for the RSS2 and Atom feeds of the WordPress blog that it is installed on.

My thanks to Brett Slatkin for helping test PuSHPress. His tests revealed a few bugs that were quickly addressed.

How fast?

When talking about these realtime-ish update features this question often comes up. Fair enough, I’d want to know too :-)

The PuSHPress plugin schedules pings to go out right away via the wp_schedule_single_event function in WordPress. I suspect for most people this will be quite fast. If your server is under tremendous load and really slow, then this will be really slow too, just like the rest of the server ;-)

On WordPress.com the details are a bit different. Instead of scheduling pings to be sent out with wp_schedule_single_event it adds them to our asynchronous jobs system ( more info available at http://code.trac.wordpress.org/ ). The jobs system is setup to do exactly these kinds of tasks really, really fast. Don’t be surprised if the ping shows up before you get a chance to finish the first Mississippi.

The future

So where is all this going? Who knows, but it is fun to watch! :-)

133t Google H4×0r

Joseph Scott — Wed, 03 Mar 2010 04:41:22 +0000

Wandered across the 133t Google translation today. Wait for the rest of the page to fade in for the full effect. Related posts:Google Translate Bookmarklet Google SEO Searching For ‘Search Engine’ On Google Related posts:

Thinking Clearly About Performance

Joseph Scott — Wed, 24 Feb 2010 19:05:16 +0000

Cary Millsap has a great paper out – Thinking Clearly About Performance. It has reminded me how critical details can be in measuring performance, throughput, response time, efficiency, skew and load.

The beauty of this paper isn’t that it has some magical new technique, quite the opposite actually. The methods talked about are solid, fundamental approaches that can be used to methodically figure out what’s going in your system.

It’s only 13 pages long and is a pleasant, brisk read. I recommend it for anyone doing software development, database administration, or system administration.

Database Indexes on Domain Names

Joseph Scott — Wed, 24 Feb 2010 04:20:43 +0000

As part of a separate conversation Matt suggested something I had not given much thought to before. Say you have a database column that is used to store domain names. Something as simple as `domain` varchar(255) NOT NULL.

Of course you’ll want to search all of that data. Perhaps you want to lookup all of the sub-domains for example.com. That query might look like

SELECT * FROM table_name WHERE domain LIKE '%.example.com'

Having a wildcard ( % ) at the front of a LIKE comparison generally results in poor performance. Depending on the database you may find that either it can’t make use of an index on that column, or it must do a full index scan in order to find the right entry. To make better use of an index the wildcard should be at the end of the string, which we could do if we reverse the domain name ordering:

SELECT * FROM table_name WHERE domain LIKE 'com.example.%'

So that’s what we do. This of course brings up the question of how to correctly reverse the domain name parts. In PHP this can be done with a combination of explode(), array_reverse(), and implode():

$domain = 'joseph.example.com';
$reverse_domain = implode( '.', array_reverse( explode( '.', $domain ) ) );

That turns joseph.example.com into com.example.joseph. The reversed domain name gets stored in the database and that allows us to use the more efficient database query.

PHP Helpers: make_slug

Joseph Scott — Sat, 20 Feb 2010 00:19:16 +0000

New function, make_slug -

if ( !function_exists( 'make_slug' ) ) {
    function make_slug( $str ) {
        $url_str = strtolower( trim( $str ) );

        $url_str = preg_replace( '/[\s_]+/', '-', $url_str );
        $url_str = preg_replace( '/-{2,}/', '-', $url_str );
        $url_str = preg_replace( '/[^a-z0-9-]/', '', $url_str );

        return $url_str;
    }
}

This takes a string and returns a slug that is more URL friendly. For example “Do This_Today -- Please” would become “do-this-today-please“.

The approach is pretty simple:

lower case everything (line 3)
replace space and underscore with dash (line 5)
replace two or more dashes with a single dash (line 6)
remove any other characters that aren’t a-z, 0-9 or a dash (line 7)

PHP Helpers: redirect_url

Joseph Scott — Fri, 12 Feb 2010 23:55:35 +0000

New function: redirect_url

if ( !function_exists( 'redirect_url' ) ) {
    function redirect_url( $url, $status = 302 ) {
        header( "Location: {$url}", TRUE, $status );
        exit;
    }
}

A special case wrapper around the header function to do URL redirection. I specifically added the exit call inside the function so that redirect happens immediately. That way I don’t have a chance to forget about it later on :-)

iPhone OS Feature Request – Longer ‘Fetch New Data’ Intervals

Joseph Scott — Wed, 10 Feb 2010 19:30:11 +0000

I’ve setup my iPhone to sync my calendar and contacts with my Google account. The sync process itself works fine, my complaint is with the ‘Fetch New Data’ options.

I don’t use push for this, I’m happy to get more battery life at the expense of not having immediate updates. That leaves me with these interval options:

Every 15 Minutes
Every 30 Minutes
Hourly
Manually

I really don’t need hourly updates for my calendar and contacts, so I’m left with ‘Manually’. What I’d really like is longer intervals to select from, more in the range of 8, 12 and 24 hours. Syncing in the background a few times day would be plenty for me.