Joseph Scott

A gradually growing list of people have run into a very odd problem using XML-RPC methods in WordPress, where the left angle bracket ( < ) gets stripped. There's been a fair bit of discussion about this on ticket #7771. The bottom line: the behavior of the PHP XML extension when built against newer versions of libxml2 changed, such that left angle brackets get stripped when parsing XML.

There’s been some back and forth between libxml2 folks (email list) and the PHP folks (bug 45996), with no real solution for those using the tainted versions of libxml2. So what are your options if you’ve got this problem? Here’s two:

• Stick with older, known to work versions of libxml2. It’s been reported by others that libxml2 <= 2.6.32 work. I've personally only tested up to 2.6.30, which has been working fine for me.
• Build the PHP XML module against the expat parser instead of libxml2

Both of these options require some server admin abilities and know how, making them unrealistic options for many WordPress users. Undoubtedly many hosting services will role out these newer versions of libxml2 as part of their regular updates. This will leave some WordPress users with sudden errors that weren’t there before.

As this was spurred by a change in behavior by libxml2, I think the ideal solution would be to provide a backwards compatible mode that would restore the old parsing mechanism (you know, the one that doesn’t strip angle brackets). Short of that happening perhaps the XML extension for PHP will need to grow to work correctly with the new way that libxml2 works. Either way, I’d like to see PHP XML parsing work correctly again.

If you aren’t having any of these problems right now I recommend NOT upgrading libxml2 on your system until this has been sorted out.

WordPress.com just passed 5 million blogs:

5 Million Blogs

There are a number of stats that are publicly available for WordPress.com.

Today brings the official release of WordPress 2.7. I love what Michael Pick has been doing with the videos for the new WordPress releases:

There’s lots of information about the new UI and features in the announcement. Here’s what’s changed for XML-RPC when you compare the 2.6 branch to the new 2.7 code:

• New method: wp.getTags (ticket 7744)
• New comments API methods (ticket 7446):
  • wp.getComment
  • wp.getComments
  • wp.deleteComment
  • wp.editComment
  • wp.newComment
  • wp.getCommentStatusList
• Site ID fix for wp.getUsersBlogs (ticket 8255)
• New optional argument for wp.getPages - 4th parameter is now the maximum number of pages to return, which defaults to 10.
• metaWeblog.getPost now returns enclosure information (ticket 7772)
• Correct metaWeblog.getCategories to provide the actual comment description field (ticket 8494)

And the changes for AtomPub:

• Specify a status code of 401 when requiring authentication (ticket 7361)
• Work around for setups that do not basic HTTP basic authentication headers, like Apache when using PHP as a CGI (ticket 7361)

The new comments API methods are already getting some use. The next version of the WordPress iPhone App makes use of them for comment moderation. I also recently came across a WordPress comment moderation web app, which gives you way to manage WP comments outside of WP. Here’s a video demo.

If you have an interest in XML-RPC or AtomPub development in WordPress please join the wp-xmlrpc email list.

In late October Jeff Atwood wrote about The Problems With URLs, describing the problems of parsing out URLs in text and transforming them into links. Here’s a simple example:

My website is at http://josephscott.org/

Would be changed into:

My website is at <a href=’http://josephscott.org/’>http://josephscott.org/</a>

Sounds simple right? Once you start looking at what the valid character set is for URLs things get tricky. I won’t rehash all of items, go the The Problem With URLs post to see an example of some of the problems.

I knew that WordPress had a make_clickable function (in wp-includes/formatting.php) that did this exact thing. After testing this against some of the problems that Jeff points out it became clear that make_clickable() didn’t handle these edge cases. I made some rather crude tweaks to the WordPress code to fix some of these and opened ticket 8300 with my patches. Then filosofo came along and not only cleaned up my hacks, but reduced the amount of code needed in general. Major kudos to filosofo!

At this point it looks like we’ve got code to make make_clickable() work correctly with problem URLs. I’m going to wait until after WordPress 2.7 is released to push for getting this code committed since we’re trying to get 2.7 wrapped up.

I got thinking, this bit of code would be really handy to have as a stand alone library. So I pulled out the various pieces of code needed to make this work and put it together in a single PHP class: MakeItLink

class MakeItLink {
    protected function _link_www( $matches ) {
        $url = $matches[2];
        $url = MakeItLink::cleanURL( $url );
        if( empty( $url ) ) {
            return $matches[0];
        }

        return "{$matches[1]}<a href='{$url}'>{$url}</a>";
    }

    public function cleanURL( $url ) {
        if( $url == '' ) {
            return $url;
        }

        $url = preg_replace( "|[^a-z0-9-~+_.?#=!&;,/:%@$*'()x80-xff]|i", '', $url );
        $url = str_replace( array( "%0d", "%0a" ), '', $url );
        $url = str_replace( ";//", "://", $url );

        /* If the URL doesn't appear to contain a scheme, we
         * presume it needs http:// appended (unless a relative
         * link starting with / or a php file).
         */
        if(
            strpos( $url, ":" ) === false
            && substr( $url, 0, 1 ) != "/"
            && !preg_match( "|^[a-z0-9-]+?.php|i", $url )
        ) {
            $url = "http://{$url}";
        }

        // Replace ampersans and single quotes
        $url = preg_replace( "|&([^#])(?![a-z]{2,8};)|", "&#038;$1", $url );
        $url = str_replace( "'", "&#039;", $url );

        return $url;
    }

    public function transform( $text ) {
        $text = " {$text}";

        $text = preg_replace_callback(
            '#(?<=[\s>])(\()?([\w]+?://(?:[\w\\x80-\\xff\#$%&~/\-=?@\[\](+]|[.,;:](?![\s<])|(?(1)\)(?![\s<])|\)))*)#is',
            array( 'MakeItLink', '_link_www' ),
            $text
        );

        $text = preg_replace( '#(<a( [^>]+?>|>))<a [^>]+?>([^>]+?)</a></a>#i', "$1$3</a>", $text );
        $text = trim( $text );

        return $text;
    }
}

It’s very easy to use, just load up the text you want to search for link and call the transform method:

$text = MakeItLink::transform( $text );

All of this code came out of WordPress, which is licensed under the GPL, so consider the MakeItLink code GPL as well. If you’ve got some improvements let me know and make sure that it gets back into the original WordPress functions as well.

WordPress has an extensive set of actions and filters. All of the hooks allow you add/change/remove low level features via plugins and themes. Using this technique plugins can expose new XML-RPC methods for your WordPress blog.

In the xmlrpc.php file you’ll see a few lines that looks like:

        $this->methods = apply_filters('xmlrpc_methods', $this->methods);
        $this->IXR_Server($this->methods);

This allows plugins to filter the array of XML-RPC methods before the service starts. Adding a new function involves three parts: calling add_filter(), defining a function to manipulate the method list and a function that does the actual work behind the new method. I’ll go through a simple example, one step at a time.

add_filter( 'xmlrpc_methods', 'add_new_xmlrpc_methods' );

This tells WordPress to pass the xmlrpc_methods filter data to our function add_new_xmlrpc_methods.

function add_new_xmlrpc_methods( $methods ) {
    $methods['demo.multiplyTwoNumbers'] = 'multiply_two_numbers';
    return $methods;
}

This is our add_new_xmlrpc_methods function that does the filtering work. We’re adding a new key/value pair to the methods array. The key (demo.multiplyTwoNumbers) is the method name that is exposed via XML-RPC. The value (multiply_two_numbers) is the function that will do the work when that method is invoked.

function multiply_two_numbers( $args ) {
    $first  = (int) $args[0];
    $second = (int) $args[1];

    return $first * $second;
}

The final piece is the function that is called when our new XML-RPC method is invoked. The XML-RPC service will pass all of the arguments to your function as an array. After working with the provided parameters we return some value. The XML-RPC service will figure out what data type you are returning and encode it in appropriately way so that the client can understand it. This is means you don’t have to think about any of the XML-RPC level issues, just do the work and return the data.

Multiplying two numbers isn’t particularly interesting, so use your imagination. There are additional internal blog or WordPress information you could expose. For that matter you could run some of your data against third party APIs and then return the results.

To get you started here’s the entire plugin code:

<?php
/*
Plugin Name:    New XML-RPC Methods
Plugin URI:     http://josephscott.org/archives/2008/11/adding-xml-rpc-methods-to-wordpress
Description:    Demo how to add XML-RPC methods to WordPress.
Version:        0.1
Author:         Joseph Scott
Author URI:     http://josephscott.org/
 */

add_filter( 'xmlrpc_methods', 'add_new_xmlrpc_methods' );
function add_new_xmlrpc_methods( $methods ) {
    $methods['demo.multiplyTwoNumbers'] = 'multiply_two_numbers';
    return $methods;
}

function multiply_two_numbers( $args ) {
    $first  = (int) $args[0];
    $second = (int) $args[1];

    return $first * $second;
}

If you create a plugin to add new XML-RPC methods be sure to leave a comment letting us know what you came up with.

While the Automattic crew was in Colorado last week we were working on new features and tonight we’re opening a beta for the one of those - comment replies via email. This is a new capability that we’ve added to WordPress.com that allows you to reply to a comment simply by replying to the comment notification email that gets sent out to post authors when someone leaves a new comment. Here’s a quick video that shows how it works:

To start with we are calling for beta testers. If you want to get in on the beta leave a comment on the announcement post specifying what the URL of your WordPress.com blog is. We’ll be adding users to the beta until 7pm (Utah time) Friday 24 October 2008.

I have a personal attachment to this feature, Barry, Jon Fox (from IntenseDebate) and I worked on this while in Colorado. So I’ll be keeping on eye on how the beta testing goes.