WordPress & libxml2 Episode IV: A New Plugin

Despite the libxml2 problems having been solved by the release of PHP 5.2.9 and libxml2 2.7.3 there are still hosts out there who refuse to update their known to be broken setup. This really bummed me out, I didn’t want to include a hack to deal with this in core WordPress.

Just as I was ready to throw in the towel and look at getting a work around into core WordPress Peter Westwood suggested a method that would allow this “fix” to work in a plugin. His idea was spot on and I wrote a small WordPress plugin, LibXML2 Fix, the next day and tested it on a WordPress install with this problem. It fixed the stripped brackets problem for XML-RPC requests, so I’ve submitted it to the plugin directory and it’s now available at http://wordpress.org/extend/plugins/libxml2-fix/.

To be clear, the best way to fix this is to upgrade to PHP 5.2.9+ and libxml2 2.7.3+. But if you are stuck in a hosting situation where that’s not possible then the LibXML2 Fixed plugin will fix XML-RPC requests so that brackets don’t get stripped.

I’ve closed ticket 7771 and will be pointing people to this plugin if they can’t upgrade PHP and libxml2.

Conclusion of libxml2 Issues – Use PHP 5.2.9 & libxml2 2.7.3

17 Mar 2009 @ 3:05pm : WordPress & libxml2 Episode IV: A New Plugin

With the release of PHP 5.2.9 all of the pieces are available to solve the XML bracket stripping puzzle. To make it perfectly clear, you need to have BOTH libxml2 version 2.7.3 and PHP version 5.2.9 in order to make this work. Mixing either one with older versions will not work. When testing this also confirm that your build of PHP 5.2.9 is using the correct version of libxml2 and not some older version (you can easily check this using phpinfo( ) ).

I went back to my test setup and built PHP 5.2.9 with libxml2 2.7.3 and added new posts to a WordPress blog via XML-RPC with lots of tags in the post body. They all came through intact. Not a single bracket missing.

If you are on a host that is using known broken version combinations of PHP & libxml2 now you can tell them exactly which versions do work. Feel free to point them this and previous posts (Problems With libxml2 For WordPress XML-RPC Users and Update On libxml2 Issues) if they want some history and context of the problem.

WordPress 2.7.1 Released – XML-RPC and AtomPub Changes

Ryan just tagged the 2.7.1 release of WordPress. This being a maintenance release there are only a few changes to XML-RPC:

  • Fixed bugs in wp.newComment (ticket 8672)
  • Fix break of metaWeblog.getCategories that was introduced in 2.7 (ticket 8494)
  • Added the new categoryDescription field to metaWeblog.getCategories in light of the problems introduced in 2.7 as part of ticket 8494

And there were no changes to AtomPub in 2.7.1.

As usual, if you have an interest in XML-RPC and AtomPub APIs in WordPress please come join the wp-xmlrpc email list.

Update On libxml2 Issues

4 Mar 2009 @ 2:38pm : Conclusion of libxml2 Issues – Use PHP 5.2.9 & libxml2 2.7.3

17 Mar 2009 @ 3:05pm : WordPress & libxml2 Episode IV: A New Plugin

At the end of December I detailed problems people were seeing with WordPress, XML-RPC and libxml2. I’ve got good news, both PHP and libxml2 have been updated to fix the issue. You can send your many thanks to Rob Richards who addressed this a couple of weeks ago with updates to ext/xml (PHP) and libxml2.

You’ll need version 2.7.3 or higher of libxml2 and 5.2.9 (not released yet) of PHP. I’ve tried this out on a system that had libxml2 2.7.1 with known problems and then upgraded it to 2.7.3 and a nightly build of PHP. I’m not sure if we’ll see 5.3 or 5.2.9 as the next PHP release, so I tested snapshots from both branches with libxml2 2.7.3 and confirmed that they both work.

After this all started there were a few patches making the rounds that basically changed the encoding for the HTML entities that were having problems. I’ve done some light testing of those for the XML-RPC case and found that they do appear to work. I’m not sure yet about adding that to WordPress, especially now that we’ve got viable fixes at the PHP and libxml2 level. I think if we see a PHP release soon that includes this fix then it’s best to encourage people to upgrade to address the problem.

Problems With libxml2 For WordPress XML-RPC Users

3 Feb 2009 @ 2:00pm : Update on libxml2 issues

4 Mar 2009 @ 2:38pm : Conclusion of libxml2 Issues – Use PHP 5.2.9 & libxml2 2.7.3

17 Mar 2009 @ 3:05pm : WordPress & libxml2 Episode IV: A New Plugin

A gradually growing list of people have run into a very odd problem using XML-RPC methods in WordPress, where the left angle bracket ( < ) gets stripped. There's been a fair bit of discussion about this on ticket #7771. The bottom line: the behavior of the PHP XML extension when built against newer versions of libxml2 changed, such that left angle brackets get stripped when parsing XML.

There’s been some back and forth between libxml2 folks (email list) and the PHP folks (bug 45996), with no real solution for those using the tainted versions of libxml2. So what are your options if you’ve got this problem? Here’s two:

  • Stick with older, known to work versions of libxml2. It’s been reported by others that libxml2 <= 2.6.32 work. I've personally only tested up to 2.6.30, which has been working fine for me.
  • Build the PHP XML module against the expat parser instead of libxml2

Both of these options require some server admin abilities and know how, making them unrealistic options for many WordPress users. Undoubtedly many hosting services will role out these newer versions of libxml2 as part of their regular updates. This will leave some WordPress users with sudden errors that weren’t there before.

As this was spurred by a change in behavior by libxml2, I think the ideal solution would be to provide a backwards compatible mode that would restore the old parsing mechanism (you know, the one that doesn’t strip angle brackets). Short of that happening perhaps the XML extension for PHP will need to grow to work correctly with the new way that libxml2 works. Either way, I’d like to see PHP XML parsing work correctly again.

If you aren’t having any of these problems right now I recommend NOT upgrading libxml2 on your system until this has been sorted out.

WordPress 2.7 Released – XML-RPC and AtomPub Changes

Today brings the official release of WordPress 2.7. I love what Michael Pick has been doing with the videos for the new WordPress releases:

There’s lots of information about the new UI and features in the announcement. Here’s what’s changed for XML-RPC when you compare the 2.6 branch to the new 2.7 code:

  • New method: wp.getTags (ticket 7744)
  • New comments API methods (ticket 7446):
  • Site ID fix for wp.getUsersBlogs (ticket 8255)
  • New optional argument for wp.getPages – 4th parameter is now the maximum number of pages to return, which defaults to 10.
  • metaWeblog.getPost now returns enclosure information (ticket 7772)
  • Correct metaWeblog.getCategories to provide the actual comment description field (ticket 8494)

And the changes for AtomPub:

  • Specify a status code of 401 when requiring authentication (ticket 7361)
  • Work around for setups that do not basic HTTP basic authentication headers, like Apache when using PHP as a CGI (ticket 7361)

The new comments API methods are already getting some use. The next version of the WordPress iPhone App makes use of them for comment moderation. I also recently came across a WordPress comment moderation web app, which gives you way to manage WP comments outside of WP. Here’s a video demo.

If you have an interest in XML-RPC or AtomPub development in WordPress please join the wp-xmlrpc email list.

Adding XML-RPC Methods to WordPress

WordPress has an extensive set of actions and filters. All of the hooks allow you add/change/remove low level features via plugins and themes. Using this technique plugins can expose new XML-RPC methods for your WordPress blog.

In the xmlrpc.php file you’ll see a few lines that looks like:

        $this->methods = apply_filters('xmlrpc_methods', $this->methods);

This allows plugins to filter the array of XML-RPC methods before the service starts. Adding a new function involves three parts: calling add_filter(), defining a function to manipulate the method list and a function that does the actual work behind the new method. I’ll go through a simple example, one step at a time.

add_filter( 'xmlrpc_methods', 'add_new_xmlrpc_methods' );

This tells WordPress to pass the xmlrpc_methods filter data to our function add_new_xmlrpc_methods.

function add_new_xmlrpc_methods( $methods ) {
    $methods['demo.multiplyTwoNumbers'] = 'multiply_two_numbers';
    return $methods;

This is our add_new_xmlrpc_methods function that does the filtering work. We’re adding a new key/value pair to the methods array. The key (demo.multiplyTwoNumbers) is the method name that is exposed via XML-RPC. The value (multiply_two_numbers) is the function that will do the work when that method is invoked.

function multiply_two_numbers( $args ) {
    $first  = (int) $args[0];
    $second = (int) $args[1];

    return $first * $second;

The final piece is the function that is called when our new XML-RPC method is invoked. The XML-RPC service will pass all of the arguments to your function as an array. After working with the provided parameters we return some value. The XML-RPC service will figure out what data type you are returning and encode it in appropriately way so that the client can understand it. This is means you don’t have to think about any of the XML-RPC level issues, just do the work and return the data.

Multiplying two numbers isn’t particularly interesting, so use your imagination. There are additional internal blog or WordPress information you could expose. For that matter you could run some of your data against third party APIs and then return the results.

To get you started here’s the entire plugin code:

Plugin Name:    New XML-RPC Methods
Plugin URI:     http://josephscott.org/archives/2008/11/adding-xml-rpc-methods-to-wordpress
Description:    Demo how to add XML-RPC methods to WordPress.
Version:        0.1
Author:         Joseph Scott
Author URI:     http://josephscott.org/

add_filter( 'xmlrpc_methods', 'add_new_xmlrpc_methods' );
function add_new_xmlrpc_methods( $methods ) {
    $methods['demo.multiplyTwoNumbers'] = 'multiply_two_numbers';
    return $methods;

function multiply_two_numbers( $args ) {
    $first  = (int) $args[0];
    $second = (int) $args[1];

    return $first * $second;

If you create a plugin to add new XML-RPC methods be sure to leave a comment letting us know what you came up with.

Windows Live Writer and WordPress

Windows Live Writer Beta 2 is out, now with more WordPress features. Most of the new WordPress related features focus on page management. There is also better category and author support. And of course bug fixes along the way.

Adding support for these features drove some of recent the XML-RPC work on WordPress, which I’ve really enjoyed doing.

API Only Services?

Along with most folks, I’m thrilled when I see APIs made available for cool services. It seems to me that this what Tim O’Reilly means when using the term Web 2.0. With open and usable APIs others are free to create entirely new “products” by mix and matching APIs from various sources. This is great because it allows so many people to be involved in the process, not just the companies who came up with the original services. The more people involved, the bigger the pool of imagination from which new products and services can emerge.

For the most part, a service is made available (like Google Maps), reaches some level of popularity and then, sometimes much later, an API is made available for others to use. This isn’t always the case, Yahoo!’s My Web 2.0 was announced and then made some initial APIs available on day one. What I haven’t really seen much of is APIs being put out first and letting services grow around them.

So are we going to see API only services in the future? I’m not sure, but I’d argue that API only/focused services are the way to go in the future. Just like job specialization allows companies to scale in ways not possible before, I believe that API specialization would allow the Internet to scale in ways we’ve yet to see. Let’s take a basic example, spell check. Virtually anytime you use a textarea on a web page it would be helpful to offer a spell check feature. When Gmail offered an AJAXified spell check feature we were all impressed and wanted to use it in other places. But who wants to recreate the wheel for spell check every where that it might be handy to have? In part the popularity of AJAX has solved some of this problem, by providing frameworks that make adding such features to web pages easier.

Going down one more layer, what actually does the spell checking? Most languages have hooks into Aspell that allows them to does this. But does everyone who wants to use spell check on their website need to have Aspell installed and working with their language of choice? Why isn’t there a central service providing a spell check API? Why can’t they simply make use of a publicly available spell check API (via REST/SOAP/XML-RPC, etc)? This wouldn’t have to be limited to web pages either, any application could simply talk to the spell check server and get the information it needed.

There’s no reason to only have one spell check server, competition is a good thing and if one was offline for awhile then having others to fall back on would be helpful. For that matter there’s no reason why you couldn’t have a local spell check server just for your company/office, as long at it made use of the same APIs so that client applications would be able to use as a drop in replacement. Imagine being able to enter a URL in Microsoft Word in a settings area and have it make use of your company spell check server, how cool would that be? For better performance caching could be done either at some local network server and/or on the client application. In many ways this is the Web/DNS model all over again, but for APIs.

If we really are going to fully embrace the capabilities that the Internet makes possible I think that specialization at the API level is where things will go.