WordPress 4.2.3, Shortcodes, and the Future

The WordPress Core team is hard at work trying to figure out how to move forward with shortcodes in the system. They posted an initial draft of the shortcode roadmap earlier this week, and it was met with a lot of discussion (some very productive, some not so much). They went back to the drawing board, and proposed an updated draft of the roadmap yesterday. No doubt, this discussion will take a while to be completely fleshed out, but I, for one, am extremely happy to see them taking this seriously. It is imperative, for the future of WordPress, that everyone gets involved in the discussion, and continues to move the process in a positive direction.

Background

This change, and the ensuing discussion about ignoring the breakages because they were outside of the original intentions, directly countered the spirit of innovation that runs so deep in WordPress.

A few weeks ago, a new, urgent security update was released for WordPress. The main tag of the release was 4.2.3, but the same security patches were applied to 4.1.7 and further back.

This release broke a lot of sites.

The problem arose in the way shortcodes were being parsed and handled in WordPress. For time immemorial in WordPress, shortcodes have pretty much been processed unchecked. They acted as placeholders for dynamic content everywhere in the system. For the most part, they were used to output blocks of dynamic HTML content in-place of the shortcode keyword, whether that meant putting together and outputting a photo gallery, a slideshow, embedded media, content from other places, etc.

However, many plugin developers found other uses for shortcodes over the years. In some cases, the shortcodes were being used to actually output HTML attribute values. In some instances, the shortcode was used to output a URL as the href attribute of a link or the src attribute of an img tag; in other cases, it was used to add extra CSS classes to an element, or even to attempt to add inline styles to an element.

When 4.2.3 was released, many of the implementations referenced in the paragraph above stopped working. The problem, as it turned out, was that the new security updates caused WordPress to attempt to validate and sanitize HTML attributes before shortcodes were processed. If a shortcode didn’t include any attributes/arguments, it would pass through just fine. However, if that shortcode did include some attributes, and those attributes were quoted in the same style as the HTML element’s attributes, the parser gave up and sent the full, unprocessed HTML element back for output.

As some examples:

1. `<a href=”[[home-url]]” title=”Visit the home page of [[site-name]]”>`
2. `<a href=”[[home-url scheme=”https”]]” title=”Visit the home page of [[site-name sanitize=”true”]]”>`
3. `<a href=”/” style=”background-image: url(‘[[custom-bg-image]]’)”>`
4. `<a href=”/” style=”background-image: url(‘[[custom-image context=”primary-background”]]’)”>`

All four of those examples worked fine prior to the 4.2.3 security patches. The resulting HTML output on the screen would look something like:

1. `<a href=”http://www.example.com/” title=”Visit the home page of My Example Site”>`
2. `<a href=”https://www.example.com/” title=”Visit the home page of My Example Site”>`
3. `<a href=”/” style=”background-image: url(‘/wp-content/uploads/2015/09/sample-background.png’)”>`
4. `<a href=”/” style=”background-image: url(‘/wp-content/uploads/2015/09/primary-background-image.png’)”>`

After the security release, however, only examples 1 & 3 continued to work. Examples 2 and 4 would return the unprocessed shortcodes inside of the HTML instead.

The Hour of My Discontent

I first reported the “bug”, along with some technical details of the problem. After some initial discussion, the bug ticket was closed, and some core contributors indicated that shortcodes were never intended to be used the way they were being used, and so the broken implementations were not going to be fixed. There was some discussion back and forth between core team members, and it was finally left open for a while.

Needless to say, the situation started to get my blood boiling.

I wasn’t so upset that shortcodes themselves stopped working. While that was a problem, and broke a lot of things across a lot of sites I’m responsible for managing, it was easily corrected in many cases (partially because most of the problems encountered on my sites were related to the Views plugin, and they released some work-arounds to the issue pretty quickly).

The thing that upset me much more than the specific issue of shortcodes was the fact that, somewhere along the line, someone decided that there were a limited subset of feature implementations that were going to be supported in core, and if anything outside of that subset of features broke, it wasn’t a problem.

That message flew in the face of two aspects of WordPress I hold most dear:

1. Backwards-compatibility – I’ve said this before, and I’ll say it again, but one of the reasons WP has been able to make it as far as it has is because things (generally) don’t break when you perform an update. Throughout the history of the project, the core team has always done an extremely admirable job of making sure of that. If things are going to break each time WordPress is updated, or if major code changes are going to be required, then WordPress loses much of the advantage it has over other systems.
2. Encouragement of Innovation – the various WordPress marketing folks and evangelists are always praising the WordPress community for its innovation. They are constantly calling for examples and case studies showing WordPress doing things they never imagined it could. We, as WordPress users, are consistently asked, implicitly and explicitly, to push WordPress to its breaking-point, and to figure out what can be done with the system. This change, and the ensuing discussion about ignoring the breakages because they were outside of the original intentions, directly countered the spirit of innovation that runs so deep in WordPress.

A Glimmer of Hope

The new roadmap for shortcodes still has a long way to go, but it’s a glimmer of hope in this situation. One core contributor, Chris Christoff, replied to a comment on the first draft proposal with the following, beautiful statement:

This is partially about efficiency. This has a lot to do with security, and basically making 4.2.3 impossible as possible to re-occur

That tells me that the core team has been paying attention, not only to the fact that there were many people upset about the specific breakage that occurred in 4.2.3, but also that they’re focusing on a future where WordPress will hopefully not break any existing implementations, regardless of whether those implementations were part of the original vision for WordPress or not.

Contribute

If you have any invested interest in WordPress (which I’m assuming you do, if you’re reading this post), please head on over to Make WordPress and contribute your opinions, your concerns, and your suggestions for improvement. Don’t limit your participation to the discussion on shortcodes, either. Please, for the continued success of WordPress and your own endeavors, contribute to as many discussions as you can. All voices should be heard when it comes to the future of WordPress.