Drupal as Open Architecture

By Kurt Cagle
August 5, 2008 | Comments: 16
drupal.org.png

I have a confession to make - after close to a decade covering XML, I have something of a new love ... and the name of that love is Drupal. Drupal's become one of those interesting hobbies that is rapidly becoming both a profession and a passion. It wasn't supposed to happen this way ... by rights, I should be deeply in the world of Ruby on Rails right now, or learning the latest deep programming secrets of Python, but somewhere along the line I realized one of those ugly little fundamental truths that good programmers should never actually learn - that at some point, recreating the wheel yet again begins to lose its luster, and, indeed, become rather ... well ... dull.

The fundamental issue I've faced with Drupal is one that was actually brought up in a talk about successful Open Source architectures by Roy Fielding at OSCON 2008, entitled Open Architectures at REST. Fielding, of course, knows a great deal about both REST and architectures, as he was one of the key architects for the Apache Web Server and the person whose doctoral thesis made REST something beyond what you do when you go to sleep.

Fielding's talk discussed a number of characteristics that most successful open source projects have in common, what he termed Open Architectures. Such architectures reflect the model of decentralized software evolution, in which the role of the core architecture is to act as a platform on which the community can effectively build functionality specific to their needs. In this model, the role of the project team is to provide continuity of that platform and to give to the community the tools to build their own subordinate modules (extensions), as well as providing the messaging architecture that makes it possible for these extensions to play nicely with one another.

Fielding cited a number of examples of such open architectures, including Apache itself (with all of its mod_foo extensions), Firefox and its myriad XPI extensions and the Eclipse IDE with the thousands of plugins that have made Eclipse one of the most widely disseminated IDE on the planet. In each of these cases, the role of the architectures was to insure that the base platform was capable of supporting the extensions, often at multiple levels of abstraction, and that changes in that foundational infrastructure could be made without completely wiping out the community-developed plugins.

To this list I would also hazard to add Drupal (http://www.drupal.org) - which is one of the reasons that I've become so bullish about the software. For those not familiar with Drupal, it started out as a PHP-based bulletin board software in 2000, but the idea of making it extensible was on the radar even then.

The core concept of drupal is simple - you can represent a "content type" as a bundle of information including a title, a body, and links to that body's type, author and publication metadata. This bundle of information is known as a node. A general theme (made up of one or more CSS pages and one or more PHP-based templates) let you establish the presentation aspect of the node.

One significant aspect of such nodes is that each node can be shown as a RESTful URI. For instance, all nodes can be represented in the form http://www.myserver.com?q=node/n, where n is the node identifier number, or nid. In some servers (and with the right support on the server) you can even dispense with the query string notation altogether, so that the node would be given as http://www.myserver.com/node/n.

Additionally, however, that same node can have one or more aliases. For instance, suppose that the node in question was an administration page that showed the content listings of all of the nodes. This page might (and in fact in Drupal does) have the alias admin/content/node. Internally, Drupal maintains a table of such aliases to map to the underlying node.

However, Drupal is also very RESTlike in that you can even extend these aliases. For instance, the alias blog identifies the page that lists all of the blog posts on the system (for those Drupal instances where blogs are enabled). However, if you add the author id after this, such as blogs/4, then you can get the list of blogs for just the author with the id of 4.

What makes this so powerful is that users to the system can create these aliases just as readily as the site creator (who is in fact just a specialized superuser). It also means that extensions to drupal can take advantage of this to add new functionality without getting in the way of existing functionality.For instance, you can create your own new content types - such as an article type or even something wacky like a game character type with a name such as game_character, then adding a new game character becomes as simple for the user as going to a URL called node/add/game_character.

The Drupal community started out in blogging, but just as the Eclipse space exploded because of easy to develop user extensions, so too did the Drupal community begin to go ever farther afield because of user created modules. The Drupal core development team stayed abreast of those modules, and when a module emerged that became widely adopted, would take the module in, clean it up so that it fit as the most minimally invasive piece of functionality possible, and would then add it to the core.

Periodically, revisions would also be made to the core itself in the name of providing better architectural support, but this idea of minimal invasiveness applied there too. Backwards compatibility was maintained, but only within major levels - i.e., a plugin that worked on version 5.2 should be able to work on version 5.14, but not on 4.7. When a major version upgrade occurrs (always transparently and with plenty of time in beta), this forms the floor for a new generation of modules, some of which are upgrades themselves, some of which are new.

This in turn has had some interesting ramifications. By splitting the work in this fashion, the upgrade process no longer becomes a necessity for people - if you've built a stable site, then you can count on occasional security updates on the older branch, though over time the number of new modules for that branch decline to zero. However, this is a process that occurs relatively slowly, and in many cases similar functionality will be enabled in modules for two or three different major releases at the same time.

Thus, at the time of this writing, most module development has stopped on the 4.7 branch, the 5.x branch is still quite active, the recently published 6.x branch is now stocking up on new modules, and the 7.x branch is in beta development.  In many ways this is the essence of open source agile development - there is no true "development" phase, only a continuous (major or minor) maintainence phase.

What makes all this effort so worthwhile is that Drupal has now reached a point where you can create a sophisticated web portal without having to know how to program precisely because of this modularity. With Drupal, I can use the Content Construction Kit (CCK) in order to build new content types, can create forums and community blogs, can use blocks and panels in order to position content in specific areas on the page (and establish parameters on those pages), can work with views to create the relevant database queries that let me create a page of gallery images or videos with specific keywords or RSS and Atom feeds, can work with taxonomies that let me build tag clouds and dynamic navigation, and so on ... and I don't need to write a single line of PHP or SQL code to make any of it happen.

I can write such code if I want to - the extensibility mechanisms for Drupal are well documented and are generally not that hard to work with - but the importance here is that I don't have to write that code. Indeed, for all that I thoroughly enjoy working with Ruby, one thing that's rubbed at me for a while is that Ruby on Rails is still based upon this paradigm that you have to write code in order to build a site, which means that Ruby will always be of use only to those people who can write Ruby code in the first place.

With Drupal, that's no longer a requirement, and it means that people can get Drupal sites up and running quickly without needing to understand the first thing about programming - and if they can't find a module that does what they need (and with nearly 4000 such modules available just on the Drupal site alone this is becoming less and less of a problem) then they can find a programmer that will create just that functionality without having to rebuild the entire site. It's one of the reasons why Drupal is beginning to become the de facto environment for smaller news organizations and PR departments.

That doesn't necessarily mean that Drupal is either easy to use or doesn't come at a price. There are a number of fairly radical new concepts that Drupal uses that take a certain amount of time playing with it to fully understand. Moreover, while there are fairly strict guidelines for making modules, there are modules that get published that don't necessarily play nice with your underlying data, and debugging Drupal when something doesn't work can be a challenge in its own right.

It can also become instable (especially the 4.x branch) and tends to consume a fairly inordinate amount of memory once you start piling on the modules. This tends to be the bane of all open source community projects - the flexibility inherent in extensibility comes at the cost of module overhead and difficulty in the kind of optimization that dedicated systems can employ.

Yet if its a choice between optimized dedicated code and modular community code, in the long term, the latter alternative will always win. At least for the foreseeable future, the cost of memory and hard drive space will continue to fall, the move towards multiple parallel processors will increase the overall throughout of such applications and network latency will continue to drop as broadband becomes the rule rather than the exception.

This means that in general flexibility trumps efficiency. When a particular module becomes critical to a large number of people within the community, it is also usually a candidate for the core, where the component is optimized to better work within the core architecture. Indeed, in some cases, such as the Flock browser (http://www.flock.com) or the Songbird media player (http://www.getsongbird.com), it was the core architecture itself that was "spun-off" and optimized for community building or media playing compared to the original Firefox focus on web browsing.  That both of the spin-offs still retain (and actively court) community module development attests to how effective this model can be even when the core is changed.

As to Drupal, it has effectively become to web portals what Eclipse is to application development, and has the potential to significantly challenge Microsoft's Sharepoint (http://www.microsoft.com/Sharepoint/default.mspx) or similar commercial portal applications in that space. It will also be a topic that O'Reilly Media will be following closely from here on it.

Kurt Cagle is Online Editor for O'Reilly Media, focusing on web services, web applications, XML, publishing systems and so forth. He lives in Victoria, BC, where he hangs out with his daughter and the mastodon at the Royal BC Museum.


You might also be interested in:


16 Comments

I like drupal too, it is a good product, just like what you said that it is my new love.

What makes this so powerful is that users to the system can create these aliases just as readily as the site creator (who is in fact just a specialized superuser). It also means that extensions to drupal can take advantage of this to add new functionality without getting in the way of existing functionality.For instance, you can create your own new content types - such as an article type or even something wacky like a game character type with a name such as game_character, then adding a new game character becomes as simple for the user as going to a URL called node/add/game_character.(If you want more knowledge about creation of a website you can look at kswchina,there are some knowledge about it but chinaese.It's a good learning Chinese place too! )

The Drupal community started out in blogging, but just as the Eclipse space exploded because of easy to develop user extensions, so too did the Drupal community begin to go ever farther afield because of user created modules. The Drupal core development team stayed abreast of those modules, and when a module emerged that became widely adopted, would take the module in, clean it up so that it fit as the most minimally invasive piece of functionality possible, and would then add it to the core.

What you said is just what ilike, This is really an interesting article ,I love drupal, php , xml , and website,thanks for you, Kurt !

we've use drupal for our site. to be honest, the site isnt the prettiest from a design standpoint and most importantly....we cant get google to pick up our blogs!!

Getting google to pick up your blog was a problem evident in older versions of Drupal that required some acrobatics in order to achieve SEO friendly urls. In the last year or two this has become much easier.

Getting your URLs to be google friendly (not to mention using the google sitemap feature) requires planning -- you can't just build a default site and expect it to be SEO friendly -- you need to plan your SEO strategy before development, just like you would with any other kind of CMS.

If you google "Google and Drupal SEO issues" you'll find a lot of guides on how to make a google-friendly Drupal based site. Also here's a nice top 10 list:

http://tips.webdesign10.com/basic-drupal-seo-on-site-optimization

Sharepoint Alternative,

If you're using Drupal out of the box, I'd agree with you about design. The default theme is one of the uglier ones that I've seen, and it isn't really all that well suited for more complex work. I work heavily with the 5.x branch right now (simply because that has the best modular support) but one thing that I've found invaluable is to use panels.

Panels let you create considerably more complex structures, especially if you include the Panel Block module as part of it. I've found that this lets me nearly any layout I could think of without having to spend much time in CSS.

You may also want to look into views, especially with some of the bonus pack views. I've found that I can create fairly sophisticated galleries with views, and if you tie views in with panels, you can get decent layouts.

-- Kurt

This is an excellent overview. The extensibility of Drupal makes it incredibly powerful for assembling (vs. programming) modern website with minimal effort. It brings together the best of social software (blogs, wikis, forums), the best of web content management, and even many aspects of web application development frameworks into one package.

You can see a quick demo of Drupal at:

http://acquia.com/blog/drupal-and-acquia-web-20-expo

The page you find when you go to admin/content/node is not a node and there is no entry for that page in Drupal's URL alias table. A node, which is a piece of content, can only have one alias, which makes sense since you wouldn't want to link the same content using different URLs.
In Drupal the menu API provides for defining URLs and callback functions for these URLs, for example to generate listings of content or administration pages like the one mentioned above.

Hello, I really enjoyed the perspective of foundation that you share. It's hard to buck a good trend, and strong players are needed on the field to spur on better technology. I would appreciate very much to have you really kick the tires of http://xlsuite.com and let me know your thoughts.

Thanks, Lee
Community Relations

Kurt,
Good Article.
I hate to sidetrack, but the xforms site has been hijacked and I didn't know of any other way to contact you.
Love you articles.

thanks a lot for the feedback and suggestions driveby and kurt!

I am trying to pick a platform for making a multilingual Web site; I don't have "deep" expertise in coding, but know my way around HTML, and have worked with various open source products. I've kept my eye on Joomla and Drupal -- and am basically trying to find something that offers the ease of blogging, but integrates translation management (for human translation, and ideally machine, when a human translation is not yet available).

I was encouraged by a drupal enthusiast to try a few things, and I got into TinyMCE, some other seemingly related plug-ins, and the dependencies and versioning basically got too complex for me. All I wanted to do really was to have the easy of something like wordpress - make a post, add a picture inline -- but within the context of a cms, where I could go back and get an article translated.

Any suggestions on how best to keep track of when it will become easier to achieve this -- or to find someone who knows multilingual Joomla/Drupal who I could hire to help? It's for my PhD research.

Wish I had seen this in August after OSCON, but it really strikes a chord. I've been a Drupal dev for a long time, and have recently been focusing on the newer "agile frameworks" in Python and Ruby, and I've found the same shortcoming: there is no component architecture, at least not on the level of Drupal with its myriad callback hooks and "monkey patching" facilities. I've been ruminating on the idea of following the Drupal architecture to build a similar framework on top of WSGI. It would be lovely to make something that can use all of the existing (WSGI) components generically but provide a cohesive platform allowing components which specifically address it (by following template conventions etc) to effortlessly integrate.

I hate to sidetrack, but the xforms site has been hijacked and I didn't know of any other way to contact you.

i like drupal and very new to. i am having a hard time understnding it fully but I am sure that will come with time. Thanks for your post!

Kurt,
Good Article.
I hate to sidetrack, but the xforms site has been hijacked and I didn't know of any other way to contact you.
Love you articles.

As much as I love Drupal Wordpress is giving it a run for its money.

Popular Topics

Archives

Or, visit our complete archives.

Recommended for You

Got a Question?