Thursday, November 25, 2010

How free software/open source means never being unsupported

The strangest aspect of promoting free software (software that has been freed, as opposed to software available for no money), also known as open source, is that many people are concerned about the lack of support. "If we switch to Ubuntu", argues the skeptic, "we don't have the support of a giant corporation like Microsoft if things go wrong."

I'm going to give a contrary view, and use my experience with Android, the popular free software operating system for touchscreen smartphones, as a great example of how wrong this view is in practice.

My conversion to GNU/Linux

But before I do, let me explain my history a little. My first computers ran proprietary operating systems. I bought a computer from Sinclair (called a QL), and then a computer from Commodore (called an Amiga), both of which suffered from a substantial lack of support. The issues were different in both cases: Sinclair went bust, and was bought out by a company, Amstrad, that had no interest in the computer I'd bought. It was discontinued, as was all support. Commodore suffered with financing and mismanagement issues, and while the technical people at Commodore put together an extraordinary device, it became clear after a while that the platform itself - despite being decades ahead of the competition technically - had no future. What support there was was third party.

After Commodore went bankrupt, I faced a choice of jumping ship again to yet another proprietary product vendor, which would probably have been Microsoft, who was producing a software product I considered inferior to what I already had, or going in a different direction. I choose the fledgling GNU/Linux platform, buying some cheap hardware to experiment with, and eventually using it as my primary system.

My needs were relatively modest and non-mainstream in the beginning, and so the lack of support from hardware manufacturers and big software companies wasn't an issue. Over time, the community around GNU/Linux filled in the gaps. Drivers for even relatively obscure devices started to be written, and the software infrastructure issues - the need to be able to open the occasional Word document or Excel spreadsheet - started to be filled in by free software/open source enthusiasts who reverse engineered these file formats and created full applications able to handle them.

Still, in the early part of the 21st Century, GNU/Linux still seemed awkward and difficult to work with, and my own needs required something more integrated with the rest of the world. So, with Mac OS X looking increasingly good, I bought a Mac and used Macs for a few years.

And then I switched back to GNU/Linux. Why? Because the Mac was constricting. Because support wasn't what I thought it would be. To do something as simple as using the "latest" version of Java (which in Apples case was always at least a year behind), I'd have to upgrade to the latest version of Mac OS X, which in turn would only run on some of my hardware, and which, to be quite frank with you, I didn't even like much. I could not run the operating system on a Thinkpad, despite disliking the Powerbook hardware and pointing device. I was reliant on Apple for certain core software, for all hardware, and there was only a limited amount third parties could do to fix the gaps.

In the mean time, the major issues with GNU/Linux were resolved, because they could be! The system was entirely open, anyone could fix the issues, and so people did. A community of developers, including some provided by major corporations like Novell, IBM, and Redhat worked together on a graphical user interface called GNOME 2, that made the system slick and easy to use. Canonical worked on putting together a package that could install on virtually any hardware and provide a default environment that was feature full and easy to use, and called it Ubuntu. All of these organizations were able to do this because the system was free, because it was entirely open, nobody had to ask the permission of anyone else to work on this, and as long as people wanted support, the support would be there.

And now we're at the stage where Ubuntu, the most popular desktop variant of GNU/Linux, is arguably a better package over all, for the majority of people, than either Mac OS X or Windows, being, arguably, more functional than either, and being close in terms of ease of use to the former.


My first Android phone I bought a whole six months ago. The phone is a T-Mobile myTouch 3G Slide, which I bought because I was buying my wife one, because it had exceptional reviews, and because I didn't know the G2 was just around the corner. The Slide is a fairly capable phone, but there are a number of issues with the operating system on it - certain apps, such as the phone, crash regularly, the Bluetooth headset support is virtually unusable (you can't use voice dialing through the headset, for example, and even if you did you have to interact with the phone's touchscreen to "confirm" the operation you requested. Oh, and just to add insult to injury, the voice dialer "recognizes" commands like "Turn off Bluetooth", which it does without asking for confirmation!), and so on. The version of Android running is 2.1, a recent and respectable release, but nothing to write home about.

HTC released an update in August that fixed some bugs in the phone, but left the headline issues above still very much alive. The next step was to hope that HTC would get around to fixing the issues with the next operating system update, perhaps fixing them if and when it releases Android 2.2.

Well, that's what my option would have been had Android been proprietary. After all, that's what I'd have to put up with if I had an iPhone, or webOS phone, or Blackberry, or Windows Mobile phone. In all of these cases, I'd be dependent upon three things:

  • The maker of the operating system (Apple, Palm/HP, RIM, or Microsoft) fixing the bug if it's their bug.
  • The maker of the phone (Apple, RIM, or one of a swathe of third parties) fixing the bug if it's a bug they themselves introduced when customizing the operating system to work with their product.
  • The maker of the phone actually planning to release an update, and getting the update out.
But this is Android, and Android is free software, and quite frankly, there are a lot of people from the free software/open source worlds that simply aren't willing to rely upon third parties to support them. They're like me - people who have relied upon companies like Sinclair, Commodore, Apple, and even Microsoft, to provide them with what they need, only to be let down again and again and again and again - and these people have said "You know, let's take Android, let's fix the issues that bother us, and share our fixes with the rest of the world."

My phone hasn't been running the stock HTC Android operating system for several months now. Instead, I've been running something called CyanogenMod. CyanogenMod is a third party variant of Android, where a developer called Steve Kondik (he uses the online nick "Cyanogen", hence the name), and other like minded developers, have taken Android, fixed the headline issues they consider important, and ported this version to as many devices as they can.

What's the result? Well, with this latest version (CyanogenMod 6.1RC2), every major issue I had with the Slide's operating system is fixed. The system is stable, all the apps work, Bluetooth headsets work properly (including voice dialing), and I also have the benefit of running a more recent version of Android, 2.2, that includes some nice new features like easy "tethering" to a laptop or similar device.

If that were all there is to it, that would be very positive all by itself: I've had issues fixed that those reliant upon HTC are still waiting for. But, in fact, the benefits are actually greater. The first Android phone was the G1 (also known as the HTC Dream), which is barely two years old. HTC stopped supporting this around a year ago, which means owners reliant upon HTC for updates are out of luck if they want a more recent version of the operating system, or any severe bugs fixed.

The same will, ultimately, happen to most of our devices. Now, I'm drooling over various Android phones, and wouldn't mind an upgrade, but I've only had this one for six months, and like most real people, I don't want to spend $400-500 on a new phone every few months. Most people keep their devices for at least two years. So the knowledge that, in all seriousness, it's very unlikely HTC will release updates for my phone in six months from now, is somewhat disconcerting. Or would be if I were reliant upon HTC.

And again, if I was using a proprietary operating system, I would be reliant upon HTC. Those buying a Windows phone right now will find it impossible to get updates a year from now - not without buying new hardware.

But plenty of people with G1s are running Android 2.2, despite HTC cutting off the air. They're able to do this because Cyanogen, and others, have released versions of their variants of Android for the G1. As long as the hardware is capable of running it, and people are still using the phones in question, you will see ports of the latest versions of Android for these phones, because the phones are supportable.

And that's the key word: supportable. Free software makes hardware supportable. Proprietary software means that only a small group of people can support something, and only companies with unlimited resources (and no agendas!) will ever provide unlimited support.

Interested? You can find information about CyanogenMod here. The CyanogenMod operating system is only available for a subset of Android devices, you can check the Wiki for the full list. But if it runs now on a device now, then that device is, by definition, supportable.

If support is one of your priorities, you need to ask yourself what kind of support you need: the support of people with the same concerns as you? Or the support of someone who benefits from putting you on an upgrade treadmill, and who may, in time, decide it's just not worth the effort.

Sunday, November 14, 2010

Choosing a web technology

When I first started web development, which would have been around 1997 or 1998, writing web-based applications was not pleasant, and the choices were slim. You generally had to use CGI, using C or Perl, although some alternatives were being developed at that time, many of which have become major frameworks today.

For the most part, the successful frameworks, with one exception, have recognized that programmers do not make great UI designers, and UI designers do not make great programmers. I've worked with people who do fantastic jobs on the UI design front, backed with the most hideous, poorly written, unreliable, difficult to maintain, junk code on the back-end. And, well, I'd have to admit that while my UI work of late has been better, my early attempts were somewhat limited.

So, says the majority of modern frameworks, what you need to do is separate the application logic from the HTML. Different systems have achieved this to different degrees, but most now at least make a passable effort, if only by using a system of HTML templates, in which code can be embedded. A UI designer need only create the HTML templates, knowing what hooks to use to bring up the application specific logic.

Let's go through four popular web development systems, in my mind from least successful to most, and explain what's going on.


ColdFusion is, well, "interesting". The system pretty much goes against the logic described above, taking the view that the UI designer should be the programmer, and designing a "programming environment" that's supposed to be familiar to anyone who can mark up HTML. And in some ways that explains its success, many UI designers love ColdFusion because the concepts are easy for them to understand, allowing them to handle the entire product development themselves.

Add additional hands, especially a split of programmer and designer people, to a CF project, and it starts to show its flaws. ColdFusion requires the same people handle both ends of the project, with a certain amount of logic embedded on every page - indeed, every CF page is by default a program. You can't design something in an HTML editor and have it run in CF without making significant changes.

And the advantage for designers, that the "code" syntax is similar to that of HTML, is a massive handicap when you put a CF project in front of a programmer. CF's native code format is difficult to read and follow. While CF also supports a more programmer friendly syntax, that syntax is a bolt-on, and so any project started by a person who used the HTML-like syntax will contain a large amount of that code that needs to be maintained.

There are workarounds for CF's issues. The major successful CF project I've been exposed to used web services to abstract as much of the programming logic as possible, ensuring the front-end only contained the code that was absolutely necessary. I still wouldn't have used CF, but it meant a UI designer could be involved who would have otherwise had issues with a different environment.

CF also, for what it's worth, has other issues, the major one being that the only complete implementation of the system is proprietary, and has licensing issues. It's hard for me to think of an occasion in which I'd recommend the use of ColdFusion for a new project, but others would.


PHP is, like ColdFusion, an established web application development system with a long history, based around a custom programming language. Unlike CF, PHP makes no attempt to provide a UI designer friendly programming system - PHP scripts look similar in many ways to Perl, a language notorious for its confusing, punctuation-based, syntax.

However, PHP changes things around in one important respect: PHP pages are, by default, HTML. Code is embedded in the HTML where needed. This critical difference makes it easier for programmers and UI designers to work together: UI designers can develop and prototype HTML templates, and programmers can modify those templates with embedded PHP mark-up, with the integrity of the HTML remaining.

PHP is free software, that is, there are no licensing implications from merely using it, and modern iterations of PHP have moderately advanced programming features. It's not a bad system, not ideal, but it works fairly well.


Early in the history of Java, there was some debate as to where the language was going and what it would be most suited to. After an abortive effort trying to promote it as a solution to cross platform development of desktop applications, its real niche as an enterprise-class secure language started to become more apparent. Sun developed various standards for web development, starting with two technologies, servlets and JavaServer Pages (JSP.)

While servlets took a strictly programmer-oriented approach to web development, similar in some ways to CGI, JSP took the same template approach that PHP was taking, and took it further. JSP pages are, like PHP, standard HTML pages with embedded Java. But JSP is a substantially more advanced framework, oriented towards larger, more difficult to maintain, applications. In particular, it is easier for a UI designer to have complete control over the HTML generation, maintaining the HTML template system, while handing over business logic tasks to a programmer. JSP achieves this in a variety of ways, allowing programmers to work at different levels with the HTML, from embedding raw code on a JSP page (as with PHP), to providing tag libraries (custom "HTML" tags the designer can embed in the page, whose logic is controlled by the programmer.) Because the programming side is Java, programmers have a wide variety of options and can make use of a well supported, modern, scalable, environment with a huge amount of support.

JSP is rarely used as-is. For all of its advantages over something like PHP, Java's infamous over-engineering frequently makes it hard to do the simplest things, and as a result a large library of third party frameworks have come into being to make it easier to develop full applications. But JSP is an extremely flexible system, and coupled with good Javascript libraries like GWT and jQuery, it's one of two I'd generally recommend for all but the smallest of projects.

JavaServer Faces and Facelets

Originally built-upon JSP, JSF has come a long way from its origins as an attempt to make an MVC-friendly Java framework. Early versions were, frankly, abysmal, notorious for inheriting Java's over-engineering and bureaucracy, undermining the principle advantages of the framework.

Over time, the major issues have been fixed, with JSF 1.2 introducing "Facelets", a replacement for JSP that more closely matched JSF's requirements, and a large collection of major improvements with JSF 2, that allowed Facelets to be built around HTML templates, and that undid massive amounts of the bureaucracy associated with JSF using a system of attributes.

In some respects, it's better to pretend JSF didn't exist until JSF2. So let's do that and discuss what JSF2 has over JSP.

With JSF2, pages are marked up using standard HTML, as with JSP and PHP before. However, standard HTML tags that need to be manipulated in some way, such as form controls, can be linked to internal JSF2 controls using a custom "jsfc" attribute, making the link between the original HTML and Facelet version even more transparent.

But that's not the major advantage. Whereas the systems mentioned previously simple allow code and HTML to be mixed, JSF introduces the concept of a "view", which matches closely the user's perception of how a website is navigated. Each view consists of an HTML template, and a set of "beans" (high level Java objects) that link to different fields on the template. When the user updates things - changes the text on forms, hits submit buttons, etc - those beans are similarly updated, and the beans are also asked whether the user should be directed to a new page (or view), or the same template with updates (ie the same view.)

This is a much more high level approach than JSP, and from a split-of-development standpoint, it makes for a much more scalable system. Programmers and UI designers can easily understand each other's work, with the UI designers doing barely any programming, and the programmers putting together reusable interfaces to the business logic. And (with JSF2, at least) development is faster - the developers do not have to concern themselves with parsing the results of every form submission, they simply have to put together the containers, the beans, that contain the data. Even if no web designer is involved, JSF is a great platform.

It's a staggeringly powerful system, and I suspect that if it wasn't for earlier releases, JSF2 would be the most popular Java web platform around. As you've probably figured out by now, JSF2 is the other of the two frameworks I would recommend (the first, as I said earlier, was something JSP based), but you'd probably want to know why I might recommend a JSP solution over a JSF solution.

JSF's Achilles heel is that it is entirely session and form based. Navigation must be done via submitting forms. The results of those form submissions modify the beans associated with the current view or session. This means that pages cannot be "spidered" - that is, a JSF application is pretty close to impossible to put into a search engine, and  users cannot bookmark pages. There are workarounds, such as PrettyFaces, but they, to a certain extent, aren't really designed for the kinds of applications you'd want JSF for in the first place.

If I was writing a Javascript-heavy interactive application where much of the logic is embedded in the scripts, there'd be little reason for me to use JSF, which would mostly get in the way. That's not a hard and fast rule, but it works as a generalization. And if I was writing something like, say, a retail front-end, where each product has its own product page, I certainly wouldn't want to use JSF, and JSP would be fine for those applications.

Conversely, if I'm writing an actual application, like an airline reservation system, or an Intranet-located front-end to a database, JSP would work, but JSF would be easier to maintain, and easier to work with.


In the end, there's a certain comfort factor involved, and its hard to get people used to knocking up quick and dirty tools using CF or PHP to move to a more robust, powerful, system. But I must admit, after this long in the industry, it's hard for me to recommend those kinds of systems, even for the "quick and dirty" tools. Java provides two extremely good, robust, options for web development, with features not present in CF or PHP, that suit team work and rapid development practices. So if you ever commission something from me, and want to know why I'm using JSP or JSF, you now know why.