Ubuntu is ready for the desktop. But.

For as long as I can remember, the joke has been that "This year will be the year of Linux on the desktop", which ceased to be funny around 2006 or so, when Ubuntu genuinely became a usable, just-works, great to work with operating system for the rest of us. Ubuntu may never have been the clear leader in any category, but "Almost as user friendly as Mac OS X (and light-years ahead of Windows)" and "Almost as functional as Windows (and light-years ahead of Mac OS X)" certainly isn't a bad place to be.

There's just one irony here though, while the Ubuntu people have done a great job making an operating system traditionally associated with cheap servers into a platform for the rest of us, their attempts to make a server have become increasingly ridiculous.

I'm going to explain why.

Domain management

I've had an interesting experience recently. I wanted to put together a "poor man's Active Directory" for myself, and I also wanted to integrate a product with an existing domain.

Active Directory is a technology from Microsoft that, in turn, is built upon (amongst others) two major open standards, Kerberos and LDAP. The two technologies are largely complementary although they have some overlap, which we'll come to in a moment. AD is an example of something called a domain controller system. Domain control is a centralized security system, whereby a single server stores all of the information about all of the users on a network, plus security and configuration information.

How important is it? Answer, very. Virtually every corporate entity with a network with more than a handful of users should be using a system like this. Every non-trivial network has a number of services running on it, that at minimum include people's ability to log in to their PCs, network shares for storing data, VPNs, and email. Without a centralized security system, without a domain controller, each of these subsystems needs to be managed separately, with multiple passwords, multiple concepts of what constitutes a "user", and with it being increasingly the case that complex security will have no security substituted for it.

Domain controllers use a system called LDAP to store information about all the users and other entities on the network. When a server needs to know whether to grant access to a resource to a specific user, the server simply makes a query from the LDAP database, and uses the results to make a decision. Typically users are added to one or more groups, and then the group membership determines their roles. But the LDAP database also stores other information, so a server can extract information such as the user's email address or even their phone number.

Take a typical email system. There are multiple ways in which email and LDAP can be interlinked. Email clients can use the LDAP database to configure themselves. The email server can use LDAP to configure itself, determining which users to store and what groups (mailing lists) to set up. When a user is composing an email, they can use LDAP to look up other people in the corporate directory.

In a modern domain controller, the LDAP system is complemented by Kerberos. The job of Kerberos is to identify the person trying to access a resource. Typically when a user logs in (say, into their PC in a Windows environment), what's happening behind the scenes is that they're actually having their credentials checked by Kerberos, which then issues them a ticket. From that point on, until the user logs out, whenever the user does some kind of network activity, the ticket is available for if a server asks for it. And servers will ask for a ticket when they're trying to determine who is using their services.

Let's take the email example again to explain how Kerberos fits in to all of this. Usually when you make a connection to an email server, you need to tell the server who you are so that the server can determine who's email to give you. Without a Kerberos system, that involves sending a username and password. However, if the email system accepts Kerberos, then your email client will simply send the Kerberos ticket you received when you logged in to your PC identifying you to the server, which means you can safely use the email client to access your email without you having to give the email client your username and password.

So domain management is a pillar of modern networking, up there with DHCP and even DNS as something no corporate network should be without.  And virtually any piece of software designed to run within a corporate network needs to be built to support it.

And that's not hard. There's nothing proprietary about the technologies used with even Active Directory. (AD does include some non-standard extensions to Kerberos, but they only matter if you plan to create a complex network that has several Kerberos servers from different vendors within it.)

Now, it's important to note that while the systems are built upon these two common standards, the way LDAP is used differs from implementation to implementation. Most non-AD implementations standardize on something called the Posix schema (a schema is a set of rules about how to store information in LDAP and what to store.) The Posix schema is built for Unix-like operating systems, and is based on the Unix security model. The Unix security model is, to be honest, woefully outdated, which is why when Microsoft built Active Directory they virtually ignored the Posix schema and built their own, based upon the model of security used by the Windows NT (200x, XP, Vista, and 7) operating systems. Despite the poor reputation of Windows in the security sphere, the actual model it uses (as opposed to the implementation...) is extremely solid, so this was a wise choice.

So, what's the problem?

Domain controllers and Ubuntu

As I said above, I tried doing two things involving domain controllers in the last month or two. One was setting up a "poor man's Active Directory server". Now, by that I mean a cheap domain controller.

Microsoft's own solutions to domain management are actually fairly (well, perhaps a better term would be "ridiculously") expensive. For Active Directory, Microsoft requires that you buy a server (upwards of $1,000) version of Windows, upgrade all of your XP, Vista, and Windows 7 installations to Professional or above (add around $100 per client), and then add an additional $70ish/client "Client Access License" to that.

So the cheap solution is to install a free or open source server, and jump through the necessary hoops needed to integrate it with non-Posix boxes.

Ubuntu has instructions here on how to set this up.Theoretically, if you follow the instructions, you end up with a Kerberos domain controller with an LDAP back-end, which in theory is what you're after. In practice, those instructions do not work.

They've never worked. They can't work. And even if you fix the obvious issues, such as the wrong LDAP administrator account being given, you still end up with entirely separate LDAP and Kerberos systems. Add users to one, they will not appear in the other. If a user changes their password, they need to change it in two places.

And that's leaving aside personal preference issues such as the decision to go with MIT Kerberos rather than the infinitely superior Heimdal system.

What I want to know is why that page even exists? Why post bogus guides on your website that will cause people to waste enormous amounts of time trying to get something to work? There is no way in a million years anyone put that information together in good faith, it's simply not possible it ever worked, for anyone.

And that's my first issue with Ubuntu server. It's not that it's not capable of being a domain controller - I mean, if all else fails you can compile and configure, by hand, virtually any combination of open source software on Ubuntu, so it can be one. It's the attitude, here's something fairly important in the great scheme of things, and they don't really know what it is, but they don't want to admit it, so they're willing to post complete crap rather than actually say "This isn't supported yet."

To make matters worse, part of the problem here appears to be a decision by the Ubuntu people to completely change the way the OpenLDAP LDAP server works on 10.04. There doesn't appear to be any advantage in the changes they've done, changes that effectively remove support for the most common format used to distribute schemas. Someone, somewhere (perhaps the OpenLDAP people, but Canonical, the producers of Ubuntu, at least has the ability to delay such changes if they're not ready for prime time - which clearly they aren't) decided to go ahead and incorporate the changes in Ubuntu without considering for a second what the actual consequences of such a move would be.

The second problem came when I attempted to integrate a server with an Active Directory system. This should have been easy, the Ubuntu people ship a tool with Ubuntu called likewise-open that does everything on the back-end and ensures things like ssh and ftp "just work", and then as Apache natively supports Kerberos, LDAP, and a whole host of other protocols, it should just be a matter of telling your web apps to use Apache's built-in authentication. How hard could it be?

Likewise-open is broken. There's a bug on it, but nobody seems to be interested in fixing it. Essentially, the system doesn't support default domains at the time of writing - it says it does, but it ignores the setting. Every user needs to log in as COMPANYDOMAIN\userid. What? Why?

Now, in fairness, you can use Kerberos to get around some of this, except where the apps you install that actually use the usernames have restrictions upon what can be in them, or where client software still hasn't been Kerberized and requires that you use a username and password anyway. As an example, the popular secure shell client Putty is awaiting full Kerberization, and certain tools, such as the popular Subversion source code management system, actually use Putty to talk to the back-end. So much for transparent "just works" operation.

And that brings me on to the second part: despite the ease of doing so, most open source applications eschew using the built-in authentication schemes offered by the underlying platforms, instead taking the trouble to re-invent, and re-invent, and re-invent the wheel, over and over again. Those that are willing to talk to something like LDAP rarely do so cleanly or generically, usually requiring a specific schema be supported, and in some cases the ability to write back to the database.

Did I get to do what I planned? Well, yes, but never by doing things the official way. One web app I was integrating supported a plug-in architecture, and so after a lot of searching through the code I ended up writing a plug-in to handle authentication and integration with the AD back-end.  The "Poor man's AD" I haven't completed, largely because I need to update the hardware involved, but I did get a proof of concept running based around Apache's excellent Apache Directory project.


Why is it so bad?

Full blown domain controllers are either easy to set up, or they're cheap, but never both. Microsoft gets away with charging what it does for Active Directory because the alternatives are just too difficult to get working in the majority of cases. I might add that as a developer it's always been awkward to play with the technologies because in the context of employment, the people who have domain servers are rarely the people who need to experiment with them and integrate their security with them. There are several implications to this, but one is that the number of people who understand domain security is limited to a select few, a handful who have had exposure to it, plus a very small subset of geeks who are interested in it and are willing to spend a small fortune on experimenting.

This is ultimately somewhat damaging. In order to be integrated into anything but the smallest of networks, a web server, web app, file server, or any other type of server, really needs to be capable of being integrated into that network's security system. If it isn't, it becomes not merely another box to be managed, but another system, another domain, to be managed too. Quite literally, just adding one domain-less Ubuntu server to a Windows network, used by everyone in the organization, potentially doubles the amount of work required to administer the network. Right now, it's not clear that enough people in the open source world understand this.

• Canonical clearly understands that the concept is necessary, but has no idea what it is or how to do it. The most obvious solution for Canonical right now is to get someone in who knows what they're doing to set up a proper domain management system, based upon the Posix schema, that can be installed as easily as, say, Apache, and to ensure packages are shipped that makes it easy to set up Ubuntu systems as clients of both that solution, and Active Directory. Likewise, every app in Ubuntu's repository that acts as a server, be it a web app or anything else, should be capable of using domain based security. No excuses. Oh, and before anything else is done, take out the bogus documentation - it doesn't work, it wastes people's time, it shouldn't be there.
• Kudos to the Apache Directory Project, and to Samba, both of whom, from separate angles, know what needs to be done on the domain controller end, and are working on it. Also kudos to RedHat for FreeIPA, although I personally think the Apache system makes more sense as an architecture, making Kerberos and LDAP part of the same physical server rather than having them awkwardly communicate with one another.
• Anyone considering writing a non-trivial open source web app or anything similar should ask themselves whether re-inventing the wheel to make their own super special cookie-based authentication system is really, actually, a good idea. Stop it! Knock it off! Learn what authentication is, and do it properly.
• The client side needs to be thought out. At this point, Ubuntu clients can, if set up correctly, authenticate against an Active Directory system and have its security managed by that system, but making Windows clients authenticate against a Posix domain controller is a little less clean.  While Windows Professional allows a machine to be joined to an arbitrary Kerberos domain, the security of the system isn't going to be managed by such a controller. As such, while systems like the Apache Directory Project and FreeIPA are very welcome, given the extent to which no centralized security system can be taken seriously unless it takes into account the number of Windows desktops out there, some effort needs to be made to make a good domain client for Windows.

How hard could it be?