Monday, September 27, 2010

Ubuntu is ready for the desktop. But.

For as long as I can remember, the joke has been that "This year will be the year of Linux on the desktop", which ceased to be funny around 2006 or so, when Ubuntu genuinely became a usable, just-works, great to work with operating system for the rest of us. Ubuntu may never have been the clear leader in any category, but "Almost as user friendly as Mac OS X (and light-years ahead of Windows)" and "Almost as functional as Windows (and light-years ahead of Mac OS X)" certainly isn't a bad place to be.

There's just one irony here though, while the Ubuntu people have done a great job making an operating system traditionally associated with cheap servers into a platform for the rest of us, their attempts to make a server have become increasingly ridiculous.

I'm going to explain why.

Domain management

I've had an interesting experience recently. I wanted to put together a "poor man's Active Directory" for myself, and I also wanted to integrate a product with an existing domain.

Active Directory is a technology from Microsoft that, in turn, is built upon (amongst others) two major open standards, Kerberos and LDAP. The two technologies are largely complementary although they have some overlap, which we'll come to in a moment. AD is an example of something called a domain controller system. Domain control is a centralized security system, whereby a single server stores all of the information about all of the users on a network, plus security and configuration information.

How important is it? Answer, very. Virtually every corporate entity with a network with more than a handful of users should be using a system like this. Every non-trivial network has a number of services running on it, that at minimum include people's ability to log in to their PCs, network shares for storing data, VPNs, and email. Without a centralized security system, without a domain controller, each of these subsystems needs to be managed separately, with multiple passwords, multiple concepts of what constitutes a "user", and with it being increasingly the case that complex security will have no security substituted for it.

Domain controllers use a system called LDAP to store information about all the users and other entities on the network. When a server needs to know whether to grant access to a resource to a specific user, the server simply makes a query from the LDAP database, and uses the results to make a decision. Typically users are added to one or more groups, and then the group membership determines their roles. But the LDAP database also stores other information, so a server can extract information such as the user's email address or even their phone number.

Take a typical email system. There are multiple ways in which email and LDAP can be interlinked. Email clients can use the LDAP database to configure themselves. The email server can use LDAP to configure itself, determining which users to store and what groups (mailing lists) to set up. When a user is composing an email, they can use LDAP to look up other people in the corporate directory.

In a modern domain controller, the LDAP system is complemented by Kerberos. The job of Kerberos is to identify the person trying to access a resource. Typically when a user logs in (say, into their PC in a Windows environment), what's happening behind the scenes is that they're actually having their credentials checked by Kerberos, which then issues them a ticket. From that point on, until the user logs out, whenever the user does some kind of network activity, the ticket is available for if a server asks for it. And servers will ask for a ticket when they're trying to determine who is using their services.

Let's take the email example again to explain how Kerberos fits in to all of this. Usually when you make a connection to an email server, you need to tell the server who you are so that the server can determine who's email to give you. Without a Kerberos system, that involves sending a username and password. However, if the email system accepts Kerberos, then your email client will simply send the Kerberos ticket you received when you logged in to your PC identifying you to the server, which means you can safely use the email client to access your email without you having to give the email client your username and password.

So domain management is a pillar of modern networking, up there with DHCP and even DNS as something no corporate network should be without.  And virtually any piece of software designed to run within a corporate network needs to be built to support it.

And that's not hard. There's nothing proprietary about the technologies used with even Active Directory. (AD does include some non-standard extensions to Kerberos, but they only matter if you plan to create a complex network that has several Kerberos servers from different vendors within it.)

Now, it's important to note that while the systems are built upon these two common standards, the way LDAP is used differs from implementation to implementation. Most non-AD implementations standardize on something called the Posix schema (a schema is a set of rules about how to store information in LDAP and what to store.) The Posix schema is built for Unix-like operating systems, and is based on the Unix security model. The Unix security model is, to be honest, woefully outdated, which is why when Microsoft built Active Directory they virtually ignored the Posix schema and built their own, based upon the model of security used by the Windows NT (200x, XP, Vista, and 7) operating systems. Despite the poor reputation of Windows in the security sphere, the actual model it uses (as opposed to the implementation...) is extremely solid, so this was a wise choice.

So, what's the problem?

Domain controllers and Ubuntu

As I said above, I tried doing two things involving domain controllers in the last month or two. One was setting up a "poor man's Active Directory server". Now, by that I mean a cheap domain controller.

Microsoft's own solutions to domain management are actually fairly (well, perhaps a better term would be "ridiculously") expensive. For Active Directory, Microsoft requires that you buy a server (upwards of $1,000) version of Windows, upgrade all of your XP, Vista, and Windows 7 installations to Professional or above (add around $100 per client), and then add an additional $70ish/client "Client Access License" to that.

So the cheap solution is to install a free or open source server, and jump through the necessary hoops needed to integrate it with non-Posix boxes.

Ubuntu has instructions here on how to set this up.Theoretically, if you follow the instructions, you end up with a Kerberos domain controller with an LDAP back-end, which in theory is what you're after. In practice, those instructions do not work.

They've never worked. They can't work. And even if you fix the obvious issues, such as the wrong LDAP administrator account being given, you still end up with entirely separate LDAP and Kerberos systems. Add users to one, they will not appear in the other. If a user changes their password, they need to change it in two places.

And that's leaving aside personal preference issues such as the decision to go with MIT Kerberos rather than the infinitely superior Heimdal system.

What I want to know is why that page even exists? Why post bogus guides on your website that will cause people to waste enormous amounts of time trying to get something to work? There is no way in a million years anyone put that information together in good faith, it's simply not possible it ever worked, for anyone.

And that's my first issue with Ubuntu server. It's not that it's not capable of being a domain controller - I mean, if all else fails you can compile and configure, by hand, virtually any combination of open source software on Ubuntu, so it can be one. It's the attitude, here's something fairly important in the great scheme of things, and they don't really know what it is, but they don't want to admit it, so they're willing to post complete crap rather than actually say "This isn't supported yet."

To make matters worse, part of the problem here appears to be a decision by the Ubuntu people to completely change the way the OpenLDAP LDAP server works on 10.04. There doesn't appear to be any advantage in the changes they've done, changes that effectively remove support for the most common format used to distribute schemas. Someone, somewhere (perhaps the OpenLDAP people, but Canonical, the producers of Ubuntu, at least has the ability to delay such changes if they're not ready for prime time - which clearly they aren't) decided to go ahead and incorporate the changes in Ubuntu without considering for a second what the actual consequences of such a move would be.

The second problem came when I attempted to integrate a server with an Active Directory system. This should have been easy, the Ubuntu people ship a tool with Ubuntu called likewise-open that does everything on the back-end and ensures things like ssh and ftp "just work", and then as Apache natively supports Kerberos, LDAP, and a whole host of other protocols, it should just be a matter of telling your web apps to use Apache's built-in authentication. How hard could it be?

Likewise-open is broken. There's a bug on it, but nobody seems to be interested in fixing it. Essentially, the system doesn't support default domains at the time of writing - it says it does, but it ignores the setting. Every user needs to log in as COMPANYDOMAIN\userid. What? Why?

Now, in fairness, you can use Kerberos to get around some of this, except where the apps you install that actually use the usernames have restrictions upon what can be in them, or where client software still hasn't been Kerberized and requires that you use a username and password anyway. As an example, the popular secure shell client Putty is awaiting full Kerberization, and certain tools, such as the popular Subversion source code management system, actually use Putty to talk to the back-end. So much for transparent "just works" operation.

And that brings me on to the second part: despite the ease of doing so, most open source applications eschew using the built-in authentication schemes offered by the underlying platforms, instead taking the trouble to re-invent, and re-invent, and re-invent the wheel, over and over again. Those that are willing to talk to something like LDAP rarely do so cleanly or generically, usually requiring a specific schema be supported, and in some cases the ability to write back to the database.

Did I get to do what I planned? Well, yes, but never by doing things the official way. One web app I was integrating supported a plug-in architecture, and so after a lot of searching through the code I ended up writing a plug-in to handle authentication and integration with the AD back-end.  The "Poor man's AD" I haven't completed, largely because I need to update the hardware involved, but I did get a proof of concept running based around Apache's excellent Apache Directory project.


Why is it so bad?

Full blown domain controllers are either easy to set up, or they're cheap, but never both. Microsoft gets away with charging what it does for Active Directory because the alternatives are just too difficult to get working in the majority of cases. I might add that as a developer it's always been awkward to play with the technologies because in the context of employment, the people who have domain servers are rarely the people who need to experiment with them and integrate their security with them. There are several implications to this, but one is that the number of people who understand domain security is limited to a select few, a handful who have had exposure to it, plus a very small subset of geeks who are interested in it and are willing to spend a small fortune on experimenting.

This is ultimately somewhat damaging. In order to be integrated into anything but the smallest of networks, a web server, web app, file server, or any other type of server, really needs to be capable of being integrated into that network's security system. If it isn't, it becomes not merely another box to be managed, but another system, another domain, to be managed too. Quite literally, just adding one domain-less Ubuntu server to a Windows network, used by everyone in the organization, potentially doubles the amount of work required to administer the network. Right now, it's not clear that enough people in the open source world understand this.
  • Canonical clearly understands that the concept is necessary, but has no idea what it is or how to do it. The most obvious solution for Canonical right now is to get someone in who knows what they're doing to set up a proper domain management system, based upon the Posix schema, that can be installed as easily as, say, Apache, and to ensure packages are shipped that makes it easy to set up Ubuntu systems as clients of both that solution, and Active Directory. Likewise, every app in Ubuntu's repository that acts as a server, be it a web app or anything else, should be capable of using domain based security. No excuses. Oh, and before anything else is done, take out the bogus documentation - it doesn't work, it wastes people's time, it shouldn't be there.
  • Kudos to the Apache Directory Project, and to Samba, both of whom, from separate angles, know what needs to be done on the domain controller end, and are working on it. Also kudos to RedHat for FreeIPA, although I personally think the Apache system makes more sense as an architecture, making Kerberos and LDAP part of the same physical server rather than having them awkwardly communicate with one another.
  • Anyone considering writing a non-trivial open source web app or anything similar should ask themselves whether re-inventing the wheel to make their own super special cookie-based authentication system is really, actually, a good idea. Stop it! Knock it off! Learn what authentication is, and do it properly.
  • The client side needs to be thought out. At this point, Ubuntu clients can, if set up correctly, authenticate against an Active Directory system and have its security managed by that system, but making Windows clients authenticate against a Posix domain controller is a little less clean.  While Windows Professional allows a machine to be joined to an arbitrary Kerberos domain, the security of the system isn't going to be managed by such a controller. As such, while systems like the Apache Directory Project and FreeIPA are very welcome, given the extent to which no centralized security system can be taken seriously unless it takes into account the number of Windows desktops out there, some effort needs to be made to make a good domain client for Windows.
How hard could it be?

Friday, September 17, 2010

Issues with the monoculture

A few years ago, there was a brief blip when people started to talk about a major security issue in modern computing being "the monoculture". Part of this was a response to the anti-monopolist lawsuits against Microsoft, plus an attempt to explain why Microsoft's operating systems received so much attention from hackers, and non-Microsoft operating systems barely any.

The issue is this: if everyone runs the same operating system, then anyone who writes virus or worm that exploits a fault in that operating system will find their virus or worm impacts virtually everyone. There are two angles to this:

  • From the point of view of a victim, the consequences of a monoculture can be devastating. A single virus can destroy a business's ability to function, as every employee's personal computer becomes infected and non-functional, as well as the computers belonging to their suppliers (and clients.)
  • From the point of view of the hacker, spreading the virus becomes merely a step of finding a mechanism to identify other computers, as it's guaranteed that those computers can be infected.
The discussion died down for a variety of reasons. Ironically, many of those who would benefit from the argument refused to accept it because it meant accepting their own chosen, non Microsoft, platforms were just as flawed as Windows. A case in point is Mac OS X. Mac OS X has always had security holes. Earlier versions, up until a point release of 10.3, actually were so insecure that all a programmer had to do to "deliver" malware to a user was to ensure a website they browsed to sent it. Safari would, automatically, without the user's involvement or say so, by default download and unarchive the "application", and its mere presence on the user's disk would "install" it. To get it to run,all you had to do was ensure the application was associated with a common file extension or two, so that the next time the user clicked on that type of file, it would open. This was something the hacker/programmer could do, they didn't need the user to do anything.

Now that particular hole (which was open for years, undermining the notion that Mac OS X was ever built by anyone who considered security a priority) has been fixed, but holes still pop-up. Mac OS X has, since its release, had security updates delivered automatically every month or so. These updates would be, by definition, unnecessary if Mac OS X was already secure. Yet many Apple enthusiasts, to this day, didn't and don't accept the idea that Mac OS X might be insecure. The fact no virus has hit OS X users has been used as evidence of this.

But in actual fact, the reason Mac OS X hasn't been hit by a virus has been because it has such a comparatively small market share, in any community of users. Now, that last bit takes some explaining.

Back in the 1980s, viruses were common and were successful on a range of platforms, even platforms that weren't particularly popular. MS DOS had many, but so did the Commodore Amiga and the Atari ST, both of whose market shares were dwarfed in comparison to the PC. Part of the reason for this had to do with the "networks" at the time. Networks, in the 1980s, generally consisted of people swapping disks with one another. Those disks were intended for a single platform - people didn't expect a disk for a Mac to even be readable by an Amiga or PC. Largely because, without special software, they weren't. So while Macs might have had a small market share, Mac viruses were successful because they were monocultures within the networks they belonged to. As were Amigas and STs.

Fast forward to today, and that's not the case. Someone who writes a virus for a Mac will find that the vast majority of machines their virus "hits" will be unable to run it. Worse, if they use something like the old "Disguise an executable as a JPG and email it to everyone" trick that plagued users during the late 1990s and early 21st Century, they'd find that Mac users would be warned pretty quickly by all the non-Mac users who would receive the corrupt JPG. The fact the virus would attempt to infect uninfectable computers would makes alarms ring.

So, anyway, the point here is not to analyze Mac security holes, but explain why standardizing on a single platform has negative implications for any business. By standardizing on a single platform, regardless of what that platform is, you do many things that will negatively impact your security:
  • You significantly improve a viruses' chance of success by ensuring the majority of machines it will hit can be infected.
  • You reduce the likelihood of early warning and detection, by ensuring that viruses don't attempt (and fail) to infect machines they weren't built for.
  • You ensure that any virus that hits your organization will have a devastating, crippling, affect on your business.
Moving away from the monoculture tends to scare many system administrators. Licensing seems easier if everyone uses the same platform (although it can also be much more expensive), and certain tools work better with specific platforms. Still, much of that is changing. Apple is doing its best to ensure Mac OS X fits transparently into organizations that are primarily based upon Windows, and while Ubuntu seems to lack a community of developers that understand, say, domain-based security, it is at least moving in the right direction.

There's certainly no reason to prevent your users from using the right tools for the job, and by doing so, you also help make your own network more robust, and you become a better Internet citizen. The right choice is choice.