And Then…

Both Windows and UNIX (and Linux and Mac OS X) support some type of event log. The event log consists of a set of timestamped entries that document significant operating system or application activity. Events might be strictly informational (Successful logon for Joe) or might document errors (Unsuccessful logon for Joe or Out of disk space!)

As you might imagine, event logs are important. They let help system administrators in several ways:

• Problem diagnosis. When a computer is misbehaving, administrators can look at its event log to see if there are messages in it that provide clues to what’s broken.
• Security enforcement and intrusion detection. Event logs contain information that can be used to detect intrusion attempts and unauthorized access to protected resources.
• Auditing. Event logs contain information that helps administrators keep track of who is accessing what systems and what they’re doing while they’re there. This information can be used to satisfy security auditors that adequate practices are in place.

Although both UNIX and Windows have event logs, their mechanisms are very, very, different. The Windows event log consists of structured data with explicit fields for “Type”, “User”, “Computer”, “Category” and other fields. Windows provides an API that allows event logs to be read from local or remote computers. The API allows monitoring programs to securely access event information from Windows computers.

The UNIX event log is stored in text files. The syslog API is typically used to write to the event log text files but syslog does not support remote access of UNIX event logs. Although syslog provides for a facility code and a priority field in its API, the data actually stored in the text files is mostly unstructured.

Both mechanisms have their problems.

Although syslog data is unstructured, it is in a great form for full-text search. There are good log file analysis programs (for example, Splunk) that do a good job of searching through syslog files for well known patterns. Windows data is structured and in a much better form to be formally searched “by field” but it is less amenable to full-text searching.

The Windows event log offers the advantage of a remote API and structured data, but doesn’t provide a good mechanism for event log storage. With syslog, you can set up a syslog server that collects data from other machines that have been configured to send their logs to that server. The syslog server, then, creates a consolidated log file that contains information from multiple computers. Performing full-text searches on this file is even more useful. The Windows event log does not have this functionality. True, you can run Microsoft “MOM” (Systems Center Operations Manager) and accomplish this but only at great cost (money, time).

As weird as it sounds, some Windows sites run a program called Snare whose Windows version takes structured event log data, “flattens it”, and sends it to a central syslog server allowing it to be stored and analyzed using the same tools used for UNIX.

If the syslog server functionality is sounding cool to you, keep in mind that the standard syslog daemon only communicates to syslog servers using UDP protocol. UDP is unreliable so you can lose (important event) data. The data sent over UDP is also sent in clear text format so anyone can “sniff” the network and analyze it. The syslog server offers no security mechanism to control who can write to it. A rogue machine could spam it or could effect a denial-of-service-attack by flooding its incoming UDP socket.

There are newer versions of syslog, syslog-ng (next generation), for example, that overcome some of these limitations. syslog-ng, for example, can communicate to a syslog server using TCP instead of UDP. There are ways of hacking stunnel into the process to allow data to be SSL encrypted. The overall architecture, however, remains clunky and fragile.

Wouldn’t it be great if some clever company could offer a better solution? 😉

Wouldn’t it be great if you could write software that ran on Windows and non-Windows systems? Imagine if there was a programming language and a run-time system that supported object oriented programming and automatic garbage collection so that you could write software quickly and not have to worry about pesky memory pointer problems. Imagine, too, if that language came with an assortment of run-time libraries that let you write a variety of applications without having to call system-specific funtions. Wouldn’t it be great, too, if the language came accompanied by a first-class development environment with fast, incremental, compilation, a good editor and, most importantly, a great debugger?

Doesn’t exist. Sorry.

Java’s not it. Java promises to be this but it’s not. Here’s what’s wrong with it:

1. Sun, and the Java community in general, wants to be so operating system independent that it wants to repeatedly reinvent the wheel. Instead of using JNI (the Java Native Interface) to call what are, for all intents and purposes, standard libraries (in our case, OpenLDAP, gssapi, and others) Java brings its own implementations of these, written in Java. Unfortunately, this means that Java libraries are always slower, slow to implement new features and more broken than their native analogs.
2. Writing modern, web-based, Java applications is still dependent on which platform you choose to use: Tomcat, JBoss, WebSphere, Weblogic, Oracle Application Server, and more! All of these might be J2EE platforms but they’ve all their own unique architectures when it comes to security and other features.
3. The Java run-time is huge. In our case (a product I worked on a couple of years ago), no degree of fussing with the Java run-time would get its virtual memory footprint below 200Mb when you added the classes needed web services. The physical memory use was around 50Mb which was a ridiculously large amount of memory for what was essentially a simple management agent.
4. The Java UI classes (AWT, JFC/Swing) are complicated to use and result in applications that are uglier than their native counterparts. Admittedly, it is possible to write Java GUI apps that aren’t terrible (Eclipse, for example) but the fact that it’s so easy to identify Java apps by their contrast with native ones demonstrates the fundamental problem.
5. Java is disliked by Microsoft and Microsoft is disliked by Javans!

This last point is important enough that it warrants further study. Microsoft dislikes Java because it can’t control it. I worked at Microsoft for 11 years (many of them, in the Developer Tools group). Control of APIs and how programmers write software is the single most important asset that Microsoft has. As long as Microsoft owns the hearts (at least, the minds) of the world’s developers it wields massive influence over the software industry.  Microsoft will continue to develop new APIs, tools and goodies for the developer community. Writing web services is ridiculously easy with .NET. Writing forms based application with .NET is equally so. Nothing comes close.

The Java community will generally scoff at .NET and have its own way of doing things. It will take longer to come to market and it will be almost as good. Except, maybe, the UI will look a little off and the Web services implementation won’t completely interoperate with .NET because Microsoft chose to not follow some specification exactly right. For the Java folk, being right is more important than being useful.

So, along comes Miguel de Icaza. If you don’t know Miguel, you should read about him right now. Miguel is a very, very, smart guy. He’s also a fellow Latino — an extra-added bonus. Anyway, Miguel gets the idea to implement .NET on Linux. Since Microsoft made .NET an ECMA standard, they had to document it! Miguel and his team developed a C# compiler and the .NET CLR (Common Language Runtime). They also implemented several of the other Microsoft libraries (including the 1.1 version of the forms library). They did this several years ago and released the Mono project.

The goals of Mono/.NET are similar to those of Java – provide an object-oriented programming system with automatic garbage collection and a set of useful libraries. The advantage of Mono is that it builds on what Microsoft is developing on Windows. You can write your apps for Windows, using Visual Studio for example, and then run them on Linux with Mono (you don’t actually even need to recompile them since .NET compiles source code to intermediate form and it’s the run-time that performs that final native compilation). The Mono run-time is smaller than the Java run-time and since all the necessary HTTP classes are available in the run-time library, it’s easy to implement web-services and web applications without having to bring in Tomcat, JBoss or some other application server. There is a small sample application, xsp, that demonstrates this.

Alas, like Java, Mono, too, fails to deliver the “holy grail”.

Miguel’s company, Ximian, was bought by Novell in 2003. While it’s easy to criticize Novell for not supporting Mono to a greater degree (Miguel has a small team), this is unfair. It is not easy for Novell to “monetize” Mono. The little money it makes off the product it uses to fund Miguel’s team.

Mono works but…for some things better than others. The .NET 2.0 libraries are still not complete. The Winforms libraries are still somewhat fragile. If you don’t have fonts that you can ship with your application it’s difficult to get the same application to look good on both Windows and Linux. Debugging Mono apps is still much, much, more complicated than debugging .NET apps with Visual Studio. A cross-platform debugging solution that allows Visual Studio to perform remote debugging of Mono apps would be killer.

In spite of its limitations, we’ve made great progress with Mono and have been able to port code from Windows to Linux. We’re still strugging with the Mac version and have not had much luck with Mono on UNIX.

Some (well, the Java folk) scoff at Mono. They whine that C# ripped off Java. They say they like Eclipse or IntelliJ better than Visual Studio. They think that supporting .NET is inherently bad because it supports Microsoft. Microsoft loves that kind of talk. Think about it: the pro-Microsoft people write good looking applications, in a very productive manner, that run only on Windows. The anti-Microsoft people write crappy looking applications, in a less productive manner, that run on Linux and, begrudgingly, on Windows (but require installing the JDK and lots of standard-and-OS-independent-but-not-friendly twiddling around). The net result is that users associate Windows with good looking applications and associate portable applications with poor aesthetics and unfriendliness.

Mono is a great way to attract developers to non-Windows systems. Let’s hope that Novell continues to invest (and to increase its investment) in Mono so that it can deliver on its promise.

My company makes security software for UNIX, Linux and Mac OS X computers. Those of you not familiar with these operating systems might be asking why we don’t do this for Windows since it’s so often Windows that’s the subject of security flaws in the popular press. The answer is that is a bit more complicated than you might think.

A lot of our business is driven by security regulations. SOX, PCI, HIPAA and other laws/initiatives require that computers that are in scope employ adequate security measures. The cost of non-compliance can be signficant: shareholder lawsuits, criminal charges or, worst of all, increased credit card transaction costs. If a company fails to attain the top PCI rating, it can be forced to pay higher fees that can run into tens millions of dollars a year for large e-commerce firms.

A computer is considered in scope from the perspective of security regulations if it is involved in the processes covered by the regulation. For HIPAA, any system that stores patient data would be considered in scope. For SOX, any system that feeds data into corporate financial reports would be in scope.

It is not surprising to find many non-Windows systems in scope (or put another way, it is not surprising to find that a disproportionate number of in scope systems are running operating systems other than Windows). Since Windows is more frequently the target of hacker attacks and has more known exploits, companies like to use non-Windows systems for security critical applications, especially if the applications run “on the network edge” — they’re available on the Internet. By definition, e-commerce applications are on the network edge and PCI, particularly, brings a lot of non-Windows systems into regulatory scope.

Now here’s the kicker: for the most part, non-Windows systems employ terrible security practices!

What good is it that the operating system is well-written and tested if the root password is weak? How about if all the admins regularly login as root and, thus, all know its password? How about if adequate logging facilities aren’t available because everyone is logging in as root?

These hypothetical scenarios occur much more often than you think. Why? Because most companies do not have any centralized account management for their non-Windows systems.

Their Windows systems typically use Microsoft Active Directory (AD). Users login with their personal accounts and are granted special privileges by being made members of specific administrative groups. Files and programs are tagged with access control lists (ACLs) that only allow authorized users to access them. Windows and AD support group policy features that allow standardized security practices to be enforced throughout the corporation. Special audit ACLs can be defined to keep track of accesses to restricted resources.

These practices, for the most part, are completely alien to non-Windows systems. Although there are several LDAP directories (from Sun, IBM, Red Hat, Novell, others) that can be used for centralized account management, these are rarely implemented. Either they’re too expensive or they are too clumsy to use.

A few companies use NIS servers (developed by Sun) for this purpose but NIS is, itself, not secure and the “NIS replacement business” is also a healthy source of income for us.

There is no analog for group policy in the non-Windows world. Implementing standard security practices across all non-Windows systems usually means using manual processes and settling for the least common denominator solution. Some old versions of UNIX restrict passwords to 8 characters in length, for example, and disallow the use of punctuation marks. They may not provide any mechanism for forcing periodic password changes, for requiring strong passwords or for reusing recent passwords.

The Likewise Software solution is to connect these systems to Microsoft Active Directory and to implement a powerful group policy solution for non-Windows systems. We allow companies to connect over 110 flavors of UNIX, Linux and Mac OS X to AD and to use the same username and password on all these systems. We enable effective and efficient use of sudo to implement role-based access control and eliminate the broad use of root logins.

So, the next time you’re considering the security of your company’s computing infrastructure, look beyond the evening news and spend a little time thinking about how your own practices affect the vulnerability of your systems.