Both Windows and UNIX (and Linux and Mac OS X) support some type of event log. The event log consists of a set of timestamped entries that document significant operating system or application activity. Events might be strictly informational (Successful logon for Joe) or might document errors (Unsuccessful logon for Joe or Out of disk space!)
As you might imagine, event logs are important. They let help system administrators in several ways:
- Problem diagnosis. When a computer is misbehaving, administrators can look at its event log to see if there are messages in it that provide clues to what’s broken.
- Security enforcement and intrusion detection. Event logs contain information that can be used to detect intrusion attempts and unauthorized access to protected resources.
- Auditing. Event logs contain information that helps administrators keep track of who is accessing what systems and what they’re doing while they’re there. This information can be used to satisfy security auditors that adequate practices are in place.
Although both UNIX and Windows have event logs, their mechanisms are very, very, different. The Windows event log consists of structured data with explicit fields for “Type”, “User”, “Computer”, “Category” and other fields. Windows provides an API that allows event logs to be read from local or remote computers. The API allows monitoring programs to securely access event information from Windows computers.
The UNIX event log is stored in text files. The syslog API is typically used to write to the event log text files but syslog does not support remote access of UNIX event logs. Although syslog provides for a facility code and a priority field in its API, the data actually stored in the text files is mostly unstructured.
Both mechanisms have their problems.
Although syslog data is unstructured, it is in a great form for full-text search. There are good log file analysis programs (for example, Splunk) that do a good job of searching through syslog files for well known patterns. Windows data is structured and in a much better form to be formally searched “by field” but it is less amenable to full-text searching.
The Windows event log offers the advantage of a remote API and structured data, but doesn’t provide a good mechanism for event log storage. With syslog, you can set up a syslog server that collects data from other machines that have been configured to send their logs to that server. The syslog server, then, creates a consolidated log file that contains information from multiple computers. Performing full-text searches on this file is even more useful. The Windows event log does not have this functionality. True, you can run Microsoft “MOM” (Systems Center Operations Manager) and accomplish this but only at great cost (money, time).
As weird as it sounds, some Windows sites run a program called Snare whose Windows version takes structured event log data, “flattens it”, and sends it to a central syslog server allowing it to be stored and analyzed using the same tools used for UNIX.
If the syslog server functionality is sounding cool to you, keep in mind that the standard syslog daemon only communicates to syslog servers using UDP protocol. UDP is unreliable so you can lose (important event) data. The data sent over UDP is also sent in clear text format so anyone can “sniff” the network and analyze it. The syslog server offers no security mechanism to control who can write to it. A rogue machine could spam it or could effect a denial-of-service-attack by flooding its incoming UDP socket.
There are newer versions of syslog, syslog-ng (next generation), for example, that overcome some of these limitations. syslog-ng, for example, can communicate to a syslog server using TCP instead of UDP. There are ways of hacking stunnel into the process to allow data to be SSL encrypted. The overall architecture, however, remains clunky and fragile.
Wouldn’t it be great if some clever company could offer a better solution? 😉