Evaluation and Comparison of different logging Systems

Based on mantis ID 62 this article contains a report of a comparison of three different logging systems in order to preprae a relevant developement decision. The three different systems are:

1. The openpetra internal logging system called TLogging
2. The Log4Net system from Apache
3. The nLog system

In the very first step only the documentation is refered because this has led to a predecison against nLog. In the next step some log4net benchmarks has been done in order to get some experience with log4net.

Comparison of nlog and log4net

First Impression

Both systems ([Apache log4net] and [nLog]) are something similar. Both systems provide an Interface to [Chainsaw] and both systems provide a message system to a remote instance. Both systems handle the log-levels DEBUG, INFO, WARN, ERROR and FATAL and both systems can handle an output to a file or a set of files, to a database or to a xml-report or else and simultaneusly.

Furthermore both systems can be configured by external xml-files and programmatically. And of course both systems can be used by mono. In Mono we cannot use the log4net.Appender.ColoredConsoleAppender but the log4net.Appender.ConsoleAppender (I think that is something we can live with), and it makes no sense to use the NetSendAppender and the OutputDebugStringAppender. The NetSendAppender writes logging events to the Windows Messenger service and this only can be used by using a .net framework and the OutputDebugStringAppender sends the message to a Debugger. This interface actually is used by Microsoft Software only.

The OutputDebugStringAppender may be an interesting option for troubleshooting. May be we can switch between frameworks ...

NLog itself does not provide such an easy feature overview. Referring [this page] you can find different versions of nlog. If you open either the [Version for mono] and [for .net 4.0] you will see the problem. You have to study the documentation in detail to see what is going on and what is not going on.

The apache solution is not a standalone solution but [one of four logging services] for java, c++, c# and php and so experience on one of thoose frameworks will help to understand the others.

Referring to the [Apache-FAQ] there is a set of requirements for a logging system.

Base Requirements

Reliability

Reliability means that a system logs something before an action is performed. Typical solutions are the log files of a web server. If the process does not have the privilege to write something to a file an error message is shown and the site is not delivered to the client.

This means that an Exception has been thrown if the logging does not work. A troubleshooting log system mustn't fulfill such an requirement. It's not a good idea to troubleshoot your software and find a problem in your logging system only.

1. log4net is not reliable. it reclaims to be a "best-effort and fail-stop logging system"
2. nlog does not provide any information about it

Thread Safetyness

Both systems are thread safe.

Example for detailed differences: Remote-Syslog

I've not investigated all differences but only three and I've described it only for one example because the following differences are typical.

Apache knows two Remote-Classes, [the RemotingAppender] and the [RemoteSyslogAppender].

The first provides a communication to a client providing a [IRemoteLoggingSink-Interface] and the second a very common and convenient udp-Interface. The documentation is reduced down to the most important facts.

The nlog provides "Targets" and among a set of targets there exists a DatabaseTarget Class and a DatabaseParameterInfo Class. But the Database parameters itself are not a target. They are used to define a specific database target. So the hierarchical code structure is something strange.

First Decision between nLog and log4net

In this case the decision is relative simple. We'll take log4net.

Log4net benchmarks

The benchmark tests

Basic idea of this test is to get some ideas about the using of the system, and the relative performance to TLogging. The basic question is: Makes it sense to change or not?

The openpetra sources contains source tree (csharp\ICT\Testing) which handles different software tests. Inside of this tree. A class is added to csharp\ICT\Testing\Common\Logging.Benchmark.Test.cs. This test is not a unit test in a convenient way because no assert is realy done. Insted of this the System.Diagnostics.Stoppwatch is used to measure the time which is necessary to log 10.000 lines one by another.

The base routine is:

stopwatch.Start();
 // Do 10.000 Logs
stopwatch.Stop();
// Log the time in Millisecons

The different tests are:

TLogging

for (int i=0; i<10000; i++)
  {
    TLogging.Log("Entry: " + i);
  }

Log4NetDirect

 for (int i=0; i<10000; i++)
   {
     log.Debug("Entry: " + i);
   }

LogOnLevelDependency

 for (int i=0; i<10000; i++)
   {
     if (log.IsDebugEnabled) log.Debug("Entry: " + i);
   }

UseConsole

 for (int i=0; i<10000; i++)
   {
     System.Console.WriteLine("Entry: " + i);
   }