CS 596: Client-Server Programming
Spring Semester, 1997
Doc 25, Logging and Configuration Files

Doc 25, Logging and Configuration Files, Slide #1

Logging

Why logging?

• Performance tuning
• Upgrade justification
• Problem tracking
• Access counting

Who does logging?

• Client
• Server

Doc 25, Logging and Configuration Files, Slide #2

What should be logged?

Normally a log entry contains several pieces of information:

• Date and time
• Service that caused the entry
• Client address that caused the entry
• Host on which the server runs
• Event

Doc 25, Logging and Configuration Files, Slide #3

How should clients and servers log?

Basic choices:

• Appending to a log file
• System logging facility

Doc 25, Logging and Configuration Files, Slide #4

Appending to a log file

Creating a log entry: (Simple version)

1. Server opens log file for append
2. Server writes log entry
3. Server closes log file

Doc 25, Logging and Configuration Files, Slide #5

Simple appending problems

This simplistic approach can cause problems with the following:

• Concurrency: Two server processes or threads writing to the same logfile
• Performance: Every log entry requires lots of overhead (opening and closing the logfile)

Some solutions:

• Rely on the OS to perform atomic appends to the log file. (One unbuffered write per log entry)
• Do all logging in the parent process or a dedicated logging thread (Keep the log file open)
• Synchronize the writing to the log file.

Doc 25, Logging and Configuration Files, Slide #6

Use a system logging facility

Unix provides a flexible client-server based logging facility:

Unfortunately, syslog cannot be directly accessed from within a java program.

Also, the system administrator has to configure syslog, although any user can log to it.

Advantages of a system logging facility like syslog:

• Different logging levels and facilities
• It allows the syslog administrator to determine how much is logged and where. The logger does not
• have to be modified.
• Client-server based: The logs can be written on a different machine from where the logger runs.
• It solves any concurency problems.
• It solves most performance problems.
• It creates log entries in a fairly consistent fashion for all loggers.

Doc 25, Logging and Configuration Files, Slide #7

Syslog priorities

Priorities are encoded as facility and level.

Levels:

              LOG_EMERG       Panic conditions
              LOG_ALERT       Urgent things
              LOG_CRIT        Critical stuff
              LOG_ERR         Errors
              LOG_WARNING     Warnings
              LOG_NOTICE      Non-errors
              LOG_INFO        Information
              LOG_DEBUG       Debugging
      

Facilities:

              LOG_KERN        Kernel messages
              LOG_USER        Random user processes
              LOG_MAIL        The mail system
              LOG_DAEMON      System daemons
              LOG_AUTH        Login, su, getty
              LOG_LPR         Printing system
              LOG_NEWS        USENET news
              LOG_UUCP        Never used
              LOG_CRON        Cron system
              LOG_LOCAL0-7    Reserved for local use
      

Doc 25, Logging and Configuration Files, Slide #8

Syslog sample log entries

Log entry generated by sendmail through syslog:

Log entry generated by the printing system through syslog:

The format used depends on what the logs are going to be used for.

Note the use of name-value pairs in the sendmail log entry.

Doc 25, Logging and Configuration Files, Slide #9

Types of logs

Two basic log file types:

• Plain text
• Binary (record based, either fixed or variable length)

Plain text is preferable, but there are cases where record based logs are desirable:

• Reducing size of log entries
• Generating log entries at a rapid rate
• Reduce overhead
• Easy to read/parse by a log analyzing program.

Disadvantages of binary log files:

• Need a tool or program to actually look at the log file
• Errors in the log file itself may make the complete log file useless.

Doc 25, Logging and Configuration Files, Slide #10

Log file analysis

Raw log files are normally not very useful.

Log analysis tools can show information that cannot easily been seen by looking at the raw logs.

Example: SDSUanalyze

      Report of usage for the last 5 days in intervals of 60 minutes
      Time  Logins    Relative logins
      00:00 (123) *********
      01:00 (121) *********
      02:00 (121) *********
      03:00 (122) *********
      04:00 (122) *********
      05:00 (143) ***********
      06:00 (180) **************
      07:00 (252) *******************
      08:00 (491) **************************************
      09:00 (678) *****************************************************
      10:00 (678) *****************************************************
      11:00 (731) *********************************************************
      12:00 (767) ************************************************************
      13:00 (754) **********************************************************
      14:00 (655) ***************************************************
      15:00 (592) **********************************************
      16:00 (391) ******************************
      17:00 (244) *******************
      18:00 (220) *****************
      19:00 (178) *************
      20:00 (176) *************
      21:00 (176) *************
      22:00 (168) *************
      23:00 (126) *********
      8178 succesful logins, 31 unsuccesful logins
      

Doc 25, Logging and Configuration Files, Slide #11

Java log file I/O

Java doesn't have an explicit "append" file IO system.

Use the RandomAccessFile class.

For example:

      import java.io.RandomAccessFile;
      class AppendSomething
      {
        public static void main(String args[])
        {
          RandomAccessFile   file;
          try
          {
            file = new RandomAccessFile("foo",
                                        "rw");
            file.seek(file.length());
            file.write("Appended " + args[0] +
                                     "\n");
          }
          catch (Exception e)
          {
            System.err.println(e);
          }
        }
      }
    

Doc 25, Logging and Configuration Files, Slide #12

Runtime configuration files

What is a configuration file?

• A file which sets runtime attributes for a program.

Who needs a configuration file?

• Client
• Server

Kinds of information found in configuration files:

• Location of data and log files
• Port numbers to use
• Hostname to advertise
• Security parameters
• etc.

Doc 25, Logging and Configuration Files, Slide #13

Location of program parameters

Assignment of configuration parameters can be specied in different ways:

• Command line arguments
• Interactive prompts
• Parameters compiled into program
• Runtime configuration file

Issues to decide what to use:

• Permanence of the attributes
• Automatic startup at boot time (server)
• Ease of use

Doc 25, Logging and Configuration Files, Slide #14

Configuration file formats

Runtime configuration files need to be interpreted by a program.

Issues with the format:

• Does the file need to be human readable? (ASCII text vs. binary)
• Will a program generate the file?
• Will a program modify the file?

Best of both worlds: A configuration file which is human readable but also easy to read/modify by a program.

Many of the same rules that we have seen for protocol design apply to configuration files:

• The definition should be complete
• It should be extendable
• If possible, use name-value pairs
• ASCII text is easy to deal with (debugging, etc).
• Allow for comments.

Doc 25, Logging and Configuration Files, Slide #15

Two basic types of config files

There seem to be two main ideas with configuration files:

• Commands and parameters
• Name-value pairs (possibly sectioned off)

Some examples of "Good" configuration file formats

Microsoft .INI files:

	    [GROUP Identifier]
		name1=value1
		name2=value2
		etc.
	      

X11 defaults files:

	    class.application.name: value1
	    *.name: value2
	      

Fvwm configuration file:

	    AddToMenu "Utilities" "Utilities" Title
		+  "Recapture" Recapture
		+  "Paging" TogglePage 
		+  "Refresh" Refresh 
		+  "Windows" WindowList
	  

Doc 25, Logging and Configuration Files, Slide #16

Problems with some configuration files

Configuration programs need to be complete.

They should be able to do ALL possible things that the configuration file allows.

Some examples where this is not true:

• Solaris admintool
• Some Macintosh applications. (Need to use resedit to change some things)
• Windows 95/Windows NT programs that use the registry for special features