Table of Contents

Syslog service

This work was done based on requirements on the task: http://www.freelancer.com/projects/Delphi-Software-Architecture/Windows-Log-Syslog-Service.html

The project was done during 1 month. This is twice longer than the bid I did, but there was a lot of low-level WinAPI details. This is more low-level windows-based work than the Delphi native code. Therefore the project was remade 2 times. I refined their classes several time to come for better, reliable, clear and to be more understood results.

Ok. Here I want to describe this work in details. How it's working, correct, or, not.

Let's go. Main parts:

The Eventlog monitoring block

This block is working based on universal 'eventlog' unit, described here

Main features:

Ok. This block creates the list of messages in the internal proprietary format. These message ready to be sending out inside “Transaction”.

How it's working:

Every known branch (“Application”, “Sytem”, and “Security”) has the variable “last_seen” – TDateTime, this is the date/time when this event was added into the windows eventlog facility. Everytime the block read all the messages from the log, and take only those that have the generated time >= “last_seen” time.

These messages are added into the list, and the “last_seen” variable is updated by the value of last generated message (not “now” because when the program is working newer messages could be generated).

Also when the program takes messages – it check the selected messages parameters based on http://msdn.microsoft.com/en-us/library/windows/desktop/aa363648(v=vs.85).aspx (or any other)

After that the message is converted into the simple string line on the next pattern:

'record number: ' + IntToStr(Number) + #9 + 'source: ' + SourceName + #9 + 'message: ' + event_message + #9 + 'event id: ' + IntToStr(Code) +#9 + 'event type: '+IntToStr(EventType);

Where

Example of the message placed into the output text log file:

record number: 6604	source: Security	message: The description for Event ID ( 593 ) in Source ( Security ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. The following information is part of the event: 3820C:\WINDOWS\system32\arp.exesysadminSERVER(0x0,0x12995)	 event id: 593	 event type: 8

The Files monitor

This part do the monitoring of the files based on the list of files added with help of the The interface part

We apply the next common parameters for all the files together:

How it's working:

  1. On the program start – read 'last_size' variable from the registry
  2. If the 'Skip all lines on startup' is ON – set 'last_size' into the current size in bytes of every files
  3. Every time after the program read messages from the eventlog facility – it go to the list of monitored files, and check the size of the file – if it greater than 'last_size'
  4. It open the file and read all bytes after the 'last_size' position into the binary 'buffer'
  5. Then it split 'buffer' into usual lines by split delimiter = [#10, #13]
  6. Store 'new_last_size' for every monitored file

On successful Transaction

  1. Assign all 'new_last_size' into the 'last_size' variables for every monitored file
  2. Save new file sizes the into the registry

The monitored scheme

The communication part

UDP / TCP communications with the syslog service. Also here's email sending block on TCP errors.

Used components – Indy TIdSyslog for UDP syslog communication, and TIdTCPClient – for TCP syslog communication. TIdSMTP – for simple email sending procedure based on simple communication with an external SMTP-server.

Since this part was not described in the initial technical requirements – it was made with minimum features using the well-known indy project.

UDP Connection

This is the simplest variant to communicate with Syslog service, and not reliable as well – messages could be omitted due protocol realization.

TCP Connection

Better way. The connection will be established and be more reliable. Also transaction will be not completed when the TCP connection will fail or broken, therefore syslog service will wait until the connection will success or it'll send out mail message about failed connection and stop the monitoring.

The debug logging messages

Is working when the 'debug.txt' file is exists in the directory with the program. Needed for testing / debugging purposes.

File 'debug.txt' – being exists in the directory with the service will enable the service to write out the file 'yeti_debug#####.txt' in the append mode where it'll log every points in/out into procedures to have a log. Where ##### is a timestamp when the program was started.

The interface part

There is a set of 3 user friendly forms based on the additional customer requirements.

First screen

Second screen

Third screen

Forth (debugging) screen

Also the application has an icon in the windows tray and simple popup menu with next items:

The interface part read the status of the service part (the running status is shown in the interface part header). It can also start or stop it.

The service part

Hidden service software

InnoSetup installer

The *.iss source that compile the program into self-installed software.

Concepts

In block descriptions we defined some things, here's their descriptions.

Transaction

The process of sending out of taken data. On success – it update the variables and the registry. Otherwise – send out email (for TCP 2 syslog monitoring)

When the service read messages from the eventlogNT, and lines from monitored files – it should send them out to syslog. We can define this process as a “transaction”. If transaction fail – the program will not update the Last_events[k] and lastsize[k] variables.

After 'try_cnt' fails – it sends out an email about this on defined email if any.

Successful – the program update the values:

Last_events processing

All messages from the eventlogNT are read in the reverse order (from latest back to oldest), result – the TStringList variable. On successful transaction – it takes the DateTime “Generated” value from the latest message and place it into the “Last_events” variable.

Last_size processing

Ok, as I mentioned before, every monitored file has the last size – the length of the file in bytes. The program is thinking about the file size is grows up. Every time the program open the file, seek into the position of previous size, and read all new bytes into the “buffer” in the binary mode. The it split this buffer into TStringList variable by #10 and #13 (usual windows line breaks).

In the case if last_size less then the current size – the program thinks about the monitored file was rewritten from scratch and read all lines from their beginning.

Options in Registry

The program keeps all the options and work variables in the windows registry. Both Service block and the Interface block use the same registry branch.

Here's the description of every parameter in registry.

General settings in the registry

Units short description

syslog_pref

Contain two classes that read data from registry into the properties.

Also it save changed variable back into registry.

This unit is used by both parts – by the interface and the service.

syslog_interface

The interface form that consist of 3 pages based on initial 3 gifs.

Runtime_service_unit

The service unit

syslog_writer

Used as syslog facility, email about bugs, etc. For now – it write messages into the windows/temp directory, file “test_output.txt”. (note: in Win 2008 R2 – a 'temp' directory is under /users/…/application data/)

Usage

Interface part

Interface part contains one form with 3 tabs and the popup menu that used to start/stop service, show About page and exit.

All changes are saved into the registry.

Service part

Service part is working in hidden (system) mode. It uses the settings from registry. If they were not added – it uses the settings by default.

By default: ipv4; UDP:127.0.0.1:1468; all checkboxes are checked; the program use the testing smtp-service for email connection.

Troubleshootings

TCP connection unavailable

In the case the TCP is not working, and the service send an email about it. Then the service stop their work because of no destination with an appropriate message into eventlog. To turn it ON – change the settings and start the service.

Any other bug

Create the file 'debug.txt' (any size) and place it into the service directory (by default “C:\Program Files\Yetidi syslog”). Now during the service work – it'll write the temporary file “test_output.txt” into “C:\Windows\Temp” (or another place based on system settings). Also every bug will be written into the file “log_file.txt” in the windows temporary directory.