How-to automatically filter and file your syslog

For me, there were two main problems with FREESCO's /var/messages file. Firstly, the important stuff was logged among a whole bunch of useless (or just really wordy) stuff, and that made it a pain to find things at times. Secondly, the logs are lost when you reboot.

Here's what I did about it.


All you need is tail. I actually built tail from source on my Slackware devel machine and then copied it to FREESCO, but I believe you can get it in dingetje's util package or something like that.

FIXME In which package can you find tail?


The key is a shell script (which I have called /usr/bin/logprocess) to which we feed log lines. It searches them for keywords and then writes them to a certain file.


This shell script simply passes any new lines coming into the syslog onto /usr/bin/logprocess.


/usr/bin/tail -f /var/messages | /usr/bin/logprocess

I used a script because I couldn't figure out how to include a pipe in a fork subcommand, so instead I just forked this shell script.


And now for what to do with all that text!

This is the way I did it:



  echo "$LINE" >> /boot/log/ftp/$(date +%y%m.log)

  echo "$LINE" >> /boot/log/ssh/$(date +%y%m.log)

  echo "$LINE" >> /boot/log/fw/$(date +%y%m.log)

  echo "$LINE" >> /boot/log/misc/$(date +%y%m.log)

until false; do
  read LINE
  if [ -n "$(echo $LINE | grep ftpd)" ]; then
  elif [ -n "$(echo $LINE | grep sshd)" ]; then
  elif [ -n "$(echo $LINE | grep fw)" ]; then

Obviously you could get the same effect without all the functions, but I wanted to have the flexibility to further filter each type of log item, for instance if I'm running a program that generates firewall entries that I don't need to record, I can just throw away those entries in the fw function.

$(date +%y%m.log) generates a filename based on the year and month; e.g. in January 2007 the filename 0701.log would be used.

You can adjust all this to suit your needs.

Finishing touches


fork /usr/bin/logdispatch

to /rc/rc_user's start section.

Make sure the two scripts are executable, for instance by typing

chmod u+x /usr/bin/logdispatch /usr/bin/logrotate

Make sure the directories you have chosen (e.g. /boot/log/ftp etc.) exist. Also ensure that they are preserved across a reboot.

freesco/howtos/log_sort.txt (135876 views) · Last modified: 2007/03/10 23:45 (external edit)
Recent changes RSS feed Creative Commons License Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki