Automatically rotating log files on Linux
I'm rather busy at the moment with University, but I thought I'd post about Linux's log rotating system, which I've discovered recently. This post is best read as a follow-up to my earlier post, creating a system service with systemd, in which I talk about how to write a systemd service file - and how to send the output of your program to syslog - which will put it in /var/log
for you.
Log rotating is the practice of automatically renaming and moving log files around at regular intervals - and keeping only so many log files at once. For example, I might define the following rules:
- Rotate the log files every week
- Keep 10 log files in total
- Compress log files past the 2nd one
This would yield me a set of log files like this, for instance:
dpkg.log
dpkg.log.1
dpkg.log.2.gz
dpkg.log.3.gz
dpkg.log.4.gz
dpkg.log.5.gz
dpkg.log.6.gz
dpkg.log.7.gz
dpkg.log.8.gz
dpkg.log.9.gz
dpkg.log.10.gz
When the logs are next rotated, the last one is deleted and all the rest are renamed sequentially - like 10 in the bed.
Compressing log files is good for saving space, but in order to read them again we have to fiddle about with zcat
/ gzip
.
The log rotating system on Linux is a cron job that runs at regular intervals - it doesn't run as a system service. It's configured by a series of files in /etc/logrotate.d/
- 1 for each service that has log files that want rotating automatically. Here's an example definition file:
/var/log/rhinoreminds/rhinoreminds.log {
rotate 12
weekly
missingok
notifempty
compress
delaycompress
}
Basically you specify the filename first, and then a bunch of directives to tell it what to do inside { }
. The above is for RhinoReminds, an XMPP reminder bot I've written, and defines the following:
- Keep 12 log files in the rotation cycle
- Rotate the logs every week
- It's ok if the log file doesn't exist
- Don't rotate the log file if it's empty
- Compress log files on rotation if they aren't already
- .....but delay this by 1 rotation cycle
Very cool! This should produce the following:
/var/log/rhinoreminds/rhinoreminds.log
/var/log/rhinoreminds/rhinoreminds.log.1
/var/log/rhinoreminds/rhinoreminds.log.2.gz
/var/log/rhinoreminds/rhinoreminds.log.3.gz
/var/log/rhinoreminds/rhinoreminds.log.4.gz
/var/log/rhinoreminds/rhinoreminds.log.5.gz
/var/log/rhinoreminds/rhinoreminds.log.6.gz
/var/log/rhinoreminds/rhinoreminds.log.7.gz
/var/log/rhinoreminds/rhinoreminds.log.8.gz
/var/log/rhinoreminds/rhinoreminds.log.9.gz
/var/log/rhinoreminds/rhinoreminds.log.10.gz
/var/log/rhinoreminds/rhinoreminds.log.11.gz
/var/log/rhinoreminds/rhinoreminds.log.12.gz