Starbeamrainbowlabs

Stardust
Blog

Automatically rotating log files on Linux

I'm rather busy at the moment with University, but I thought I'd post about Linux's log rotating system, which I've discovered recently. This post is best read as a follow-up to my earlier post, creating a system service with systemd, in which I talk about how to write a systemd service file - and how to send the output of your program to syslog - which will put it in /var/log for you.

Log rotating is the practice of automatically renaming and moving log files around at regular intervals - and keeping only so many log files at once. For example, I might define the following rules:

This would yield me a set of log files like this, for instance:

dpkg.log
dpkg.log.1
dpkg.log.2.gz
dpkg.log.3.gz
dpkg.log.4.gz
dpkg.log.5.gz
dpkg.log.6.gz
dpkg.log.7.gz
dpkg.log.8.gz
dpkg.log.9.gz
dpkg.log.10.gz

When the logs are next rotated, the last one is deleted and all the rest are renamed sequentially - like 10 in the bed.

Compressing log files is good for saving space, but in order to read them again we have to fiddle about with zcat / gzip.

The log rotating system on Linux is a cron job that runs at regular intervals - it doesn't run as a system service. It's configured by a series of files in /etc/logrotate.d/ - 1 for each service that has log files that want rotating automatically. Here's an example definition file:

/var/log/rhinoreminds/rhinoreminds.log {
    rotate 12
    weekly
    missingok
    notifempty
    compress
    delaycompress
}

Basically you specify the filename first, and then a bunch of directives to tell it what to do inside { }. The above is for RhinoReminds, an XMPP reminder bot I've written, and defines the following:

Very cool! This should produce the following:

/var/log/rhinoreminds/rhinoreminds.log
/var/log/rhinoreminds/rhinoreminds.log.1
/var/log/rhinoreminds/rhinoreminds.log.2.gz
/var/log/rhinoreminds/rhinoreminds.log.3.gz
/var/log/rhinoreminds/rhinoreminds.log.4.gz
/var/log/rhinoreminds/rhinoreminds.log.5.gz
/var/log/rhinoreminds/rhinoreminds.log.6.gz
/var/log/rhinoreminds/rhinoreminds.log.7.gz
/var/log/rhinoreminds/rhinoreminds.log.8.gz
/var/log/rhinoreminds/rhinoreminds.log.9.gz
/var/log/rhinoreminds/rhinoreminds.log.10.gz
/var/log/rhinoreminds/rhinoreminds.log.11.gz
/var/log/rhinoreminds/rhinoreminds.log.12.gz

Sources and Further Reading

Tag Cloud

3d 3d printing account algorithms android announcement architecture archives arduino artificial intelligence artix assembly async audio automation backups bash batch blog bookmarklet booting bug hunting c sharp c++ challenge chrome os code codepen coding conundrums coding conundrums evolved command line compilers compiling compression css dailyprogrammer data analysis debugging demystification distributed computing documentation downtime electronics email embedded systems encryption es6 features ethics event experiment external first impressions future game github github gist gitlab graphics hardware hardware meetup holiday holidays html html5 html5 canvas infrastructure interfaces internet interoperability io.js jabber jam javascript js bin labs learning library linux lora low level lua maintenance manjaro network networking nibriboard node.js operating systems own your code performance phd photos php pixelbot portable privacy problem solving programming problems project projects prolog protocol protocols pseudo 3d python reddit redis reference releases resource review rust searching secrets security series list server software sorting source code control statistics storage svg talks technical terminal textures thoughts three thing game three.js tool tutorial twitter ubuntu university update updates upgrade version control virtual reality virtualisation visual web website windows windows 10 xmpp xslt

Archive

Art by Mythdael