If our application has a lot of traffic, our log files can grow fast and reduce our free disk usage.

For my case, one of log file from my application is 400MB in size.

I use Logrotate to manage these log files in Linux.

Logrotate is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.

By default Logrotate is installed on Ubuntu Linux.

logrotate -v

If Logrotate is not installed, it can be installed via apt-get.

sudo apt-get install logrotate

Logrotate Configuration

Default configuration for the Logrotate is available in /etc/logrotate.conf. But for application-specific configuration (to override the defaults) are kept under /etc/logrotate.d/ directory.

Here is example configuration from my Rails application log files inside /etc/logrotate.d/exampleapp.

/var/www/exampleapp/shared/log/* {
  size 100M
  monthly
  rotate 9
  compress
  delaycompress
  missingok
  notifempty
  copytruncate
  su root root
  create 644 exampleuser exampleuser
}

Configuration explanation:

  • size 100M - rotate the log file once the file size reaches 100MB.
  • monthly - rotation interval, monthly means rotate the log file monthly.
  • rotate - only keep 9 days of logs.
  • compress - GZip the log file on rotation.
  • delaycompress - delay the compression process till the next log rotation.
  • missingok - avoid halting on any error or ignore if the log file doesn’t exist.
  • notifempty - avoid log rotation if the logfile is empty.
  • copytruncate - copy the log file and then empties it. This makes sure that the log file Rails is writing to always exists so our application won’t get problems because the file does not actually change. If you don’t use this, you would need to restart your Rails application each time.
  • su root root - because /var director is owned by root, Logrotate need to be run as root.
  • create 644 exampleuser exampleuser - create log files with user and group as exampleuser and use 644 as file permission.

Note that if both size and rotation interval are set, then size is taken as a higher priority. Example:

size 100M
monthly

From configuration above, once the file size reaches 100M the file will be rotated and not wait for the monthly cycle.

Checking and Running Logrotate

To check our configuration, read the /var/lib/logrotate/status file. If our log files are rotating, then it will be displayed.

cat /var/lib/logrotate/status

Example result:

"/var/www/exampleapp/shared/log/production.log" 2019-6-16-1:10:52
"/var/www/exampleapp/shared/log/puma_access.log.1" 2019-6-16-1:10:52

By default Logrotate will be run by Cron Job. To run logrotate manually, use the /usr/sbin/logrotate command. For example if we want to run configuration for exampleapp.

sudo /usr/sbin/logrotate -f /etc/logrotate.d/exampleapp

Example result:

production.log production.log.1 production.log.2.gz

References