How to configure remote logging on Linux using rsyslog

Logging is an essential aspect of managing Linux servers. Log messages are useful for root cause analysis and to prevent potential error occurrences in the future. Analyzing and debugging server errors is an essential skill for both IT engineers and system administrators.

This guide will show you how to set up a remote logging server, also known as a logging host, on Linux. A log host allows you to aggregate local Linux logs to a remote centralized server for easy access and analysis.

Why have a dedicated logging server?

The Linux operating system logs most activity on your server for auditing and debugging purposes using the syslog (system logging protocol) daemon. So you might be wondering why do I need a dedicated server for my logs? Here are some benefits of having a dedicated logging server:

  • Better security because the remote logging server has only a few ports open to the outside.
  • Improved server performance because the remote logging host does not run many services except those used for logging.
  • Facilitates archiving and management of log messages.

Log messages are important for auditing your servers and SEO and are an integral part of preventive maintenance procedures for your server infrastructure.

Step 1: Installing rsyslog on Linux

This guide focuses on Ubuntu 20.04, but the process should be much the same if you’re using other mainstream Linux distributions.

rsyslog is a remote logging service for Linux and comes pre-installed by default on most modern Linux distributions, for example, Ubuntu and other Debian-based systems.

USE VIDEO OF THE DAY

The rsyslog service is a modern and improved syslog daemon, which only allows you to manage logs locally. With the rsyslog daemon, you can send your local logs to a configured remote Linux server.

If you don’t have rsyslog installed on your PC, you can easily do so using the following command, on Debian-based distributions:

sudo apt install rsyslog

On Red Hat Linux, you can install it by typing:

yum install rsyslog

On Fedora and its derivatives, run:

dnf install rsyslog

To install rsyslog on Arch Linux:

yay -S rsyslog

To check the status of rsyslog, run the following command:

systemctl status rsyslog

Exit:


rsyslog_status

Step 2: Configuring the Log Host Server

The log host is the server configured to receive log messages from other servers or PCs. The rsyslog configuration resides in the /etc/rsyslog.conf file.

You can open the /etc/rsyslog.conf file using any text editor of your choice. In this guide, we will be using Vim.

You will need elevated privileges to make changes to the configuration file.

Before you start editing the configuration file, you should make a backup or copy of the file. To do this, run the command:

sudo cp /etc/rsyslog.conf /etc/rsyslog_original.config

Then open the /etc/rsyslog.conf file using a text editor.

sudo vim /etc/rsyslog.conf 

There are two protocols you can use to send/receive log files with rsyslog: TCP and UDP. This guide shows you how to configure both.

You don’t need to configure both UDP and TCP for remote logging to work. Choose only one of the two.

If you prefer to use UDP, find and uncomment the following lines by removing the beginning Grind (#) symbol preceding the lines. You can find these lines in the modules section of the configuration file.


module(load="imudp")
input(type="imudp" port="514")

If you prefer to use TCP, uncomment the following lines by removing the beginning Grind (#) symbol located at the beginning of the line:

module(load="imtcp")
input(type="imtcp" port="514")

The following figure shows the rsyslog configuration file configured to use UDP communication:


udp_configuration_rsyslog

Next, configure where rsyslog will store your logs. For better organization, you should classify incoming logs according to their origin. Define a pattern in your rsyslog configuration file by adding the following lines:

$template remote-incoming-logs, "/var/log/remote/%HOSTNAME%".log
*.* ?remote-incoming-logs

The aforementioned lines command rsyslog to store the logs in the folder /var/log/remote/hostnameor host name is the name of the remote client that sends log messages to the log host.

Now save the changes you have made. If you’re using Vim, here’s how to save and exit a file.

Finally, restart the rsyslog services for the changes you made to take effect.

sudo systemctl restart rsyslog

Step 3: Configure your firewall

If your firewall is enabled, make sure the port you configured above is able to communicate with the outside world. You will need to modify your firewall rules to allow incoming logs.

For Debian-based distributions, just use the UFW tool to enable UDP or TCP transfer protocol.

Related: How to Configure Firewall in Ubuntu Using UFW

If you are using UDP, run the following command, where 514 is the configured port number:

sudo ufw 514/udp

If you are using TCP on port 514, just run:

sudo ufw 514/tcp

On Fedora you can use firewall-cmd to get similar results.

firewall-cmd --zone=zone --add-port=514/udp

For Red Hat Linux, open the iptables file located at /etc/sysconfig/iptables using the text editor of your choice and add the following rule:

-A INPUT -m state --state NEW -m udp -p udp --dport 514 -j ACCEPT

Restart the iptables service for the changes to take effect.

service iptables restart

Step 4: Configuring the logging client

The client is the machine that sends its logs to a remote or centralized log host server. Open the rsyslog configuration file located at /etc/rsyslog.conf:

sudo vim /etc/rsyslog.conf

Add the following line if using UDP, where 192.168.12.123 is the IP address of the remote server, you will write your logs to:

*.* @192.168.12.123:514

If you are using TCP, add the following line instead. Note that the line has two @ symbols.

*.* @@192.168.12.123:514

Save your changes and restart the rsyslog service on the client with the command:

sudo systemctl restart rsyslog

Step 5: Viewing log messages on the server

You can use SSH to connect to your remote server and view logs sent from client servers. In this case, rsyslog is configured to store client logs in the /var/log/remote remote server directory.

cd /var/logs/remote

Next, list the contents of the directory using the ls command:

ls -l

As you can see in the output, the directory contains log messages for remote servers named andiwa and rukuru. Their log files are named andiwa.log and rukuru.log respectively.


logs_on_remote_log_host

You can then view the log files using a text editor or with Linux file viewing tools such as cat or less.

Remote logging gives you more control

This guide looked at how to set up a remote logging server (logging host) on Linux.

A log host gives you better organization and control when it comes to logging. Even in scenarios where a system is damaged or inaccessible, you can still view its logs from the log host and determine what went wrong.


two computer screens symbolizing system logging

Getting Started with System Logging in Linux

Read more


About the Author

About Jon Moses

Check Also

Intel promises “substantial contributions” to the growth of RISC-V • The Register

Analysis Here’s something that would have seemed odd just a few years ago: to help …