This is not a scenario that any real organisation or enterprise would be likely to face, but I wanted to find a way to use a Raspberry Pi as a log forwarder for Azure Sentinel. This will help me collect logs from the home network, especially from my Ubiquiti EdgeRouter firewall.

Since neither of the usual suspects for syslog forwarding - Logstash and Log Analytics Gateway - support the Pi or any ARM based Linux, I needed to find something new for this case. Luckily there is a third option that we can use: Fluentd.

Fluentd is a modular data collector that has input, output and filtering plugins for many different use cases, so in many ways it is similar to Logstash. And like Logstash, Fluentd has syslog input and Log Analytics output plugins and thus is capable of acting as a log forwarder to Sentinel.

Fluentd is also very simple to install and configure on a Pi running Raspbian OS. So far I’ve had no performance issues or problems with it. Below is a short description of the install and configuration process.

As a bonus there is some info here on parsing Syslog data from the EdgeRouter, in case someone is interested in that as well.

Installing Fluentd

With the following commands we install Ruby, latest version of Fluentd and the Log Analytics output plugin:

$ sudo apt install ruby-full
$ sudo gem install fluentd -v "~> 0.13.0"
$ sudo fluent-gem install fluent-plugin-azure-loganalytics

Configuring Fluentd for Syslog input and Log Analytics output

You can configure Fluentd by creating a new file at /etc/fluentd.conf.

Below is my example configuration, containing a Syslog listener, filterer and parser for the EdgeRouter logs and a forwarder to push the messages to Log Analytics workspace and Sentinel.

As you can see, the message handling in Fluentd is based on tags and a top-down process.

# Syslog listener for EdgeRouter hardcoded Syslog port, udp 514: 
<source>
  @type syslog
  port 514
  bind 0.0.0.0
  tag edgerouter
</source>

# This filter makes sure we only process EdgeRouter messages that contain network traffic:
<filter edgerouter.**>
  @type grep 
  <regexp>
    key message
    pattern /PROTO=/
  </regexp>
</filter>

# This filter parses the EdgeRouter log message data to Log Analytics columns:
<filter edgerouter.**>
  @type parser
  key_name message
  <parse>
    @type regexp
    expression /.*?IN=(?<inIf>[^ ]*) .*?OUT=(?<outIf>[^ ]*) .*?SRC=(?<srcIP>[^ ]*) .*?DST=(?<dstIP>[^ ]*) .*?PROTO=(?<proto>[^ ]*) .*?SPT=(?<srcPort>[^ ]*) .*?DPT=(?<dstPort>[^ ]*)/
 </parse>
</filter>

# This segment forwards the messages to two destinations:
# 1) stdout for debugging (can also be written to a file as shown later)
# 2) Log Analytics and Azure Sentinel (custom table named UbiEdgeRouter_CL)
<match edgerouter.**>
  @type copy
  <store>
    @type stdout
  </store>
  <store>
    @type azure-loganalytics
    customer_id [Log Analytics Workspace ID]
    shared_key [Log Analytics Workspace Key]
    log_type UbiEdgeRouter
  </store>
</match>

Getting the logs flowing

Starting Fluentd, with parameters to select our config file and log destination:

# /usr/local/bin/fluentd -c /etc/fluent.conf -o /var/log/fluent.log

You can follow the log file to make sure things are working as intended. The log destination config and startup options can be removed later when things run smoothly, so the logs will not fill up the Pi SD card.

As seen below, we now have the EdgeRouter logs forwarded to Sentinel: