Consuming Fastly Logs

In this guide we're going to:

1. Setup fastly to transport logs to a linux server with TLS configured.
2. Set up CrowdSec on the log server to consume Fastly logs.

Transport fastly logs to linux server:

Configuring Rsyslog with TLS

To receive logs from Fastly, you'll need to generate server and client certificates (the server certificate for machine which receives logs and client for Fastly). See this guide on how to do this.

Configure rsyslog server on crowdsec

SH

vim /etc/rsyslog.conf

VCL

global(
  defaultNetstreamDriverCAFile="/etc/pki/ca.crt"
  defaultNetstreamDriverCertFile="/etc/pki/fastly.dev.crowdsec.net.crt" # Replace this with path to cert
  defaultNetstreamDriverKeyFile="/etc/pki/fastly.dev.crowdsec.net.key" # Replace this with path to key
)

module(
  load="imtcp"
  streamdriver.name="gtls" # use gtls netstream driver
  streamdriver.mode="1" # require TLS for the connection
  streamdriver.authmode="x509/certvalid" # accept with valid cert  
)

input(
  type="imtcp"
  port="4242"
)

Add new config file so it will be processed as final /etc/rsyslog.d/99-crowdsec.conf

TEXT

template RemoteLogs,"/var/log/crowdsec_fastly.log"

if $hostname == 'ip-172-31-40-44' then ~
*.* ?RemoteLogs
& ~

We configure rsyslog to ignore local syslogs and keep only remote syslog. Then we send them to /var/log/crowdsec_fastly.log

Install crowdsec with fastly collection

On the same machine, install CrowdSec as described in the getting started guide

Setup acquisition

Append this config to the file /etc/crowdsec/acquisition.yaml

YAML

---
filename: /var/log/crowdsec_fastly.log
labels:
  type: syslog
  external_format: fastly

Install fastly collection

Install the fastly collection via:

SH

sudo cscli collections install crowdsecurity/fastly

Reload CrowdSec

TEXT

sudo systemctl reload crowdsec.service