Skip to content

Acquisition

Info

Please note that the /etc/crowdsec/acquis.yaml should be auto generated by the wizard in most case.

The acquisition configuration specifies lists of logs that crowdsec-agent will ingest and feed to parsers. Acquisition provides two information about a given log :

  • its source (a path to a file, or a journalctl filter)
  • its type, given in the form of a label

The type label is crucial as it's later used in the process to determine which parser(s) can handle lines coming from this source.

Acquisition can be found in /etc/crowdsec/acquis.yaml, for example :

Acquisition example

filenames:
  - /var/log/nginx/access*.log
  - /var/log/nginx/error.log
labels:
  type: nginx
---
filenames:
  - /var/log/auth.log
labels:
  type: syslog
---
journalctl_filter:
 - "_SYSTEMD_UNIT=ssh.service"
labels:
  type: syslog

Testing and viewing acquisition

At startup

At startup, you will see the monitored files in /var/log/crowdsec.log :

...
INFO[23-11-2020 15:21:17] [file datasource] opening file '/tmp/test.log' 
WARN[23-11-2020 15:21:17] [file datasource] no results for /tmp/ratata.log 
INFO[23-11-2020 15:21:17] [journald datasource] Configured with filters : [--follow _SYSTEMD_UNIT=ssh.service] 
...

At runtime

cscli allows you to view crowdsec-agent metrics info via the metrics command. This allows you to see how many lines are coming from each source, and if they are parsed correctly.

You can see those metrics with the following command:

sudo cscli metrics
cscli metrics example
$ sudo cscli metrics
...
...
INFO[0000] Acquisition Metrics:     
+--------------------------------------+------------+--------------+----------------+------------------------+
|                SOURCE                | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+--------------------------------------+------------+--------------+----------------+------------------------+
| /tmp/test.log                        |         10 |           10 | -              |                     11 |
| journalctl-_SYSTEMD_UNIT=ssh.service |         36 |           12 |             24 |                     17 |
+--------------------------------------+------------+--------------+----------------+------------------------+
...
...

Info

All these metrics are actually coming from crowdsec-agent's prometheus agent. See prometheus directly for more insights.

Reference documentation

Link to acquisition reference documentation