Skip to main content

Blocklist mirror

📚 Documentation 💠 Hub 💬 Discourse

This Remediation Component exposes CrowdSec's active decisions via provided HTTP(S) endpoints in pre-defined formats. It can be used by network appliances which support consumption of blocklists via HTTP.

Installation from repositories​

Setup crowdsec repositories.

$ sudo apt install crowdsec-blocklist-mirror

Docker / Podman:​

Refer to docker hub

Manual​

installation via script​

First, download the latest crowdsec-blocklist-mirror release.

$ tar xzvf crowdsec-blocklist-mirror.tgz
$ sudo ./install.sh

From source​

Run the following commands:

$ git clone https://github.com/crowdsecurity/cs-blocklist-mirror.git
$ cd cs-blocklist-mirror/
$ make release
$ cd crowdsec-blocklist-mirror-v*/
$ sudo ./install.sh

Configuration​

For manual installations before starting the crowdsec-blocklist-mirror service, please edit the configuration file to add your API URL and key.

The default configuration file is located under : /etc/crowdsec/bouncers/ as file crowdsec-blocklist-mirror.yaml.

If you need to download and restore the configuration file you can find an example on the Respository

Configuration Reference​

crowdsec_config​

Used to nest the configuration related to crowdsec.

lapi_url​

string

The URL of CrowdSec LAPI. It should be accessible from whichever network the component has access.

lapi_key​

string

It can be obtained by running the following on the machine CrowdSec LAPI is deployed on.

sudo cscli -oraw bouncers add blocklistMirror # -oraw flag can discarded for human friendly output.

cert_path​

string

Path to the certificate file used to authenticate with the LAPI.

key_path​

string

Path to the key file used to authenticate with the LAPI.

ca_path_file​

string

Path to the CA file used to trust the LAPI certificate.

update_frequency​

string (That is parseable by time.ParseDuration)

The component will poll the CrowdSec LAPI every update_frequency interval.

include_scenarios_containing​

[ ]string

Ignore IPs banned for triggering scenarios not containing the provided words.

Example
include_scenarios_containing: ["ssh", "http"]

exclude_scenarios_containing​

[ ]string

Ignore IPs banned for triggering scenarios containing the provided words.

Example
exclude_scenarios_containing: ["ssh", "http"]

only_include_decisions_from​

[ ]string

Only include IPs banned due to decisions orginating from provided sources.

Example
only_include_decisions_from: ["cscli", "crowdsec"]

insecure_skip_verify​

boolean

Set to true to skip verifying certificate usually used for self-signed certificates.

listen_uri​

string (<IP>:<PORT>)

The bindable address and port for the component to listen on

Example
listen_uri: "127.0.0.1:41412"

metrics​

Prometheus metrics

enabled​

boolean

Set to true to enable serving and collecting metrics.

endpoint​

string

The URI endpoint to serve the metrics on.

blocklists​

[ ]BlockListConfig

List of blocklists to serve. Each blocklist has the following configuration.

format​

string

Format of the blocklist, the following are supported:

  • plain_text : One IP per line
  • mikrotik : Lines for mikrotik, format is /ip|/ipv6 firewall address-list add list={list_name} address={ip} comment="{scenario} for {duration}"
  • F5 : Lines for f5 appliances, format is {ip|range},{netmask},bl,{scenario}

endpoint​

string

The URI endpoint to serve the blocklist on.

authentication​

Configuration used to enforce or bypass authentication on the blocklist.

type:​

none | basic | ip_based

The type of authetentication to impose:

  • none : No authentication required.
  • basic : Basic authentication required.
  • ip_based : IP based authentication required.
user​

string

Valid username if using basic type.

password:​

string

Password for the provided user and using basic authentication.

trusted_ips:​

[ ]string

List of valid IPv4 and IPv6 IPs and ranges which have access to blocklist. It's only applicable when authentication type is ip_based.

tls​

TLS Configuration is utilized to activate HTTPS on the mirror server.

cert_file:​

string

Path to certificate to use if TLS is to be enabled on the mirror server.

key_file:​

string

Path to certificate key file.

Global RunTime Query Parameters​

?ipv4only - Only return IPv4 addresses

Example usage

http://localhost:41412/security/blocklist?ipv4only

?ipv6only - Only return IPv6 addresses

Example usage

http://localhost:41412/security/blocklist?ipv6only

?nosort - Do not sort IP's

Only use if you do not care about the sorting of the list, can result in better performance.

Example usage

http://localhost:41412/security/blocklist?nosort

?origin= - Only return IP's by origin

Example usage

http://localhost:41412/security/blocklist?origin=cscli

You can then start the service via:

sudo systemctl start crowdsec-blocklist-mirror

If you need to make changes to the configuration file and be sure they will never be modified or reverted by package upgrades, starting from v0.0.2 you can write them in a crowdsec-blocklist-mirror.yaml.local file as described in Overriding values. Package upgrades may have good reasons to modify the configuration, so be careful if you use a .local file.

Formats​

The component can expose the blocklist in the following formats. You can configure the format of the blocklist by setting it's format parameter to any of the supported formats described below.

plain_text​

Example:

192.168.1.1
192.168.1.2

mikrotik​

If your mikrotik router does not support ipv6, then you can use the global query parameters to only return ipv4 addresses.

Example:

/ip firewall address-list remove [find list=CrowdSec]
/ipv6 firewall address-list remove [find list=CrowdSec]
/ip firewall address-list add list=CrowdSec address=192.168.1.1 comment="crowdsecurity/ssh-bf for 152h40m24.308868973s"
/ip firewall address-list add list=CrowdSec address=192.168.1.2 comment="crowdsecurity/postfix-spam for 166h40m25.280338424s"/ipv6 firewall address-list add list=CrowdSec address=2001:470:1:c84::17 comment="crowdsecurity/ssh-bf for 165h13m42.405449876s"

mikrotik query parameters​

?listname=foo - Set the list name to foo, by default listname is set to CrowdSec

example output:

/ip firewall address-list remove [find list=foo]
/ipv6 firewall address-list remove [find list=foo]
/ip firewall address-list add list=foo address=192.168.1.1 comment="crowdsecurity/ssh-bf for 152h40m24.308868973s"
/ip firewall address-list add list=foo address=192.168.1.2 comment="crowdsecurity/postfix-spam for 166h40m25.280338424s"/ipv6 firewall address-list add list=foo address=2001:470:1:c84::17 comment="crowdsecurity/ssh-bf for 165h13m42.405449876s"

F5​

Example:

192.168.1.1,32,bl,ssh-slow-bf
192.168.1.2,32,bl,ssh-slow-bf
CrowdSec ConsoleCrowdSec Console