Blocklist mirror

ModeStream only

MetricsUnsupported

MTLSSupported

PrometheusSupported

This Remediation Component exposes CrowdSec's active decisions via provided HTTP(S) endpoints in pre-defined formats. It can be used by network appliances which support consumption of blocklists via HTTP.

Installation from repositories

Setup crowdsec repositories.

Debian/Ubuntu
RHEL/Centos/Fedora

$ sudo apt install crowdsec-blocklist-mirror

$ sudo yum install crowdsec-blocklist-mirror

Docker / Podman:

Refer to docker hub

Manual

installation via script

First, download the latest crowdsec-blocklist-mirror release.

$ tar xzvf crowdsec-blocklist-mirror.tgz
$ sudo ./install.sh

From source

Run the following commands:

$ git clone https://github.com/crowdsecurity/cs-blocklist-mirror.git
$ cd cs-blocklist-mirror/
$ make release
$ cd crowdsec-blocklist-mirror-v*/
$ sudo ./install.sh

Configuration

For manual installations before starting the crowdsec-blocklist-mirror service, please edit the configuration file to add your API URL and key.

The default configuration file is located under : /etc/crowdsec/bouncers/ as file crowdsec-blocklist-mirror.yaml.

If you need to download and restore the configuration file you can find an example on the Respository

Configuration Reference

`crowdsec_config`

Used to nest the configuration related to crowdsec.

`lapi_url`

The URL of CrowdSec LAPI. It should be accessible from whichever network the component has access.

`lapi_key`

It can be obtained by running the following on the machine CrowdSec LAPI is deployed on.

sudo cscli -oraw bouncers add blocklistMirror # -oraw flag can discarded for human friendly output.

`cert_path`

Path to the certificate file used to authenticate with the LAPI.

`key_path`

Path to the key file used to authenticate with the LAPI.

`ca_path_file`

Path to the CA file used to trust the LAPI certificate.

`update_frequency`

The component will poll the CrowdSec LAPI every update_frequency interval.

`include_scenarios_containing`

Ignore IPs banned for triggering scenarios not containing the provided words.

Example
include_scenarios_containing: ["ssh", "http"]

`exclude_scenarios_containing`

Ignore IPs banned for triggering scenarios containing the provided words.

Example
exclude_scenarios_containing: ["ssh", "http"]

`only_include_decisions_from`

Only include IPs banned due to decisions orginating from provided sources.

Example
only_include_decisions_from: ["cscli", "crowdsec"]

`insecure_skip_verify`

Set to true to skip verifying certificate usually used for self-signed certificates.

`listen_uri`

The bindable address and port for the component to listen on

Example
listen_uri: "127.0.0.1:41412"

`metrics`

Prometheus metrics

`enabled`

Set to true to enable serving and collecting metrics.

`endpoint`

The URI endpoint to serve the metrics on.

`blocklists`

List of blocklists to serve. Each blocklist has the following configuration.

`format`

Format of the blocklist, the following are supported:

plain_text : One IP per line
mikrotik : Generates a mikrotik script
F5 : Lines for f5 appliances
juniper: One entry per line using CIDR notation

`endpoint`

The URI endpoint to serve the blocklist on.

`authentication`

Configuration used to enforce or bypass authentication on the blocklist.

`type`:

The type of authetentication to impose:

none : No authentication required.
basic : Basic authentication required.
ip_based : IP based authentication required.

`user`

Valid username if using basic type.

`password`:

Password for the provided user and using basic authentication.

`trusted_ips`:

List of valid IPv4 and IPv6 IPs and ranges which have access to blocklist. It's only applicable when authentication type is ip_based.

`tls`

TLS Configuration is utilized to activate HTTPS on the mirror server.

`cert_file`:

Path to certificate to use if TLS is to be enabled on the mirror server.

`key_file`:

Path to certificate key file.

Global RunTime Query Parameters

?ipv4only - Only return IPv4 addresses

Example usage

http://localhost:41412/security/blocklist?ipv4only

?ipv6only - Only return IPv6 addresses

Example usage

http://localhost:41412/security/blocklist?ipv6only

?nosort - Do not sort IP's

Example usage

http://localhost:41412/security/blocklist?nosort

?origin= - Only return IP's by origin

Example usage

http://localhost:41412/security/blocklist?origin=cscli

You can then start the service via:

sudo systemctl start crowdsec-blocklist-mirror

If you need to make changes to the configuration file and be sure they will never be modified or reverted by package upgrades, starting from v0.0.2 you can write them in a crowdsec-blocklist-mirror.yaml.local file as described in Overriding values. Package upgrades may have good reasons to modify the configuration, so be careful if you use a .local file.

Formats

The component can expose the blocklist in the following formats. You can configure the format of the blocklist by setting it's format parameter to any of the supported formats described below.

plain_text

Example:

192.168.1.1
192.168.1.2

mikrotik

Generates a MikroTik Script that the device can execute to populate the specified firewall address list.

MikroTik query parameters

Parameter	Description
`listname=foo`	Set the list name to `foo`. By default, `listname` is set to `CrowdSec`.

Example output:

/ip/firewall/address-list/remove [ find where list="foo" ];
:global CrowdSecAddIP;
:set CrowdSecAddIP do={
    :do { /ip/firewall/address-list/add list=foo address=$1 comment="$2" timeout=$3; } on-error={ }
}
$CrowdSecAddIP 1.2.3.4 "ssh-bf" 152h40m24s
$CrowdSecAddIP 4.3.2.1 "postfix-spam" 166h40m25s
$CrowdSecAddIP 2001:470:1:c84::17 "ssh-bf" 165h13m42s

Example: MikroTik import script

Using on device MikroTik scripting following is a starting point to download and import the blocklist. Ensure to adjust the global query parameters according to your needs!

:local name "[crowdsec]"
:local url "http://<IP>:41412/security/blocklist?ipv4only&nosort"
:local fileName "blocklist.rsc"
:log info "$name fetch blocklist from $url"
/tool fetch url="$url" mode=http dst-path=$fileName
:if ([:len [/file find name=$fileName]] > 0) do={
    :log info "$name import;start"
    /import file-name=$fileName
    :log info "$name import:done"
} else={
    :log error "$name failed to fetch the blocklist"
}

F5

Example:

192.168.1.1,32,bl,ssh-slow-bf
192.168.1.2,32,bl,ssh-slow-bf

Juniper

Generates a .txt file with all IP addresses (single host and subnets) in the CIDR notation format supported by the Juniper Networks SRX firewall platform.

Example:

1.2.3.4/32
4.3.2.1/32

SRX Dynamic Address configuration sample

Using the blocklist on a Juniper SRX requires that the published url ends in .txt. This can be acieved by altering the endpoint config in cfg.yaml as follows:

Sample cfg.yaml

####
blocklists:
  - format: juniper # Supported formats are either of "plain_text", "mikrotik", "juniper"
    endpoint: /security/blocklist.txt #Must have .txt for juniper formatter.
    authentication:
      type: none # Supported types are either of "none", "ip_based", "basic"
      user:
      password:
      trusted_ips: # IP ranges, or IPs which don't require auth to access this blocklist
        - 127.0.0.1
        - ::1
####

This can then be configured on the SRX firewall as follows:

Sample SRX config:

user@srx> show configuration security dynamic-address | display set

set security dynamic-address feed-server crowdsec url http://192.168.1.2:41412
set security dynamic-address feed-server crowdsec update-interval 30
set security dynamic-address feed-server crowdsec feed-name crowdsec path /security/blocklist.txt
set security dynamic-address address-name crowdsec-blocklist profile feed-name crowdsec

Further information here

A successful configuration should return a similar result when queried:

user@srx> show security dynamic-address summary


Dynamic-address session scan status            : Disable
Hold-interval for dynamic-address session scan : 10 seconds


  Server Name                 : crowdsec
    Hostname/IP               : http://192.168.1.2:41412
    Update interval           : 30
    Hold   interval           : 86400
    TLS Profile Name          : ---
    User        Name          : ---


    Feed Name                             : crowdsec
        Mapped dynamic address name       : crowdsec-blocklist
        URL                               : http://192.168.1.2:41412/security/blocklist.txt
        Feed update interval              : 30       Feed hold interval :86400
        Total update                      : 16310
        Total IPv4 entries                : 16240
        Total IPv6 entries                : 0

Installation from repositories​

Docker / Podman:​

Manual​

installation via script​

From source​

Configuration​

Configuration Reference​

crowdsec_config​

lapi_url​

lapi_key​

cert_path​

key_path​

ca_path_file​

update_frequency​

include_scenarios_containing​

exclude_scenarios_containing​

only_include_decisions_from​

insecure_skip_verify​

listen_uri​

metrics​

enabled​

endpoint​

blocklists​

format​

endpoint​

authentication​

type:​

user​

password:​

trusted_ips:​

tls​

cert_file:​

key_file:​

Global RunTime Query Parameters​

Formats​

plain_text​

mikrotik​

MikroTik query parameters​

Example: MikroTik import script​

F5​

Juniper​

SRX Dynamic Address configuration sample​

Installation from repositories

Docker / Podman:

Manual

installation via script

From source

Configuration

Configuration Reference

`crowdsec_config`

`lapi_url`

`lapi_key`

`cert_path`

`key_path`

`ca_path_file`

`update_frequency`

`include_scenarios_containing`

`exclude_scenarios_containing`

`only_include_decisions_from`

`insecure_skip_verify`

`listen_uri`

`metrics`

`enabled`

`endpoint`

`blocklists`

`format`

`endpoint`

`authentication`

`type`:

`user`

`password`:

`trusted_ips`:

`tls`

`cert_file`:

`key_file`:

Global RunTime Query Parameters

Formats

plain_text

mikrotik

MikroTik query parameters

Example: MikroTik import script

F5

Juniper

SRX Dynamic Address configuration sample