Skip to main content
Version: Next

Creating scenarios

caution

All the examples assume that you have read the Creating parsers documentation.

Foreword#

This documentation assumes you're trying to create a scenario for crowdsec with the intent of submitting to the hub, and thus create the associated functional testing. The creation of said functional testing will guide our process and will make it easier.

We're going to create a scenario for an imaginary service "myservice" from the following logs of failed authentication :

Dec  8 06:28:43 mymachine myservice[2806]: unknown user 'toto' from '1.2.3.4'Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'

Pre-requisites#

  1. Install crowdsec locally

  2. Clone the hub

git clone https://github.com/crowdsecurity/hub.git

Create our test#

From the root of the hub repository :

โ–ถ cscli hubtest create myservice-bf --type syslog --ignore-parsers
  Test name                   :  myservice-bf  Test path                   :  /home/dev/github/hub/.tests/myservice-bf  Log file                    :  /home/dev/github/hub/.tests/myservice-bf/myservice-bf.log (please fill it with logs)  Parser assertion file       :  /home/dev/github/hub/.tests/myservice-bf/parser.assert (please fill it with assertion)  Scenario assertion file     :  /home/dev/github/hub/.tests/myservice-bf/scenario.assert (please fill it with assertion)  Configuration File          :  /home/dev/github/hub/.tests/myservice-bf/config.yaml (please fill it with parsers, scenarios...)

note: we specify the --ignore-parsers flag because we don't want to test the parsers, only the scenarios.

Configure our test#

Let's add our parser and scenario to the test configuration (.tests/myservice-bf/config.yaml) file.

parsers:- crowdsecurity/syslog-logs- crowdsecurity/dateparse-enrich- ./parsers/s01-parse/crowdsecurity/myservice-logs.yamlscenarios:- ./scenarios/crowdsecurity/myservice-bf.yamlpostoverflows:- ""log_file: myservice-bf.loglog_type: syslogignore_parsers: true

note: as our custom parser and scenario are not yet part of the hub, we specify their path relative to the root of the hub directory.

Scenario creation#

Let's create a simple scenario to detect bruteforce attemp on myservice:

# myservice bruteforcetype: leakyname: crowdsecurity/myservice-bfdescription: "Detect myservice bruteforce"filter: "evt.Meta.log_type == 'myservice_failed_auth'"leakspeed: "10s"capacity: 5groupby: evt.Meta.source_ipblackhole: 1mreprocess: truelabels: service: myservice type: bruteforce remediation: true
note

We filter on evt.Meta.log_type == 'myservice_failed_auth' because in the parser myservice-logs (created in the Creating parsers part) we set the log_type to myservice_failed_auth for bad password or bad user attempt.

We have the following fields:

  • a type: the type of bucket to use (trigger or leaky).
  • a name
  • a description
  • a filter: the filter to apply on events to be filled in this bucket.
  • a leakspeed
  • a capacity: the number of events in the bucket before it overflows.
  • a groupby: a field from the event to partition the bucket. It is often the source_ip of the event.
  • a blackhole: the number of minute to not retrigger this scenario for the same groupby field.
  • a reprocess: ingest the alert in crowdsec for further processing.
  • some labels: some labels to apply on the trigger event. Don't forget to set remediation: true if you want the IP to be blocked by bouncers.

We can then "test" our scenario like this :

โ–ถ cscli hubtest run myservice-bfINFO[01-10-2021 12:41:21 PM] Running test 'myservice-bf'                WARN[01-10-2021 12:41:24 PM] Assert file '/home/dev/github/hub/.tests/myservice-bf/scenario.assert' is empty, generating assertion: 
len(results) == 1"1.2.3.4" in results[0].Overflow.GetSources()results[0].Overflow.Sources["1.2.3.4"].IP == "1.2.3.4"results[0].Overflow.Sources["1.2.3.4"].Range == ""results[0].Overflow.Sources["1.2.3.4"].GetScope() == "Ip"results[0].Overflow.Sources["1.2.3.4"].GetValue() == "1.2.3.4"results[0].Overflow.Alert.Events[0].GetMeta("datasource_path") == "myservice-bf.log"results[0].Overflow.Alert.Events[0].GetMeta("datasource_type") == "file"results[0].Overflow.Alert.Events[0].GetMeta("log_subtype") == "myservice_bad_user"results[0].Overflow.Alert.Events[0].GetMeta("log_type") == "myservice_failed_auth"results[0].Overflow.Alert.Events[0].GetMeta("service") == "myservice"results[0].Overflow.Alert.Events[0].GetMeta("source_ip") == "1.2.3.4"results[0].Overflow.Alert.Events[0].GetMeta("username") == "toto"....results[0].Overflow.Alert.GetScenario() == "crowdsecurity/myservice-bf"results[0].Overflow.Alert.Remediation == trueresults[0].Overflow.Alert.GetEventsCount() == 6
...

Please fill your assert file(s) for test 'myservice-bf', exiting

What happened here ?

  • The scenario has been triggered and is generating some assertion (for functional test)
  • In production environment, an alert would have been send to the CrowdSec Local API.

We can again understand more of what is going on thanks to cscli hubtest explain :

โ–ถ cscli hubtest explain myservice-bfline: Dec  8 06:28:43 mymachine myservice[2806]: unknown user 'toto' from '1.2.3.4'    โ”œ s00-raw    |   โ”” ๐ŸŸข crowdsecurity/syslog-logs    โ”œ s01-parse    |   โ”” ๐ŸŸข crowdsecurity/myservice-logs    โ”œ s02-enrich    |   โ”” ๐ŸŸข crowdsecurity/dateparse-enrich    โ”œ-------- parser success ๐ŸŸข    โ”œ Scenarios        โ”” ๐ŸŸข crowdsecurity/myservice-bf
line: Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'    โ”œ s00-raw    |   โ”” ๐ŸŸข crowdsecurity/syslog-logs    โ”œ s01-parse    |   โ”” ๐ŸŸข crowdsecurity/myservice-logs    โ”œ s02-enrich    |   โ”” ๐ŸŸข crowdsecurity/dateparse-enrich    โ”œ-------- parser success ๐ŸŸข    โ”œ Scenarios        โ”” ๐ŸŸข crowdsecurity/myservice-bf
line: Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'    โ”œ s00-raw    |   โ”” ๐ŸŸข crowdsecurity/syslog-logs    โ”œ s01-parse    |   โ”” ๐ŸŸข crowdsecurity/myservice-logs    โ”œ s02-enrich    |   โ”” ๐ŸŸข crowdsecurity/dateparse-enrich    โ”œ-------- parser success ๐ŸŸข    โ”œ Scenarios        โ”” ๐ŸŸข crowdsecurity/myservice-bf
line: Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'    โ”œ s00-raw    |   โ”” ๐ŸŸข crowdsecurity/syslog-logs    โ”œ s01-parse    |   โ”” ๐ŸŸข crowdsecurity/myservice-logs    โ”œ s02-enrich    |   โ”” ๐ŸŸข crowdsecurity/dateparse-enrich    โ”œ-------- parser success ๐ŸŸข    โ”œ Scenarios        โ”” ๐ŸŸข crowdsecurity/myservice-bf
line: Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'    โ”œ s00-raw    |   โ”” ๐ŸŸข crowdsecurity/syslog-logs    โ”œ s01-parse    |   โ”” ๐ŸŸข crowdsecurity/myservice-logs    โ”œ s02-enrich    |   โ”” ๐ŸŸข crowdsecurity/dateparse-enrich    โ”œ-------- parser success ๐ŸŸข    โ”œ Scenarios        โ”” ๐ŸŸข crowdsecurity/myservice-bf
line: Dec  8 06:28:43 mymachine myservice[2806]: bad password for user 'admin' from '1.2.3.4'    โ”œ s00-raw    |   โ”” ๐ŸŸข crowdsecurity/syslog-logs    โ”œ s01-parse    |   โ”” ๐ŸŸข crowdsecurity/myservice-logs    โ”œ s02-enrich    |   โ”” ๐ŸŸข crowdsecurity/dateparse-enrich    โ”œ-------- parser success ๐ŸŸข    โ”œ Scenarios        โ”” ๐ŸŸข crowdsecurity/myservice-bf

Closing word#

We have now a fully functional scenario for myservice to detect brute forces! We can either deploy it to our production systems to do stuff, or even better, contribute to the hub !

If you want to know more about directives and possibilities, take a look at the scenario reference documentation !