This module allows
CrowdSec to acquire logs from AWS's cloudwatch service, in one-shot and streaming mode.
Instead of using this datasource, we recommend setting up a log subscription filter in your AWS account to push the logs to a kinesis stream, and use the kinesis datasource to read them.
To monitor a given stream within a group :
source: cloudwatchgroup_name: /aws/my/group/stream_name: 'given_stream'aws_profile: monitoringaws_config_dir: /home/user/.aws/labels: type: apigateway
To monitor streams matching regexp within a group :
source: cloudwatchgroup_name: /aws/my/group/stream_regexp: '^stream[0-9]+$'aws_profile: monitoringlabels: type: apigateway
Look at the
configuration parameters to view all supported options.
Name of the group to monitor, exact match.
A RE2 expression that will restrict streams within the group that will be monitored.
Name of stream to monitor, exact match.
- describelogstreams_limit : control the pagination size of describelogstreams calls (default:
- getlogeventspages_limit : control the pagination size of getlogeventspages calls (default:
note : AWS SDK allows to identify streams according to the timestamp of the latest even within, and this is what we rely on.
- poll_new_stream_interval : frequency to poll for new stream within given group (default
- max_stream_age : open only streams for which last event is at most this age (default
- poll_stream_interval : frequency to poll for new events within given group (default
- stream_read_timeout : stop reading a given stream when no new events have been seen for this duration (default
When set to
false), prepend the cloudwatch event timestamp to the generated log string. This is intended for cases where you log itself wouldn't contain timestamp.
The aws profile to use to poll cloudwatch, relies on your
The path to your
~/.aws/, defaults to
cloudwatch implements a very approximative DSN, as follows :
Supported args are :
log_level: set log level of module
profile: set aws profile name
end_date: provide start and end date limits for events, see supported formats
backlog: provide a duration, events from now()-duration till now() will be read
A 'pseudo DSN' must be provided:
crowdsec -type nginx -dsn 'cloudwatch:///<path_to_my_log_stream>?backlog=12h&profile=<my_aws_profile>'
You can specify the
log_level parameter to change the log level for the acquisition :
crowdsec -type nginx -dsn 'cloudwatch:///<path_to_my_log_stream>?backlog=12h&profile=<my_aws_profile>&log_level=debug'
This data source lacks unit tests because mocking aws sdk is fastidious.