Try the CloudWatch Stream for a better and easier experience.

With this integration, you can collect Cloudwatch metrics and forward them to Logz.io using the OpenTelemetry collector. We simplify the data export and collection of your metrics. You tell us the desired namespaces and regions that you want to send your data from and we fetch the most relevant metrics to display in the Logz.io pre-built infrastructure monitoring dashboards.

Important notes

  • Extra calls and charges on AWS API requests may be generated by this project.

  • You can track the number of AWS API requests with the cloudwatch_requests_total metric, which provides labels with the API name and namespace specification. For example, cloudwatch_requests_total{action="getMetricStatistics",namespace="AWS/EC2",} 876.0.

  • For more information about the prom/cloudwatch-exporter cost, refer to the relevant Prometheus documentation and to the AWS cloudwatch API pricing page.

This integration allows you to run a Cloudwatch exporter as a Docker container with environment variables to send Cloudwatch metrics to your Logz.io account. We recommend using this configuration as default.

Set up your IAM user

You’ll need an IAM user with the following permissions:

  • cloudwatch:ListMetrics
  • cloudwatch:GetMetricStatistics
  • tag:GetResources

If you don’t have one, set that up now.

Create an Access key ID and Secret access key for the IAM user, and paste them in your text editor.

Get your metrics region

You’ll need to specify the AWS region you’re collecting metrics from.

AWS region menu

Find your region’s slug in the region menu (in the top menu, on the right side).

For example: The slug for US East (N. Virginia) is “us-east-1”, and the slug for Canada (Central) is “ca-central-1”.

Run the container with environment variables
docker run --name cloudwatch-metrics \
-e TOKEN=<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> \
-e LOGZIO_REGION=<<LOGZIO_REGION>> \
-e AWS_REGION=<<AWS_REGION>> \
-e AWS_ACCESS_KEY_ID=<<AWS_ACCESS_KEY_ID>> \
-e AWS_SECRET_ACCESS_KEY=<<AWS_SECRET_ACCESS_KEY>> \
-e AWS_NAMESPACES=<<AWS_NAMESPACES>> \
logzio/cloudwatch-metrics
Environment variable Description
AWS_REGION (Required) Your region’s slug. You can find this in the AWS Console region menu (in the top menu, to the right). Note: This is the region that you will collect metrics from.
LOGZIO_REGION (Required) Your Logz.io region code. For example if your region is US, then your region code is us. You can find your region code here: https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls.
<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> (Required) Token for shipping metrics to your Logz.io account. Find it under Settings > Manage accounts. How do I look up my Metrics account token?
AWS_NAMESPACES (Required) Comma-separated list of namespaces of the metrics you want to collect. You can find a complete list of namespaces at AWS Services That Publish CloudWatch Metrics. Note: This Environment variable is required unless you define the CUSTOM_CONFIG Environment variable
SCRAPE_INTERVAL The time interval (in seconds) during which the Cloudwatch exporter retrieves metrics from Cloudwatch, and the Opentelemtry collector scrapes and sends the metrics to Logz.io. Default = 300. Note: This value must be a multiple of 60.
P8S_LOGZIO_NAME The value of the p8s_logzio_name external label. This variable identifies which Prometheus environment the metrics arriving at Logz.io came from. Default = logzio-cloudwatch-metrics.
CUSTOM_CONFIG Mount your cloudwatch exporter configuration file -v pathToConfig/config.yml:config_files/cloudwatch.yml and set to true if you want to use custom configuration for cloudwatch exporter, Default = false
CUSTOM_LISTENER Set a custom URL to ship metrics to (for example, http://localhost:9200). This overrides the LOGZIO_REGION Environment variable.
SCRAPE_TIMEOUT The time to wait before throttling a scrape request to cloudwatch exporter, Default = 120
REMOTE_TIMEOUT the time to wait before throttling remote write post request to logz.io, Default = 120
LOG_LEVEL Opentelemetry log level, Default = debug
LOGZIO_LOG_LEVEL builder.py Python script log level. Default = info
AWS_ROLE_ARN Your IAM role to assume.
AWS_ACCESS_KEY_ID Your IAM user’s access key ID.
AWS_SECRET_ACCESS_KEY Your IAM user’s secret key.
SET_TIMESTAMP Boolean for whether to set the Prometheus metric timestamp as the original Cloudwatch timestamp. Default = false
PERIOD_SECONDS period to request the metric for. Only the most recent data point is used. Default = 300
RANGE_SECONDS how far back to request data for. Useful for cases such as Billing metrics that are only set every few hours. Default = 300
DELAY_SECONDS The newest data to request. Used to avoid collecting data that has not fully converged. Default = 300
Publish extension ports

You can monitor the container using opentelemetry extensions in the following ports:

  • 8888 - opentelemetry metrics
  • 55679 - Zpages
  • 13133 - Health check
  • 1777 - Pprof

You can also publish the ports to your host network by using the -p flag:

docker run --name cloudwatch-metrics \
-v <<path_to_config_file>>:config_files/config.yml \
-p 8888:8888 \
-p 55679:55679 \
-p 13133:13133 \
-p 1777:1777 \
logzio/cloudwatch-metrics

This integration allows you to run a Cloudwatch exporter as a Docker container with a configuration file to send Cloudwatch metrics to your Logz.io account.

Set up your IAM user

You’ll need an IAM user with the following permissions:

  • cloudwatch:ListMetrics
  • cloudwatch:GetMetricStatistics
  • tag:GetResources

If you don’t have one, set that up now.

Create an Access key ID and Secret access key for the IAM user, and paste them in your text editor.

Get your metrics region

You’ll need to specify the AWS region you’re collecting metrics from.

AWS region menu

Find your region’s slug in the region menu (in the top menu, on the right side).

For example: The slug for US East (N. Virginia) is “us-east-1”, and the slug for Canada (Central) is “ca-central-1”.

Run with configuration file

Create config.yml file:

otel:
  # your logz.io region
  logzio_region: "us"
  # custom listener address
  custom_listener: ""
  # environment tag that will be attached to all samples
  p8s_logzio_name: "cloudwatch-metrics"
  # your logz.io metrics token
  token: "<<PROMETHEUS-METRICS-SHIPPING-TOKEN>>"
  # the time to wait between scrape requests
  scrape_interval: 300
  # the time to wait before throttling remote write post request to logz.io
  remote_timeout: 120
  # the time to wait before throttling a scrape request to cloudwatch exporter
  scrape_timeout: 120
  # opentelemetry log level
  log_level: "debug"
  # python script log level
  logzio_log_level: "info"
  # aws credentials
  AWS_ACCESS_KEY_ID: ""
  AWS_SECRET_ACCESS_KEY: ""
cloudwatch:
  # set to true if you are loading a custom configuration file for cloudwatch exporter
  custom_config: "false"
  # your cloudwatch aws region
  region: "us-east-1"
  # role arn to assume
  role_arn: ""
  # list of aws cloudwatch namespaces to monitor
  aws_namespaces: []
  # The newest data to request. Used to avoid collecting data that has not fully converged
  delay_seconds: 300
  # how far back to request data for. Useful for cases such as Billing metrics that are only set every few hours
  range_seconds: 300
  # period to request the metric for. Only the most recent data point is used
  period_seconds: 300
  # boolean for whether to set the Prometheus metric timestamp as the original Cloudwatch timestamp
  set_timestamp: "false"

Replace the placeholders to match your specifics. (They are indicated by the double angle brackets << >>):

  • Replace <<PROMETHEUS-METRICS-SHIPPING-TOKEN>> with a token for the Metrics account you want to ship to.
    Here’s how to look up your Metrics token.
  • Replace <<LISTENER-HOST>> with the Logz.io Listener URL for your region, configured to use port 8052 for http traffic, or port 8053 for https traffic. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe.
Mount the configuration file to your container
docker run --name cloudwatch-metrics \
-v <<path_to_config_file>>:/config_files/config.yml \
logzio/cloudwatch-metrics
Publish extension ports

You can monitor the container using opentelemetry extensions in the following ports:

  • 8888 - opentelemetry metrics
  • 55679 - Zpages
  • 13133 - Health check
  • 1777 - Pprof

You can also publish the ports to your host network by using the -p flag:

docker run --name cloudwatch-metrics \
-v <<path_to_config_file>>:config_files/config.yml \
-p 8888:8888 \
-p 55679:55679 \
-p 13133:13133 \
-p 1777:1777 \
logzio/cloudwatch-metrics

This integration allows you to run a Cloudwatch exporter as a Docker container with both environment variables and a configuration file to send Cloudwatch metrics to your Logz.io account. Please note that the envorinment variables will always overwrite variables specified in the configuration file in case the variables overlap.

Set up your IAM user

You’ll need an IAM user with the following permissions:

  • cloudwatch:ListMetrics
  • cloudwatch:GetMetricStatistics
  • tag:GetResources

If you don’t have one, set that up now.

Create an Access key ID and Secret access key for the IAM user, and paste them in your text editor.

Get your metrics region

You’ll need to specify the AWS region you’re collecting metrics from.

AWS region menu

Find your region’s slug in the region menu (in the top menu, on the right side).

For example: The slug for US East (N. Virginia) is “us-east-1”, and the slug for Canada (Central) is “ca-central-1”.

Run with custom Cloudwatch exporter configuration

Create cloudwatch.yml file (for details refer to prom/cloudwatch_exporter project), and mount the configuration file to your container and set CUSTOM_CONFIG variable to true:

docker run --name cloudwatch-metrics \
-e TOKEN=<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> \
-e LOGZIO_REGION=<<LOGZIO_REGION>> \
-e AWS_ACCESS_KEY_ID=<<AWS_ACCESS_KEY_ID>> \
-e CUSTOM_CONFIG=true \
-e AWS_SECRET_ACCESS_KEY=<<AWS_SECRET_ACCESS_KEY>> \
-v <<path_to_cloudwatch_config_file>>:/config_files/cloudwatch.yml \
logzio/cloudwatch-metrics

Replace the placeholders to match your specifics. (They are indicated by the double angle brackets << >>):

  • Replace <<PROMETHEUS-METRICS-SHIPPING-TOKEN>> with a token for the Metrics account you want to ship to.
    Here’s how to look up your Metrics token.
  • Replace <<LISTENER-HOST>> with the Logz.io Listener URL for your region, configured to use port 8052 for http traffic, or port 8053 for https traffic. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe.
Publish extension ports

You can monitor the container using opentelemetry extensions in the following ports: * 8888 - opentelemetry metrics * 55679 - Zpages * 13133 - Health check * 1777 - Pprof

You can also publish the ports to your host network by using the -p flag:

docker run --name cloudwatch-metrics \
-v <<path_to_config_file>>:config_files/config.yml \
-p 8888:8888 \
-p 55679:55679 \
-p 13133:13133 \
-p 1777:1777 \
logzio/cloudwatch-metrics

This integration deploys the logzio/cloudwatch-metrics container directly to ECS using a dedicated Cloudformation stack. It deploys a new ECS cluster, ECS service and task definition.

Auto-deploy the Container

👇 To begin, click the start button corresponding to your region to start the automated deployment in your environment.

Region Deployment link
us-east-1 Deploy to AWS
us-east-2 Deploy to AWS
us-west-1 Deploy to AWS
us-west-2 Deploy to AWS
eu-central-1 Deploy to AWS
eu-west-1 Deploy to AWS
eu-west-2 Deploy to AWS
ap-east-1 Deploy to AWS
ap-southeast-3 Deploy to AWS
ap-south-1 Deploy to AWS
Specify the template

Keep the defaults and click Next.

Specify the stack details

Specify stack details

Specify the stack details as per the table below and select Next.

Parameter Description
AwsRegion Your region’s slug. You can find this in the AWS Console region menu (in the top menu, to the right). Note: This is the region that you will collect metrics from.
LogzioRegion Your Logz.io region code. For example if your region is US, then your region code is us. You can find your region code here: https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls.
Token Your Logz.io metrics shipping token:<<PROMETHEUS-METRICS-SHIPPING-TOKEN>>
AwsNamespaces Comma-separated list of namespaces of the metrics you want to collect. You can find a complete list of namespaces at AWS Services That Publish CloudWatch Metrics.
ScrapeInterval The time interval (in seconds) during which the Cloudwatch exporter retrieves metrics from Cloudwatch, and the Opentelemtry collector scrapes and sends the metrics to Logz.io. Default = 300. Note: This value must be a multiple of 60.
P8sLogzioName The value of the p8s_logzio_name external label. This variable identifies which Prometheus environment the metrics arriving at Logz.io come from. Default = logzio-cloudwatch-metrics.
ScrapeTimeOut Time to wait before throttling a scrape request to the Cloudwatch exporter. Default = 120
RemoteTimeOut Time to wait before throttling remote write post request to Logz.io. Default = 120
AwsAccesskeyId Your IAM user’s access key ID.
AwsSecretAccesskey Your IAM user’s secret key.
SetTimestamp Boolean for whether to set the Prometheus metric timestamp as the original Cloudwatch timestamp. Default = false
PeriodSeconds Period to request metrics for. Only the most recent data point is used. Default = 300
RangeSeconds How far back to request data for. Useful for cases such as Billing metrics that are only set every few hours. Default = 300
DelaySeconds The newest data to request. Used to avoid collecting data that has not fully converged. Default = 300
VPC The VPC to deploy the ECS resources.
ServiceName Select name for the new ECS service.
Subnet Subnet to deploy the ECS resources.
Configure the stack options

Keep the defaults and click Next.

Review the deployment

Confirm that you acknowledge that AWS CloudFormation might create IAM resources and select Create stack.