Try the CloudWatch Stream for a better and easier experience.
With this integration, you can collect Cloudwatch metrics and forward them to Logz.io using the OpenTelemetry collector. We simplify the data export and collection of your metrics. You tell us the desired namespaces and regions that you want to send your data from and we fetch the most relevant metrics to display in the Logz.io pre-built infrastructure monitoring dashboards.
Important notes
-
Extra calls and charges on AWS API requests may be generated by this project.
-
You can track the number of AWS API requests with the
cloudwatch_requests_totalmetric, which provides labels with the API name and namespace specification. For example,cloudwatch_requests_total{action="getMetricStatistics",namespace="AWS/EC2",} 876.0. -
For more information about the prom/cloudwatch-exporter cost, refer to the relevant Prometheus documentation and to the AWS cloudwatch API pricing page.
This integration allows you to run a Cloudwatch exporter as a Docker container with environment variables to send Cloudwatch metrics to your Logz.io account. We recommend using this configuration as default.
Set up your IAM user
You’ll need an IAM user with the following permissions:
cloudwatch:ListMetricscloudwatch:GetMetricStatisticstag:GetResources
If you don’t have one, set that up now.
Create an Access key ID and Secret access key for the IAM user, and paste them in your text editor.
Get your metrics region
You’ll need to specify the AWS region you’re collecting metrics from.
Find your region’s slug in the region menu (in the top menu, on the right side).
For example: The slug for US East (N. Virginia) is “us-east-1”, and the slug for Canada (Central) is “ca-central-1”.
Run the container with environment variables
docker run --name cloudwatch-metrics \
-e TOKEN=<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> \
-e LOGZIO_REGION=<<LOGZIO_REGION>> \
-e AWS_REGION=<<AWS_REGION>> \
-e AWS_ACCESS_KEY_ID=<<AWS_ACCESS_KEY_ID>> \
-e AWS_SECRET_ACCESS_KEY=<<AWS_SECRET_ACCESS_KEY>> \
-e AWS_NAMESPACES=<<AWS_NAMESPACES>> \
logzio/cloudwatch-metrics
| Environment variable | Description |
|---|---|
| AWS_REGION (Required) | Your region’s slug. You can find this in the AWS Console region menu (in the top menu, to the right). Note: This is the region that you will collect metrics from. |
| LOGZIO_REGION (Required) | Your Logz.io region code. For example if your region is US, then your region code is us. You can find your region code here: https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls. |
<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> (Required) |
Token for shipping metrics to your Logz.io account. Find it under Settings > Manage accounts. How do I look up my Metrics account token? |
| AWS_NAMESPACES (Required) | Comma-separated list of namespaces of the metrics you want to collect. You can find a complete list of namespaces at AWS Services That Publish CloudWatch Metrics. Note: This Environment variable is required unless you define the CUSTOM_CONFIG Environment variable |
| SCRAPE_INTERVAL | The time interval (in seconds) during which the Cloudwatch exporter retrieves metrics from Cloudwatch, and the Opentelemtry collector scrapes and sends the metrics to Logz.io. Default = 300. Note: This value must be a multiple of 60. |
| P8S_LOGZIO_NAME | The value of the p8s_logzio_name external label. This variable identifies which Prometheus environment the metrics arriving at Logz.io came from. Default = logzio-cloudwatch-metrics. |
| CUSTOM_CONFIG | Mount your cloudwatch exporter configuration file -v pathToConfig/config.yml:config_files/cloudwatch.yml and set to true if you want to use custom configuration for cloudwatch exporter, Default = false |
| CUSTOM_LISTENER | Set a custom URL to ship metrics to (for example, http://localhost:9200). This overrides the LOGZIO_REGION Environment variable. |
| SCRAPE_TIMEOUT | The time to wait before throttling a scrape request to cloudwatch exporter, Default = 120 |
| REMOTE_TIMEOUT | the time to wait before throttling remote write post request to logz.io, Default = 120 |
| LOG_LEVEL | Opentelemetry log level, Default = debug |
| LOGZIO_LOG_LEVEL | builder.py Python script log level. Default = info |
| AWS_ROLE_ARN | Your IAM role to assume. |
| AWS_ACCESS_KEY_ID | Your IAM user’s access key ID. |
| AWS_SECRET_ACCESS_KEY | Your IAM user’s secret key. |
| SET_TIMESTAMP | Boolean for whether to set the Prometheus metric timestamp as the original Cloudwatch timestamp. Default = false |
| PERIOD_SECONDS | period to request the metric for. Only the most recent data point is used. Default = 300 |
| RANGE_SECONDS | how far back to request data for. Useful for cases such as Billing metrics that are only set every few hours. Default = 300 |
| DELAY_SECONDS | The newest data to request. Used to avoid collecting data that has not fully converged. Default = 300 |
Publish extension ports
You can monitor the container using opentelemetry extensions in the following ports:
- 8888 -
opentelemetry metrics - 55679 -
Zpages - 13133 -
Health check - 1777 -
Pprof
You can also publish the ports to your host network by using the -p flag:
docker run --name cloudwatch-metrics \
-v <<path_to_config_file>>:config_files/config.yml \
-p 8888:8888 \
-p 55679:55679 \
-p 13133:13133 \
-p 1777:1777 \
logzio/cloudwatch-metrics
This integration allows you to run a Cloudwatch exporter as a Docker container with a configuration file to send Cloudwatch metrics to your Logz.io account.
Set up your IAM user
You’ll need an IAM user with the following permissions:
cloudwatch:ListMetricscloudwatch:GetMetricStatisticstag:GetResources
If you don’t have one, set that up now.
Create an Access key ID and Secret access key for the IAM user, and paste them in your text editor.
Get your metrics region
You’ll need to specify the AWS region you’re collecting metrics from.
Find your region’s slug in the region menu (in the top menu, on the right side).
For example: The slug for US East (N. Virginia) is “us-east-1”, and the slug for Canada (Central) is “ca-central-1”.
Run with configuration file
Create config.yml file:
otel:
# your logz.io region
logzio_region: "us"
# custom listener address
custom_listener: ""
# environment tag that will be attached to all samples
p8s_logzio_name: "cloudwatch-metrics"
# your logz.io metrics token
token: "<<PROMETHEUS-METRICS-SHIPPING-TOKEN>>"
# the time to wait between scrape requests
scrape_interval: 300
# the time to wait before throttling remote write post request to logz.io
remote_timeout: 120
# the time to wait before throttling a scrape request to cloudwatch exporter
scrape_timeout: 120
# opentelemetry log level
log_level: "debug"
# python script log level
logzio_log_level: "info"
# aws credentials
AWS_ACCESS_KEY_ID: ""
AWS_SECRET_ACCESS_KEY: ""
cloudwatch:
# set to true if you are loading a custom configuration file for cloudwatch exporter
custom_config: "false"
# your cloudwatch aws region
region: "us-east-1"
# role arn to assume
role_arn: ""
# list of aws cloudwatch namespaces to monitor
aws_namespaces: []
# The newest data to request. Used to avoid collecting data that has not fully converged
delay_seconds: 300
# how far back to request data for. Useful for cases such as Billing metrics that are only set every few hours
range_seconds: 300
# period to request the metric for. Only the most recent data point is used
period_seconds: 300
# boolean for whether to set the Prometheus metric timestamp as the original Cloudwatch timestamp
set_timestamp: "false"
Replace the placeholders to match your specifics. (They are indicated by the double angle brackets << >>):
- Replace
<<PROMETHEUS-METRICS-SHIPPING-TOKEN>>with a token for the Metrics account you want to ship to.
Here’s how to look up your Metrics token. - Replace
<<LISTENER-HOST>>with the Logz.io Listener URL for your region, configured to use port 8052 for http traffic, or port 8053 for https traffic. For example,listener.logz.ioif your account is hosted on AWS US East, orlistener-nl.logz.ioif hosted on Azure West Europe.
Mount the configuration file to your container
docker run --name cloudwatch-metrics \
-v <<path_to_config_file>>:/config_files/config.yml \
logzio/cloudwatch-metrics
Publish extension ports
You can monitor the container using opentelemetry extensions in the following ports:
- 8888 -
opentelemetry metrics - 55679 -
Zpages - 13133 -
Health check - 1777 -
Pprof
You can also publish the ports to your host network by using the -p flag:
docker run --name cloudwatch-metrics \
-v <<path_to_config_file>>:config_files/config.yml \
-p 8888:8888 \
-p 55679:55679 \
-p 13133:13133 \
-p 1777:1777 \
logzio/cloudwatch-metrics
This integration allows you to run a Cloudwatch exporter as a Docker container with both environment variables and a configuration file to send Cloudwatch metrics to your Logz.io account. Please note that the envorinment variables will always overwrite variables specified in the configuration file in case the variables overlap.
Set up your IAM user
You’ll need an IAM user with the following permissions:
cloudwatch:ListMetricscloudwatch:GetMetricStatisticstag:GetResources
If you don’t have one, set that up now.
Create an Access key ID and Secret access key for the IAM user, and paste them in your text editor.
Get your metrics region
You’ll need to specify the AWS region you’re collecting metrics from.
Find your region’s slug in the region menu (in the top menu, on the right side).
For example: The slug for US East (N. Virginia) is “us-east-1”, and the slug for Canada (Central) is “ca-central-1”.
Run with custom Cloudwatch exporter configuration
Create cloudwatch.yml file (for details refer to prom/cloudwatch_exporter project),
and mount the configuration file to your container and set CUSTOM_CONFIG variable to true:
docker run --name cloudwatch-metrics \
-e TOKEN=<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> \
-e LOGZIO_REGION=<<LOGZIO_REGION>> \
-e AWS_ACCESS_KEY_ID=<<AWS_ACCESS_KEY_ID>> \
-e CUSTOM_CONFIG=true \
-e AWS_SECRET_ACCESS_KEY=<<AWS_SECRET_ACCESS_KEY>> \
-v <<path_to_cloudwatch_config_file>>:/config_files/cloudwatch.yml \
logzio/cloudwatch-metrics
Replace the placeholders to match your specifics. (They are indicated by the double angle brackets << >>):
- Replace
<<PROMETHEUS-METRICS-SHIPPING-TOKEN>>with a token for the Metrics account you want to ship to.
Here’s how to look up your Metrics token. - Replace
<<LISTENER-HOST>>with the Logz.io Listener URL for your region, configured to use port 8052 for http traffic, or port 8053 for https traffic. For example,listener.logz.ioif your account is hosted on AWS US East, orlistener-nl.logz.ioif hosted on Azure West Europe.
Publish extension ports
You can monitor the container using opentelemetry extensions in the following ports:
* 8888 - opentelemetry metrics
* 55679 - Zpages
* 13133 - Health check
* 1777 - Pprof
You can also publish the ports to your host network by using the -p flag:
docker run --name cloudwatch-metrics \
-v <<path_to_config_file>>:config_files/config.yml \
-p 8888:8888 \
-p 55679:55679 \
-p 13133:13133 \
-p 1777:1777 \
logzio/cloudwatch-metrics
This integration deploys the logzio/cloudwatch-metrics container directly to ECS using a dedicated Cloudformation stack. It deploys a new ECS cluster, ECS service and task definition.
Auto-deploy the Container
👇 To begin, click the start button corresponding to your region to start the automated deployment in your environment.
| Region | Deployment link |
|---|---|
| us-east-1 |  |
| us-east-2 | |
| us-west-1 | |
| us-west-2 | |
| eu-central-1 | |
| eu-west-1 | |
| eu-west-2 | |
| ap-east-1 | |
| ap-southeast-3 | |
| ap-south-1 | |
Specify the template
Keep the defaults and click Next.
Specify the stack details
Specify the stack details as per the table below and select Next.
| Parameter | Description |
|---|---|
| AwsRegion | Your region’s slug. You can find this in the AWS Console region menu (in the top menu, to the right). Note: This is the region that you will collect metrics from. |
| LogzioRegion | Your Logz.io region code. For example if your region is US, then your region code is us. You can find your region code here: https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls. |
| Token | Your Logz.io metrics shipping token:<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> |
| AwsNamespaces | Comma-separated list of namespaces of the metrics you want to collect. You can find a complete list of namespaces at AWS Services That Publish CloudWatch Metrics. |
| ScrapeInterval | The time interval (in seconds) during which the Cloudwatch exporter retrieves metrics from Cloudwatch, and the Opentelemtry collector scrapes and sends the metrics to Logz.io. Default = 300. Note: This value must be a multiple of 60. |
| P8sLogzioName | The value of the p8s_logzio_name external label. This variable identifies which Prometheus environment the metrics arriving at Logz.io come from. Default = logzio-cloudwatch-metrics. |
| ScrapeTimeOut | Time to wait before throttling a scrape request to the Cloudwatch exporter. Default = 120 |
| RemoteTimeOut | Time to wait before throttling remote write post request to Logz.io. Default = 120 |
| AwsAccesskeyId | Your IAM user’s access key ID. |
| AwsSecretAccesskey | Your IAM user’s secret key. |
| SetTimestamp | Boolean for whether to set the Prometheus metric timestamp as the original Cloudwatch timestamp. Default = false |
| PeriodSeconds | Period to request metrics for. Only the most recent data point is used. Default = 300 |
| RangeSeconds | How far back to request data for. Useful for cases such as Billing metrics that are only set every few hours. Default = 300 |
| DelaySeconds | The newest data to request. Used to avoid collecting data that has not fully converged. Default = 300 |
| VPC | The VPC to deploy the ECS resources. |
| ServiceName | Select name for the new ECS service. |
| Subnet | Subnet to deploy the ECS resources. |
Configure the stack options
Keep the defaults and click Next.
Review the deployment
Confirm that you acknowledge that AWS CloudFormation might create IAM resources and select Create stack.