The logzio-monitoring Helm Chart ships your Kubernetes telemetry (logs, metrics, traces and security reports) to your Logz.io account.
Please be aware that this project is presently in its beta stage, and as such, it may undergo alterations.
To get the most out of Kubernetes 360, try out dedicated dashboard.
Log in to your Logz.io account and navigate to the current instructions page inside the Logz.io app. Install the pre-built dashboard to enhance the observability of your metrics. To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.
Check if you have any taints on your nodes
kubectl get nodes -o json | jq '"\(.items[].metadata.name) \(.items[].spec.taints)"'
If you want to ship logs from any of the nodes that have a taint, make sure that the taint key values are listed in your in your daemonset/deployment configuration as follows:
tolerations:
- key:
operator:
value:
effect:
Add the Helm Chart
helm repo add logzio-helm https://logzio.github.io/logzio-helm
helm repo update
Deploy the Chart
helm install -n monitoring \
--set logs.enabled=true \
--set logzio-fluentd.secrets.logzioShippingToken="<<LOG-SHIPPING-TOKEN>>" \
--set logzio-fluentd.secrets.logzioListener="<<LISTENER-HOST>>" \
--set logzio-fluentd.env_id="<<ENV-ID>>" \
--set metricsOrTraces.enabled=true \
--set logzio-k8s-telemetry.metrics.enabled=true \
--set logzio-k8s-telemetry.secrets.MetricsToken="<<PROMETHEUS-METRICS-SHIPPING-TOKEN>>" \
--set logzio-k8s-telemetry.secrets.ListenerHost="https://<<LISTENER-HOST>>:8053" \
--set logzio-k8s-telemetry.secrets.p8s_logzio_name="<<ENV-TAG>>" \
--set logzio-k8s-telemetry.traces.enabled=true \
--set logzio-k8s-telemetry.secrets.TracesToken="<<TRACES-SHIPPING-TOKEN>>" \
--set logzio-k8s-telemetry.secrets.LogzioRegion="<<LOGZIO-REGION>>" \
logzio-monitoring logzio-helm/logzio-monitoring
| Parameter | Description |
|---|---|
<<LOG-SHIPPING-TOKEN>> |
Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to. |
<<LISTENER-HOST>> |
Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071.. |
<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> |
Your metrics shipping token. |
<<P8S-LOGZIO-NAME>> |
The name for the environment’s metrics, to easily identify the metrics for each environment. |
<<ENV-ID>> |
The name for your environment’s identifier, to easily identify the telemetry data for each environment. |
<<ENV-TAG>> |
Your custom name for the environment’s metrics, to easily identify the metrics for each environment. |
<<TRACES-SHIPPING-TOKEN>> |
Replace <<TRACING-SHIPPING-TOKEN>> with the token of the account you want to ship to. |
<<LOGZIO-REGION>> |
Name of your Logz.io traces region e.g us or eu. You can find your region code in the Regions and URLs table. |
Check Logz.io for your data
Give your data some time to get from your system to ours, and then open Open Search Dashboards.
Log in to your Logz.io account and navigate to the current instructions page inside the Logz.io app. Install the pre-built dashboard to enhance the observability of your metrics.
To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.
Collecting span metrics and Trivy reports
To enable the logzio-monitoring Helm Chart collect span metrics and Trivy reports, add the following command to the default deployment:
helm install -n monitoring \
--set logzio-k8s-telemetry.secrets.SpmToken=<<SPM-SHIPPING-TOKEN>> \
--set securityReport.enabled=true \
--set logzio-trivy.env_id="<<ENV-ID>>" \
--set logzio-trivy.secrets.logzioShippingToken="<<LOG-SHIPPING-TOKEN>>" \
--set logzio-trivy.secrets.logzioListener="<<LISTENER-HOST>>" \
logzio-monitoring logzio-helm/logzio-monitoring
| Parameter | Description |
|---|---|
<<SPM-SHIPPING-TOKEN>> |
Your span metrics shipping token. |
<<ENV-ID>> |
The name for your environment’s identifier, to easily identify the telemetry data for each environment. |
<<LOG-SHIPPING-TOKEN>> |
Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to. |
<<LISTENER-HOST>> |
Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071.. |
Further configuration
You can modify the default logzio-monitoring Helm Chart by using the --set flag in your helm install command:
| Parameter | Description | Default |
|---|---|---|
logs.enabled |
Enable to send k8s logs | false |
metricsOrTraces |
Enable to send k8s metrics or traces | false |
Modifying the configuration for logs
You can see a full list of the possible configuration values in the logzio-fluentd Chart folder.
If you would like to modify any of the values found in the logzio-fluentd folder, use the --set flag with the logzio-fluentd prefix.
For instance, if there is a parameter called someField in the logzio-telemetry’s values.yaml file, you can set it by adding the following to the helm install command:
--set logzio-fluentd.someField="my new value"
You can add log_type annotation with a custom value, which will be parsed into a log_type field with the same value.
Modifying the configuration for metrics and traces
You can see a full list of the possible configuration values in the logzio-telemetry Chart folder.
If you would like to modify any of the values found in the logzio-telemetry folder, use the --set flag with the logzio-k8s-telemetry prefix.
For instance, if there is a parameter called someField in the logzio-telemetry’s values.yaml file, you can set it by adding the following to the helm install command:
--set logzio-k8s-telemetry.someField="my new value"
Sending telemetry data from eks on fargate
To ship logs from pods running on Fargate, set the fargateLogRouter.enabled value to true. Doing so will deploy a dedicated aws-observability namespace and a configmap for the Fargate log router. For more information on EKS Fargate logging, please refer to the [official AWS documentation]((https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html).
helm install -n monitoring \
--set logs.enabled=true \
--set logzio-fluentd.fargateLogRouter.enabled=true \
--set logzio-fluentd.secrets.logzioShippingToken="<<LOG-SHIPPING-TOKEN>>" \
--set logzio-fluentd.secrets.logzioListener="<<LISTENER-HOST>>" \
--set metricsOrTraces.enabled=true \
--set logzio-k8s-telemetry.metrics.enabled=true \
--set logzio-k8s-telemetry.secrets.MetricsToken="<<PROMETHEUS-METRICS-SHIPPING-TOKEN>>" \
--set logzio-k8s-telemetry.secrets.ListenerHost="https://<<LISTENER-HOST>>:8053" \
--set logzio-k8s-telemetry.secrets.p8s_logzio_name="<<ENV-TAG>>" \
--set logzio-k8s-telemetry.traces.enabled=true \
--set logzio-k8s-telemetry.secrets.TracesToken="<<TRACES-SHIPPING-TOKEN>>" \
--set logzio-k8s-telemetry.secrets.LogzioRegion="<<LOGZIO-REGION>>" \
logzio-monitoring logzio-helm/logzio-monitoring
| Parameter | Description |
|---|---|
<<LOG-SHIPPING-TOKEN>> |
Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to. |
<<LISTENER-HOST>> |
Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071.. |
<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> |
Your metrics shipping token. |
<<P8S-LOGZIO-NAME>> |
The name for the environment’s metrics, to easily identify the metrics for each environment. |
<<ENV-ID>> |
The name for your environment’s identifier, to easily identify the telemetry data for each environment. |
<<ENV-TAG>> |
Your custom name for the environment’s metrics, to easily identify the metrics for each environment. |
<<TRACES-SHIPPING-TOKEN>> |
Replace <<TRACING-SHIPPING-TOKEN>> with the token of the account you want to ship to. |
<<LOGZIO-REGION>> |
Name of your Logz.io traces region e.g us or eu. You can find your region code in the Regions and URLs table. |
Handling image pull rate limit
In certain situations, such as with spot clusters where pods/nodes are frequently replaced, you may encounter the pull rate limit for images fetched from Docker Hub. This could result in the following error: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits.
To address this issue, you can use the --set commands provided below in order to access an alternative image repository:
--set logzio-k8s-telemetry.image.repository=ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib
--set logzio-k8s-telemetry.prometheus-pushgateway.image.repository=public.ecr.aws/logzio/prom-pushgateway
--set logzio-fluentd.image=public.ecr.aws/logzio/logzio-fluentd
--set logzio-trivy.image=public.ecr.aws/logzio/trivy-to-logzio