Lambda extensions enable tools to integrate deeply into the Lambda execution environment to control and participate in Lambda’s lifecycle.
To read more about Lambda Extensions, click here.
The Logz.io Lambda extension for logs, uses the AWS Extensions API and AWS Logs API, and sends your Lambda Function Logs directly to your Logz.io account.
This repo is based on the AWS lambda extensions sample. This extension is written in Go, but can be run with runtimes that support extensions.
Prerequisites
- Lambda function with supported runtime for extensions.
- AWS Lambda limitations: A function can use up to five layers at a time. The total unzipped size of the function and all layers cannot exceed the unzipped deployment package size limit of 250 MB.
Important notes
- If an extension does not have enough time to receive logs from AWS Logs API, it may send the logs at the next invocation of the Lambda function. If you want to send all the logs by the time your Lambda function stops running, you will need to add a sleep interval at the end of your Lambda function code. This will give the extension enough time to do the job.
- Due to Lambda’s execution environment lifecycle, the extension is invoked at two events -
INVOKEandSHUTDOWN. This means that if your Lambda function goes into theSHUTDOWNphase, the extension will start running and send all logs that are in the queue.
Extension deployment options
You can deploy the extension via the AWS CLI or via the AWS Management Console.
Parsing logs
By default, the extension sends the logs as strings.
If your logs are formatted, and you wish to parse them to separate fields, the extension will use the grok library to parse grok patterns.
You can see all the pre-built grok patterns (for example COMMONAPACHELOG is already a known pattern in the library) here.
If you need to use a custom pattern, you can use the environment variables GROK_PATTERNS and LOGS_FORMAT.
Example
For logs that are formatted like this:
%(app_name)s : %(message)s
we will use cool app as the app_name and the message will have strings containing whitespaces, letters and numbers.
In Logz.io we wish to have app_name, message in their own fields, named my_app and my_message, respectively.
To do so, we’ll set the environment variables as follows:
GROK_PATTERNS
The GROK_PATTERNS variable should be in a JSON format.
The key is used as the pattern name, and the value should be the regex that captures the pattern.
In our case, while app_name always stays cool app, we don’t know what message will be, so we need to set GROK_PATTERNS as: {"app_name":"cool app","message":".*"}
LOGS_FORMAT
The LOGS_FORMAT variable will contain the same format as the logs, according to the pattern names that we used in GROK_PATTERNS.
The variable should be in a grok format for each pattern name: ${PATTERN_NAME:FIELD_NAME} where PATTERN_NAME is the pattern name from GROK_PATTERNS, and FIELD_NAME is the name of the field you want the pattern to be parsed to.
Note that the FIELD_NAME cannot contain a dot (.) in it.
In our case, we want app_name to appear under the field my_app, and message to appear under the field my_message. Since we know that the logs format is as mentioned above, we will set LOGS_FORMAT as: %{app_name:my_app} : %{message:my_message}.
The logs that match the configuration above will appear in Logz.io with the fields lambda.record.my_app, lambda.record.my_message.
The log: "cool app : The sky is so blue", will be parsed to look like this:
my_app: cool app
my_message: The sky is so blue
This project uses an external module for its grok parsing. To learn more about it, see the grok library repo.
Nested fields
As of v0.2.0 the extension can detect if a log is in a JSON format, and to parse the fields to appear as nested fields in the Logz.io app. For example, the following log:
{ "foo": "bar", "field2": "val2" }
Will appear under the fields:
message_nested.foo: bar
message_nested.field2: val2
Note: The user must insert a valid JSON. Sending a dictionary or any key-value data structure that is not in a JSON format will cause the log to be sent as a string.
Upgrading from v0.0.1 to v0.1.0
If you have Lambda extension v0.0.1 and you want to upgrade to v0.1.0+, to ensure that your logs are correctly sent to Logz.io:
- Delete the existing extension layer, its dependencies, and environment variables as decribed below in this topic.
- Deploy the new extension, its dependencies, and configuration as described below in this topic.
Log in to your Logz.io account and navigate to the current instructions page inside the Logz.io app. Install the pre-built dashboard to enhance the observability of your logs.
To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.
Deploying Logz.io logs extension via the AWS CLI
Deploy the extension and configuration
If you haven’t done it already, install and configure the AWS CLI.
Add the layer to your function and configure the environment variables using the following command:
aws lambda update-function-configuration \
--function-name <<FUNCTION-NAME>> \
--layers <<LAYERS>> \
--environment "Variables={<<ENV-VARS>>}"
This command overwrites the existing function configuration. If you already have your own layers and environment variables for your function, list them as well.
| Placeholder | Description | Required/Default |
|---|---|---|
<<FUNCTION-NAME>> |
Name of the Lambda Function you want to monitor. | Required |
<<LAYERS>> |
A space-separated list of function layers to add to the function’s execution environment. Specify each layer by its ARN, including the version. For the ARN, see the ARNs table in the Environment Variables & ARNs tab. | |
<<ENV-VARS>> |
Key-value pairs containing environment variables that are accessible from function code during execution. Should appear in the following format: KeyName1=string,KeyName2=string. For a list of all the environment variables for the extension, see the Lambda environment variables table in the Environment Variables & ARNs tab. |
Run the function
Use the following command. It may take more than one run of the function for the logs to start shipping to your Logz.io account.
aws lambda update-function-configuration \
--function-name <<FUNCTION-NAME>> \
--layers [] \
--environment "Variables={}"
Your lambda logs will appear under the type lambda-extension-logs.
This command overwrites the existing function configuration. If you already have your own layers and environment variables for your function, include them in the list.
Check Logz.io for your logs
Give your logs some time to get from your system to ours.
Log in to your Logz.io account and navigate to the current instructions page inside the Logz.io app. Install the pre-built dashboard to enhance the observability of your logs.
To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.
Deleting the extension
To delete the extension and its environment variables, use the following command:
aws lambda update-function-configuration \
--function-name some-func \
--layers [] \
--environment "Variables={}"
This command overwrites the existing function configuration. If you already have your own layers and environment variables for your function, include them in the list.
Deploying Logz.io log extensions via the AWS Management Console
You’ll have to add the extension
Add the extension to your Lambda Function
-
In the Lambda Functions screen, choose the function you want to monitor.
-
In the page for the function, scroll down to the
Layerssection and chooseAdd Layer.
-
Select the
Specify an ARNoption, then choose the ARN of the extension with the region code that matches your Lambda Function region from the ARNs table in the Environment Variables & ARNs tab, and click theAddbutton.
Configure the extension parameters
Add the environment variables to the function, according to the Environment variables table in the Environment Variables & ARNs tab.
Run the function
Run the function. It may take more than one run of the function for the logs to start shipping to your Logz.io account.
Your lambda logs will appear under the type lambda-extension-logs.
Check Logz.io for your logs
Give your logs some time to get from your system to ours.
Log in to your Logz.io account and navigate to the current instructions page inside the Logz.io app. Install the pre-built dashboard to enhance the observability of your logs.
To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.
Deleting the extension
- To delete the extension layer: In your function page, go to the layers panel. Click
edit, select the extension layer, and clicksave. - To delete the extension’s environment variables: In your function page, select the
Configurationtab, selectEnvironment variables, clickedit, and remove the variables that you added for the extension.
Environment Variables
| Name | Description | Required/Default |
|---|---|---|
LOGZIO_LOGS_TOKEN |
Your Logz.io log shipping token. | Required |
LOGZIO_LISTENER |
Your Logz.io listener address, with port 8070 (http) or 8071 (https). For example: https://listener.logz.io:8071. Use the listener URL specific to the region where your Logz.io account is hosted. Click to look up your listener URL. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071. |
Required |
LOGS_EXT_LOG_LEVEL |
Log level of the extension. Can be set to one of the following: debug, info, warn, error, fatal, panic. |
Default: info |
ENABLE_PLATFORM_LOGS |
The platform log captures runtime or execution environment errors. Set to true if you wish the platform logs will be shipped to your Logz.io account. |
Default: false |
GROK_PATTERNS |
Must be set with LOGS_FORMAT. Use this if you want to parse your logs into fields. A minified JSON list that contains the field name and the regex that will match the field. To understand more see the parsing logs section. |
- |
LOGS_FORMAT |
Must be set with GROK_PATTERNS. Use this if you want to parse your logs into fields. The format in which the logs will appear, in accordance to grok conventions. To understand more see the parsing logs section. |
- |
CUSTOM_FIELDS |
Include additional fields with every message sent, formatted as fieldName1=fieldValue1,fieldName2=fieldValue2 (NO SPACES). A custom key that clashes with a key from the log itself will be ignored. |
- |
ARNs
| Region Name | Region Code | AWS ARN |
|---|---|---|
| US East (N. Virginia) | us-east-1 |
arn:aws:lambda:us-east-1:486140753397:layer:LogzioLambdaExtensionLogs:6 |
| US East (Ohio) | us-east-2 |
arn:aws:lambda:us-east-2:486140753397:layer:LogzioLambdaExtensionLogs:6 |
| US West (N. California) | us-west-1 |
arn:aws:lambda:us-west-1:486140753397:layer:LogzioLambdaExtensionLogs:6 |
| US West (Oregon) | us-west-2 |
arn:aws:lambda:us-west-2:486140753397:layer:LogzioLambdaExtensionLogs:6 |
| Europe (Frankfurt) | eu-central-1 |
arn:aws:lambda:eu-central-1:486140753397:layer:LogzioLambdaExtensionLogs:6 |
| Europe (Ireland) | eu-west-1 |
arn:aws:lambda:eu-west-1:486140753397:layer:LogzioLambdaExtensionLogs:6 |
| Europe (Stockholm) | eu-north-1 |
arn:aws:lambda:eu-north-1:486140753397:layer:LogzioLambdaExtensionLogs:6 |
| Asia Pacific (Sydney) | ap-southeast-2 |
arn:aws:lambda:ap-southeast-2:486140753397:layer:LogzioLambdaExtensionLogs:6 |
| Canada (Central) | ca-central-1 |
arn:aws:lambda:ca-central-1:486140753397:layer:LogzioLambdaExtensionLogs:6 |
| South America (São Paulo) | sa-east-1 |
arn:aws:lambda:sa-east-1:486140753397:layer:LogzioLambdaExtensionLogs:7 |
| Asia Pacific (Tokyo) | ap-northeast-1 |
arn:aws:lambda:ap-northeast-1:486140753397:layer:LogzioLambdaExtensionLogs:2 |
| Asia Pacific (Singapore) | ap-southeast-1 |
arn:aws:lambda:ap-southeast-1:486140753397:layer:LogzioLambdaExtensionLogs:2 |
| Asia Pacific (Mumbai) | ap-south-1 |
arn:aws:lambda:ap-south-1:486140753397:layer:LogzioLambdaExtensionLogs:2 |
| Asia Pacific (Osaka) | ap-northeast-3 |
arn:aws:lambda:ap-northeast-3:486140753397:layer:LogzioLambdaExtensionLogs:2 |
| Asia Pacific (Seoul) | ap-northeast-2 |
arn:aws:lambda:ap-northeast-2:486140753397:layer:LogzioLambdaExtensionLogs:2 |
| Europe (London) | eu-west-2 |
arn:aws:lambda:eu-west-2:486140753397:layer:LogzioLambdaExtensionLogs:3 |
| Europe (Paris) | eu-west-3 |
arn:aws:lambda:eu-west-3:486140753397:layer:LogzioLambdaExtensionLogs:2 |
Lambda extension versions
| Version | Supported Runtimes |
|---|---|
| 0.3.2 | .NET 6, Go 1.x, Java 17, Node.js 18, Python 3.11, Ruby 3.2, Java 11, Java 8, Node.js 16, Python 3.10, Python 3.9, Python 3.8, Ruby 2.7, Custom Runtime |
| 0.3.1 | All runtimes |
| 0.3.0 | .NET Core 3.1, Java 11, Java 8, Node.js 14.x, Node.js 12.x, Python 3.9, Python 3.8, Python 3.7, Ruby 2.7, Custom runtime |
| 0.2.0 | .NET Core 3.1, Java 11, Java 8, Node.js 14.x, Node.js 12.x, Python 3.9, Python 3.8, Python 3.7, Ruby 2.7, Custom runtime |
| 0.1.0 | .NET Core 3.1, Java 11, Java 8, Node.js 14.x, Node.js 12.x, Node.js 10.x, Python 3.8, Python 3.7, Ruby 2.7, Ruby 2.5, Custom runtime |
| 0.0.1 | Python 3.7, Python 3.8 |
If your AWS region is not in the list, please reach out to Logz.io’s support or open an issue in the project’s Github repo.