Manual Lambda configuration

At the moment, our AWS Lambda-based integrations do not support working with test events. This functionality is in development.

Create a new Kinesis data stream

If you’re not already sending your GuardDuty logs through a Kinesis data stream, create one using the Kinesis console.

Save the name of the data stream—you’ll need this in the next step.

Configure CloudWatch Events

In the CloudWatch console left menu, click Events > Rules, and then click Create rule.

In the Event Source panel (on the left), set these options:

  • Choose Event Pattern.
  • In the Build event pattern section, choose GuardDuty from the Service Name list. You can choose any Event Type that you need.

In the Targets panel (on the right), click Add target, and choose Kinesis stream. Choose the Kinesis data stream from step 1 from the Stream list.

Click Configure details (lower right corner).

Create a new IAM role

Create a new IAM role and attach the AWSLambdaKinesisExecutionRole policy to the new role.

Create a new Lambda function

This Lambda function will collect CloudWatch logs and sends them to Logz.io in bulk over HTTP.

Open the AWS Lambda Console, and click Create function. Choose Author from scratch, and use this information:

  • Name: We suggest adding the log type to the name, but you can name this function whatever you want.
  • Runtime: Choose Python 3.7
  • Role: Select Use an existing role. Then, select the role that you created in the previous step. It should have the AWSLambdaKinesisExecutionRole policy.

Click Create Function (bottom right corner of the page). After a few moments, you’ll see configuration options for your Lambda function.

You’ll need this page later on, so keep it open.

Download the Kinesis stream shipper

Download the latest Kinesis stream shipper zip file from the Logz.io GitHub page.

By default, the zip file will be named logzio-kinesis-0.0.2.zip.

Upload the zip file

In the Function code section of Lambda, open the Code entry type list and select Upload a .ZIP file.

Click Upload and select the zip file you created in the previous step (logzio-kinesis-0.0.2.zip).

Set environment variables

In the Environment variables section, set your Logz.io account token, URL, and log type, and any other variables that you need to use.

Environment variables
Parameter Description Required/Default
TOKEN (Required) Your Logz.io account token. Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to. Required
REGION Two-letter region code, or blank for US East (Northern Virginia). This determines your listener URL (where you’re shipping the logs to) and API URL. You can find your region code in the Regions and URLs table. Default: blank (US East)
URL (Deprecated) Use REGION instead. Protocol, listener host, and port (for example, https://<<LISTENER-HOST>>:8071). Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071. Required
TYPE The log type you’ll use with this Lambda. This can be a built-in log type, or a custom log type. You should create a new Lambda for each log type you use. "guardduty"
FORMAT "json" or "text". If "json", the Lambda function will attempt to parse the message field as JSON and populate the event data with the parsed fields. "text"
COMPRESS Set to true to compress logs before sending them. Set to false to send uncompressed logs. false
Configure the function’s basic settings

In Basic settings, we recommend starting with these settings:

  • Memory: 512 MB
  • Timeout: 1 min 0 sec

These default settings are just a starting point. Check your Lambda usage regularly, and adjust these values if you need to.

Set the Kinesis event trigger

Find the Add triggers list (left side of the Designer panel). Choose Kinesis from this list.

Below the Designer, you’ll see the Configure triggers panel. Choose the Kinesis stream that the Lambda function will watch.

Click Add, and then click Save at the top of the page.

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Open Search Dashboards.

If you still don’t see your logs, see log shipping troubleshooting.

Automated CloudFormation deployment

At the moment, our AWS Lambda-based integrations do not support working with test events. This functionality is in development.

Before you begin, you’ll need: AWS CLI, an S3 bucket to store the CloudFormation package

Create a new Kinesis data stream

If you’re not already sending your GuardDuty logs through a Kinesis data stream, create one using the Kinesis console.

Select the button below according to the region where you need to deploy the stack.

REGION DEPLOYMENT
us-east-1 Deploy to AWS
us-east-2 Deploy to AWS
us-west-1 Deploy to AWS
us-west-2 Deploy to AWS
eu-central-1 Deploy to AWS
eu-north-1 Deploy to AWS
eu-west-1 Deploy to AWS
eu-west-2 Deploy to AWS
eu-west-3 Deploy to AWS
sa-east-1 Deploy to AWS
ca-central-1 Deploy to AWS
ap-northeast-1 Deploy to AWS
ap-northeast-2 Deploy to AWS
ap-northeast-3 Deploy to AWS
ap-south-1 Deploy to AWS
ap-southeast-1 Deploy to AWS
ap-southeast-2 Deploy to AWS

Create stack

Keep the default setting in the Create stack screen and select Next.

Specify the stack details

Specify stack details

Specify the stack details as per the table below and select Next.

Parameter Description Required/Default
LogzioTOKEN Your Logz.io account token. Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to. Required
KinesisStream The name of the Kinesis stream where this function will listen for updates. Required
LogzioREGION Two-letter region code, or blank for US East (Northern Virginia). This determines your listener URL (where you’re shipping the logs to) and API URL. You can find your region code in the Regions and URLs table. Default: blank (US East)
LogzioURL (Deprecated) Use LogzioREGION instead. Protocol, listener host, and port (for example, https://<<LISTENER-HOST>>:8071). Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071. Required
LogzioTYPE The log type you’ll use with this Lambda. This can be a built-in log type, or a custom log type. You should create a new Lambda for each log type you use. guardduty
LogzioFORMAT "json" or "text". If "json", the Lambda function will attempt to parse the message field as JSON and populate the event data with the parsed fields. "text"
LogzioCOMPRESS Set to true to compress logs before sending them. Set to false to send uncompressed logs. false
KinesisStreamBatchSize The largest number of records to read from your stream at one time. 100
KinesisStreamStartingPosition The position in the stream to start reading from. For more information, see ShardIteratorType in the Amazon Kinesis API Reference. "LATEST"
Configure the stack options

Configure stack options

Specify the Key and Value parameters for the Tags and select Next.

Review the deployment

Review deployment

Confirm that you acknowledge that AWS CloudFormation might create IAM resources and select Create stack.

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Open Search Dashboards.

If you still don’t see your logs, see log shipping troubleshooting.