General guide to shipping logs with Filebeat

Filebeat for Linux
Filebeat for Windows
Modules

Filebeat is often the easiest way to get logs from your system to Logz.io. Logz.io has a dedicated configuration wizard to make it simple to configure Filebeat. If you already have Filebeat and you want to add new sources, check out our other shipping instructions to copy&paste just the relevant changes from our code examples.

Configure Filebeat on macOS or Linux

Before you begin, you’ll need:

Filebeat installed
Port 5015 open
Root access: While support for Filebeat 6.3 and later versions is available, Logz.io recommends that you use the latest stable version
Destination port 5015 open to outgoing traffic

Download the Logz.io public certificate to your credentials server

For HTTPS shipping, download the Logz.io public certificate to your certificate authority folder.

sudo curl https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt --create-dirs -o /etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt

Disabling SSL for Filebeat log shipping

By default, Filebeat uses SSL/TLS to secure the communication between Filebeat and Logz.io. However, if you want to disable SSL, you can modify the Filebeat configuration accordingly.

To ship logs without using SSL in Filebeat:

Open the Filebeat configuration file for editing. The configuration file’s location may vary depending on your operating system, but it is commonly located at /etc/filebeat/filebeat.yml or /etc/filebeat/filebeat.yaml.
Look for the output.logstash section in the configuration file.
Uncomment the # character at the beginning of the #ssl.enabled line to disable SSL. The line should now look like this: #ssl.enabled: false
Save the changes to the configuration file and restart the Filebeat service to apply the changes.

Configure Filebeat using the dedicated Logz.io configuration wizard

Filebeat requires a file extension specified for the log input.

Log into your Logz.io account, and go to the Filebeat log shipping page to use the dedicated Logz.io Filebeat configuration wizard. It’s the simplest way to configure Filebeat for your use case.

Adding log sources to the configuration file

For each of the log types you plan to send to Logz.io, fill in the following:

Select your operating system - Linux or Windows.
Specify the full Path to the logs.
Select a log Type from the list or select Other and give it a name of your choice to specify a custom log type.
- If you select a log type from the list, the logs will be automatically parsed and analyzed. List of types available for parsing by default.
- If you select Other, contact support to request custom parsing assistance. Don’t be shy, it’s included in your plan!
Select the log format - Plaintext or Json.
(Optional) Enable the Multiline option if your log messages span multiple lines. You’ll need to give a regex that identifies the beginning line of each log.
(Optional) Add a custom field. Click + Add a field to add additional fields.

If you’re running Filebeat 8.1+, the type of the filebeat.inputs is filestream instead of logs:

filebeat.inputs:
- type: filestream
  paths:
    - /var/log/*.log

Add additional sources (Optional)

The wizard makes it simple to add multiple log types to a single configuration file. Click + Add a log type to fill in the details for another log type. Repeat as necessary.

Download and validate the file

When you’re done adding your sources, click Make the config file to download it.

You can compare it to our sample configuration if you have questions.

If you’ve edited the file manually, it’s a good idea to run it through a YAML validator to rule out indentation errors, clean up extra characters, and check if your yml file is valid. (Yamllint.com is a great choice.)

Move the configuration file to the Filebeat folder

Move your configuration file to /etc/filebeat/filebeat.yml.

Start Filebeat

Start or restart Filebeat for the changes to take effect.

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Open Search Dashboards.

If you still don’t see your logs, see Filebeat’s troubleshooting guide.

Filebeat is often the easiest way to get logs from your system to Logz.io. Logz.io has a dedicated configuration wizard to make it simple to configure Filebeat. If you already have Filebeat and you want to add new sources, check out our other shipping instructions to copy & paste just the relevant changes from our code examples.

Configure Filebeat on Windows