Bulk uploads over HTTP/HTTPS

If you want to ship logs from your code but don’t have a library in place, you can send them directly to the Logz.io listener as a minified JSON file.

The listeners accept bulk uploads over an HTTP/HTTPS connection or TLS/SSL streams over TCP.

The request path and header

For HTTPS shipping (recommended), use this URL configuration:

https://<<LISTENER-HOST>>:8071?token=<<LOG-SHIPPING-TOKEN>>&type=<<MY-TYPE>>

Otherwise, for HTTP shipping, use this configuration:

http://<<LISTENER-HOST>>:8070?token=<<LOG-SHIPPING-TOKEN>>&type=<<MY-TYPE>>

Replace the placeholders to match your specifics. (They are indicated by the double angle brackets << >>):

  • Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to.

  • Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071.

  • Declare your log type for parsing purposes. Logz.io applies default parsing pipelines to the following list of built-in log types. If you declare another type, contact support for assistance with custom parsing. Can’t contain spaces. Otherwise, the default type is http-bulk.

The request body

Your request’s body is a list of logs in minified JSON format. Also, each log must be separated by a new line. You can escape newlines in a JSON string with \n.

For example:

{"message": "Hello there", "counter": 1}
{"message": "Hello again", "counter": 2}
Limitations
  • Max body size is 10 MB (10,485,760 bytes)
  • Each log line must be 500,000 bytes or less
  • Note that if you include a type field in the log, it will be overridden by the type written in the request parameter
Code sample
echo $'{"message":"hello there", "counter": 1}\n{"message":"hello again", "counter": 2}' \
  | curl -X POST "http://<<LISTENER-HOST>>:8070?token=<<LOG-SHIPPING-TOKEN>>&type=test_http_bulk" -v --data-binary @-

Possible responses

200 OK

All logs were received and validated. Give your logs some time to get from your system to ours, and then check your Logz.io Log Management account for your logs.

The response body is empty.

400 BAD REQUEST

The input wasn’t valid. The response message will look like this:

{
  "malformedLines": 2, #The number of log lines that aren't valid JSON
  "successfulLines": 10, #The number of valid JSON log lines received
  "oversizedLines": 3, #The number of log lines that exceeded the line length limit
  "emptyLogLines": 4 #The number of empty log lines
}
401 UNAUTHORIZED

The token query string parameter is missing or not valid. Make sure you’re using the right account token.

In the response body, you’ll see either “Logging token is missing” or “Logging token is not valid” as the reason for the response.

413 REQUEST ENTITY TOO LARGE

The request body size is larger than 10 MB.

Bulk uploads over TCP

If you want to ship logs from your code but don’t have a library in place, you can send them directly to the Logz.io listener as a minified JSON file.

The listeners accept bulk uploads over an HTTP/HTTPS connection or TLS/SSL streams over TCP.

JSON log structure

Keep to these practices when shipping JSON logs over TCP:

  • Each log must be a single-line JSON object
  • Each log line must be 500,000 bytes or less
  • Each log line must be followed by a \n (even the last log)
  • Include your account token as a top-level property: { ... "token": "<<LOG-SHIPPING-TOKEN>>" , ... }

Send TLS/SSL streams over TCP

Download the Logz.io public certificate to your credentials server

For HTTPS shipping, download the Logz.io public certificate to your certificate authority folder.

sudo curl https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt --create-dirs -o /etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt
Send the logs

Using the certificate you just downloaded,

send the logs to TCP port 5052 on your region’s listener host. For more information on finding your account’s region, see Account region.

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Open Search Dashboards.

If you still don’t see your logs, see log shipping troubleshooting.