All Collections
Send your logs
Log collection from S3
Log collection from S3

Collect logs from your S3 bucket using our Lambda function

Amnon Shahar avatar
Written by Amnon Shahar
Updated over a week ago

Description

S3 to Coralogix lambda allows you to send your logs from your S3 bucket to Coralogix.

The preferred and easiest integration method will be to use our aws Serverless Application Repository. Search for 'coralogix'. Don't forget to check the 'Show apps that create custom IAM roles or resource policies' box located just under the search field to see all available applications. Select your application of choice and click on it. You will see detailed instructions in the readme section on the left.

The rest of this document describes a manual configuration of this integration and the Lambda associated with it and should be used if there is a need for special customization.

Usage

This document includes cluster dependent URL’s. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches the top level domain of your Coralogix account (.com, .in etc.). Replace the variable part of the URL with this entry.

You must add the following environment variables when using this lambda.

1. Create an "author from scratch" Node.js 10.x runtime lambda with an S3 read permissions


2. At "Code entry type" choose "Upload a ZIP file" and upload "s3ToCoralogix.zip"
https://s3-eu-west-1.amazonaws.com/coralogix-public/tools/s3ToCoralogix.zip

3. Add the mandatory environment variables: private_key, app_name, sub_name

4. Increase Memory to 1024MB and Timeout to 3 min

5. Choose the S3 bucket you want to get triggered by and change the event type from "PUT" to "Object Created(All)"

6. Multiline pattern: Coralogix supports multiline pattern by default, you can define a custom pattern with an environment variables, for example: newline_pattern [\s(?={)|(?<=})\s,\s(?={)|(?<=})\s\]. 

7. Click 'Save'

Your all set! data would be streaming to Coralogix, need help? ping us on our in-app chat for tech support or at [email protected].

Did this answer your question?