Heroku Logging | Coralogix Help Center

TL;DR - Coralogix Heroku logging addon: https://elements.heroku.com/addons/coralogix

Coralogix has recently released a logging add-on that forwards all Heroku output to a Coralogix account you specify.

Or use the CLI with:

heroku addons:create coralogix:test

Then within a few seconds, any logs that your application generates will feed right into Coralogix, ready for use the high scale, high-speed search and analytics that Coralogix offers. Here’s a simple example, that shows that even with a small application generating simple log records, as soon it starts scaling, logs become hard to traverse and understand.

Sample Application

The sample application consists of a Python and Ruby application that randomly generates a cat from the Cat API every 2 seconds and logs the value.

I started with one instance of each, then scaled to three instances, here are the logs using heroku logs one one instance.

And another example of mixed logs with messages from Heroku, debug messages and errors, already getting confusing.

Throw in two more workers and tracking down the source of messages is getting even more confusing. And the sample application has two applications running three workers each.

Let’s use Coralogix instead. You can open the dashboard page, or again use the CLI and see the initial display right after you start sending logs:

CLI:addons:open coralogix

After 24 hours, Coralogix’ Loggregation feature will kick in to automatically cluster log entries and make finding what you need faster and easier. In the screenshot below you can see that Loggregation has identified that despite being ‘random’, the CatAPI repeats itself a lot, outputting the same value over 1,000 times. In this case, it’s not important, but in a mission critical application, this helps you identify patterns clearly.

Errors and broken flows are often introduced by new versions and builds of an application. Coralogix’s integration with Heroku includes an integration to Heroku Pipelines and offers an automatic status for new Heroku builds (or tags). Coralogix presents the suspicious and common errors introduced since that build as well as the alerts and anomalies which might be related to the new version release, thus allowing you to pinpoint issues to particular versions and points in time.

After 5 days of learning, Coralogix will start to send a daily report highlighting new anomalies (errors and critical messages), further helping you identify new errors, and not constantly bombard you with logging noise.

In addition to the machine learning aspect, Coralogix offers the entire set of Logging capabilities, including Log Query, a centralized live tail, user defined alerts to email or slack, and a fully hosted Kibana for data slicing and dashboards creation.

Provision the Coralogix addon and start enjoying 3rd generation Log analytics

Heroku Drain direct integration

This is a Heroku application based on FluentD, accept logs from HTTPS drains and send it Coralogix.

Setup

Check out this repository and create a Heroku application from it:

$ export LOGGER_APPLICATION=fluentd-coralogix-`uuidgen`
$ heroku apps:create ${LOGGER_APPLICATION}
$ heroku config:set PRIVATE_KEY=YOUR_PRIVATE_KEY -a ${LOGGER_APPLICATION}
$ git push heroku master

Also, you can overwrite Application and Subsystem names:

$ heroku config:set APP_NAME=YOUR_APP_NAME -a ${LOGGER_APPLICATION}
$ heroku config:set SUB_SYSTEM=YOUR_SUBSYSTEM -a ${LOGGER_APPLICATION}

Usage

Configure Heroku HTTPS log drains to publish logs to this application:

$ heroku drains:add https://YOUR-LOGGING-APP-NAME/coralogix.APP_NAME.SUB_NAME -a YOUR-APP-NAME

Private space logging

For logging in private space you need to deploy logging application and set it as logs drain:

$ export LOGGER_APPLICATION=fluentd-coralogix-`uuidgen` $ heroku apps:create ${LOGGER_APPLICATION} --internal-routing --space YOUR_SPACE_NAME $ heroku config:set PRIVATE_KEY=YOUR_PRIVATE_KEY -a ${LOGGER_APPLICATION} $ heroku config:set APP_NAME=YOUR_SPACE_NAME -a ${LOGGER_APPLICATION} $ git push heroku master $ heroku drains:set https://${LOGGER_APPLICATION}.herokuapp.com/private --space YOUR_SPACE_NAME

Attaching multiple apps to a single Coralogix account

You can add multiple apps to the same Coralogix account you created when provisioning your first Heroku app with the Coralogix add-on. All apps will have a unified billing and you will be able to filter by our tag of “application name”.

To attach multiple apps to the same Coralogix account using CLI, follow this link: https://devcenter.heroku.com/articles/managing-add-ons#using-the-command-line-interface-attaching-an-add-on-to-another-app. Or from your Heroku dashboard: configure addons -> open details for Coralogix -> attach another app.

Note

Removing your parent Heroku app, to which Coralogix was provisioned originally, will remove the Coralogix addon from all other apps that were attached to this Coralogix account in the past.