Handlers

Reference documentation

• What is a Sensu event handler?
  • Handler types
  • The default handler
• Pipe handlers
  • Pipe handler commands
    • What is a pipe handler command?
    • Pipe handler command arguments
    • How and where are pipe handler commands executed?
• TCP/UDP handlers
• Transport handlers
• Handler sets
• Handler configuration
  • Example handler definition
  • Handler definition specification
    • Handler name(s)
    • HANDLER attributes
    • socket attributes (TCP/UDP handlers)
    • pipe attributes (Transport handlers)s

What is a Sensu event handler?

Sensu event handlers are actions executed by the Sensu server on events, such as sending an email alert, creating or resolving an incident (e.g. in PagerDuty, ServiceNow, etc), or storing metrics in a time-series database (e.g. Graphite).

Handler types

There are several types of handlers. The most common handler type is the pipe handler, which works very similarly to how checks work, enabling Sensu to interact with almost any computer program via standard streams.

• Pipe handlers. Pipe handlers pipe event data into arbitrary commands via STDIN.
• TCP/UDP handlers. TCP and UDP handlers send event data to a remote socket (e.g. external API).
• Transport handlers. Transport handlers publish event data to the Sensu transport.
• Handler sets. Handler sets (also called “set handlers”) are used to group event handlers, making it easy to manage groups of actions that should be executed for certain types of events.

The default handler

Sensu expects all events to have a corresponding handler. Event handler(s) may be configured in check definitions, however if no handler or handlers have been configured, Sensu will attempt to handle the event using a handler named default. The default handler is only a reference (i.e. Sensu does not provide a built-in default handler), so if no handler definition exists for a handler named default, Sensu will log an error indicating that the event was not handled because a default handler definition does not exist. To use one or more existing handlers as the default, you can create a Set handler called default and include the existing handler(s) in the set.

Keepalive

Sensu Client keepalives are the heartbeat mechanism used by Sensu to ensure that all registered Sensu clients are still operational and able to reach the Sensu transport. By default keepalive events are handled by the default handler.

The keepalive scope can be configured to use specific handlers on the client, as well as overriding the default threshold values.

For example:

{
  "client": {
    "name": "i-424242",
    "...": "...",
    "keepalive": {
      "handlers": [
        "pagerduty",
        "email"
      ],
      "thresholds": {
        "warning": 40,
        "critical": 60
      }
    }
  }
}

Pipe handlers

Pipe handlers are external commands that can consume event data via STDIN.

Example pipe handler definition

{
  "handlers": {
    "example_pipe_handler": {
      "type": "pipe",
      "command": "do_something_awesome.rb -o options"
    }
  }
}

Pipe handler commands

What is a pipe handler command?

Pipe handler definitions include a command attribute which are literally executable commands which will be executed on a Sensu server as the sensu user.

Pipe handler command arguments

Pipe handler command attributes may include command line arguments for controlling the behavior of the command executable. Most Sensu handler plugins provide support for command line arguments for reusability.

How and where are pipe handler commands executed?

As mentioned above, all pipe handlers are executed by a Sensu server as the sensu user. Commands must be executable files that are discoverable on the Sensu server system (i.e. installed in a system $PATH directory).

NOTE: By default, the Sensu installer packages will modify the system $PATH for the Sensu processes to include /etc/sensu/plugins. As a result, executable scripts (e.g. plugins) located in /etc/sensu/plugins will be valid commands. This allows command attributes to use “relative paths” for Sensu plugin commands;

e.g.: "command": "handler-irc.rb"

TCP/UDP handlers

TCP and UDP handlers enable Sensu to forward event data to arbitrary [TCP or UDP sockets][t] for external services to consume (e.g. third-party APIs).

Example TCP handler definition

The following example TCP handler definition will forward event data to a TCP socket (i.e. 10.0.1.99:4444) and will timeout if an acknowledgement (ACK) is not received within 30 seconds.

{
  "handlers": {
    "example_tcp_handler": {
      "type": "tcp",
      "timeout": 30,
      "socket": {
        "host": "10.0.1.99",
        "port": 4444
      }
    }
  }
}

The following example UDP handler definition will forward event data to a UDP socket (i.e. 10.0.1.99:444).

{
  "handlers": {
    "example_udp_handler": {
      "type": "udp",
      "socket": {
        "host": "10.0.1.99",
        "port": 4444
      }
    }
  }
}

Transport handlers

Transport handlers enable Sensu to publish event data to named queues on the Sensu transport for external services to consume.

Example transport handler definition

The following example transport handler definition will publish event data to the Sensu transport on a pipe (e.g. a “queue” or “channel”, etc) named example_handler_queue. One or more instances of an external process or third-party application would need to subscribe to the named pipe to process the events.

{
  "handlers": {
    "example_transport_handler": {
      "type": "transport",
      "pipe": {
        "type": "direct",
        "name": "example_handler_queue"
      }
    }
  }
}

Handler sets

Handler set definitions allow groups of handlers (i.e. individual collections of actions to take on event data) to be referenced via a single named handler set.

NOTE: Attributes defined on handler sets do not apply to the handlers they include. For example, filter, filters, and mutator attributes defined in a handler set will have no effect.

Example handler set definition

The following example handler set definition will execute three handlers (i.e. email, slack, and pagerduty) for every event.

{
  "handlers": {
    "notify_all_the_things": {
      "type": "set",
      "handlers": [
        "email",
        "slack",
        "pagerduty"
      ]
    }
  }
}

Handler configuration

Example handler definition

The following is an example Sensu handler definition, a JSON configuration file located at /etc/sensu/conf.d/mail_handler.json. This handler definition uses the mailx unix command, to email the event data to example@address.com, with the email subject sensu event. The handler is named mail.

{
  "handlers": {
    "mail": {
      "type": "pipe",
      "command": "mailx -s 'sensu event' example@address.com"
    }
  }
}

Handler definition specification

Handler name(s)

Each handler definition has a unique handler name, used for the definition key. Every handler definition is within the "handlers": {} configuration scope.

• A unique string used to name/identify the check
• Cannot contain special characters or spaces
• Validated with Ruby regex /^[\w\.-]+$/.match("handler-name")

HANDLER attributes

The following attributes are configured within the {"handlers": { "HANDLER": {} } } configuration scope (where HANDLER is a valid handler name).

"type": "pipe"

"filter": "occurrences"

"filters": ["occurrences", "production"]

"severities": ["critical", "unknown"]

"mutator": "only_check_output"

"timeout": 30

"handle_silenced": true

"handle_flapping": true

"command": "/etc/sensu/plugins/pagerduty.rb"

"socket": {}

"pipe": {}

"handlers": ["pagerduty", "email", "ec2"]

socket attributes

The following attributes are configured within the {"handlers": { "HANDLER": { "socket": {} } } } configuration scope (where HANDLER is a valid handler name).

NOTE: socket attributes are only supported for TCP/UDP handlers (i.e. handlers configured with "type": "tcp" or "type": "udp").

EXAMPLE

{
  "handlers": {
    "example_handler": {
      "type": "tcp",
      "socket": {
        "host": "10.0.5.100",
        "port": 8000
      }
    }
  }
}

ATTRIBUTES

"host": "8.8.8.8"

"port": 4242

pipe attributes

The following attributes are configured within the {"handlers": { "HANDLER": { "pipe": {} } } } configuration scope (where HANDLER is a valid handler name).

NOTE: pipe attributes are only supported for Transport handlers (i.e. handlers configured with "type": "transport").

EXAMPLE

{
  "handlers": {
    "example_handler": {
      "type": "transport",
      "pipe": {
        "type": "topic",
        "name": "example_transport_handler"
      }
    }
  }
}

ATTRIBUTES

"type": "direct"

NOTE: types direct, fanout and topic are supported by the default RabbitMQ transport. Redis and other transports may only implement a subset of these.

"name": "graphite_plaintext"

"options": {"durable": true}