Mutators reference

Sensu executes mutators during the transform stage of the observability pipeline.

Pipelines can specify a mutator to execute and transform observability event data before any handlers are applied. When the Sensu backend processes an event, it checks the pipeline for the presence of a mutator and executes that mutator before executing the handler.

Mutators accept input/data via stdin and can parse JSON event data. They output JSON data (modified event data) to stdout or stderr.

There are two types of mutators: pipe and JavaScript.

Pipe mutator examples

This example shows a pipe mutator resource definition with the minimum required attributes:

---
type: Mutator
api_version: core/v2
metadata:
  name: mutator_minimum
spec:
  command: example_mutator.go
  type: pipe
{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "mutator_minimum"
  },
  "spec": {
    "command": "example_mutator.go",
    "type": "pipe"
  }
}

The following mutator definition uses an imaginary Sensu plugin, example_mutator.go, to modify event data prior to handling the event:

---
type: Mutator
api_version: core/v2
metadata:
  name: example-mutator
spec:
  command: example_mutator.go
  eval: ""
  env_vars: []
  runtime_assets:
  - example-mutator-asset
  secrets: null
  timeout: 0
  type: pipe
{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "example-mutator"
  },  
  "spec": {
    "command": "example_mutator.go",
    "timeout": 0,
    "env_vars": [],
    "runtime_assets": [
      "example-mutator-asset"
    ],
    "secrets": null,
    "type": "pipe",
    "eval": ""
  }
}

JavaScript mutator example

JavaScript mutators use the eval attribute instead of the command attribute. The eval value must be an ECMAScript 5 (JavaScript) expression.

This example uses a JavaScript mutator to remove event attributes that are not required — in this case, the check name and entity app_id label:

---
type: Mutator
api_version: core/v2
metadata:
  name: remove_checkname_entitylabel
spec:
  eval: >-
    data = JSON.parse(JSON.stringify(event)); delete data.check.metadata.name;
    delete data.entity.metadata.labels.app_id; return JSON.stringify(data)    
  type: javascript
{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "remove_checkname_entitylabel"
  },
  "spec": {
    "eval": "data = JSON.parse(JSON.stringify(event)); delete data.check.metadata.name; delete data.entity.metadata.labels.app_id; return JSON.stringify(data)",
    "type": "javascript"
  }
}

You can also use JavaScript mutators to do things like add new attributes and combine existing attributes into a single new attribute.

Pipe mutators

Pipe mutators produce an exit status code to indicate state. A code of 0 indicates OK status. If the mutator executes successfully (returns an exit status code of 0), the modified event data return to the pipeline and the handler is executed.

Exit codes other than 0 indicate failure. If the mutator fails to execute (returns a non-zero exit status code or fails to complete within its configured timeout), an error is logged and the handler will not execute.

Pipe mutator commands

Each Sensu mutator definition defines a command to be executed. Mutator commands are executable commands that will be executed on a Sensu backend, run as the sensu user. Most mutator commands are provided by Sensu plugins.

Sensu mutator command attributes may include command line arguments for controlling the behavior of the command executable. Many Sensu mutator plugins provide support for command line arguments for reusability.

All mutator commands are executed by a Sensu backend as the sensu user. Commands must be executable files that are discoverable on the Sensu backend system (installed in a system $PATH directory).

NOTE: By default, Sensu installer packages will modify the system $PATH for the Sensu processes to include /etc/sensu/plugins. As a result, executable scripts (like plugins) located in /etc/sensu/plugins will be valid commands. This allows command attributes to use “relative paths” for Sensu plugin commands (for example, "command": "check-http.go -u https://sensuapp.org").

JavaScript mutators

Mutators that use JavaScript are an efficient alternative to pipe mutators, which fork a process on each invocation. JavaScript mutators are evaluated by the Otto JavaScript VM as JavaScript programs, which enables greater mutator throughput at scale.

JavaScript mutators do not require you to return any value — you can mutate the events that are passed to the mutator instead. However, if you do return a value with a JavaScript mutator, it must be a string. If a JavaScript mutator returns a non-string value (an array, object, integer, or Boolean), an error is recorded in the Sensu backend log.

JavaScript mutators can use dynamic runtime assets as long as they are valid JavaScript assets.

Secrets are not available to JavaScript mutators. JavaScript mutators cannot look up events from the event store.

JavaScript mutator eval attribute

Each Sensu JavaScript mutator definition includes the eval attribute, whose value must be an ECMAScript 5 (JavaScript) expression. JavaScript mutators do not use the command attribute.

All mutator eval expressions are executed by a Sensu backend as the sensu user.

JavaScript mutator eval expressions can use the environment variables listed in the env_vars attribute. For JavaScript mutators, you can define environment variables and list the names of any environment variables that are available in your environment in the env_vars attribute.

Built-in mutators

Sensu includes built-in mutators to help you customize event pipelines for metrics and alerts.

Built-in mutator: only_check_output

To process an event, some handlers require only the check output, not the entire event definition. For example, when sending metrics to Graphite using a TCP handler, Graphite expects data that follows the Graphite plaintext protocol. By using the built-in only_check_output mutator, Sensu reduces the event to only the check output so Graphite can accept it.

To use only check output, include the only_check_output mutator in the pipeline mutator array:

---
type: Pipeline
api_version: core/v2
metadata:
  name: graphite_pipeline
spec:
  workflows:
  - name: graphite_check_output
    filters:
    - name: has_metrics
      type: EventFilter
      api_version: core/v2
    mutator:
      name: only_check_output
      type: Mutator
      api_version: core/v2
    handler:
      name: graphite
      type: Handler
      api_version: core/v2
{
  "type": "Pipeline",
  "api_version": "core/v2",
  "metadata": {
    "name": "graphite_pipeline"
  },
  "spec": {
    "workflows": [
      {
        "name": "graphite_check_output",
        "filters": [
          {
            "name": "has_metrics",
            "type": "EventFilter",
            "api_version": "core/v2"
          }
        ],
        "mutator": {
          "name": "only_check_output",
          "type": "Mutator",
          "api_version": "core/v2"
        },
        "handler": {
          "name": "graphite",
          "type": "Handler",
          "api_version": "core/v2"
        }
      }
    ]
  }
}

Mutator specification

Top-level attributes

api_version
description Top-level attribute that specifies the Sensu API group and version. For mutators in this version of Sensu, the api_version should always be core/v2.
required Required for mutator definitions in wrapped-json or yaml format for use with sensuctl create.
type String
example
api_version: core/v2
{
  "api_version": "core/v2"
}
metadata
description Top-level collection of metadata about the mutator that includes name, namespace, and created_by as well as custom labels and annotations. The metadata map is always at the top level of the mutator definition. This means that in wrapped-json and yaml formats, the metadata scope occurs outside the spec scope. Review the metadata attributes reference for details.
required Required for mutator definitions in wrapped-json or yaml format for use with sensuctl create.
type Map of key-value pairs
example
metadata:
  name: example-mutator
  namespace: default
  created_by: admin
  labels:
    region: us-west-1
  annotations:
    slack-channel: "#monitoring"
{
  "metadata": {
    "name": "example-mutator",
    "namespace": "default",
    "created_by": "admin",
    "labels": {
      "region": "us-west-1"
    },
    "annotations": {
      "slack-channel": "#monitoring"
    }
  }
}
spec
description Top-level map that includes the mutator spec attributes.
required Required for mutator definitions in wrapped-json or yaml format for use with sensuctl create.
type Map of key-value pairs
example
spec:
  command: example_mutator.go
  timeout: 0
  env_vars: []
  runtime_assets: []
  secrets: null
  type: pipe
{
  "spec": {
    "command": "example_mutator.go",
    "timeout": 0,
    "env_vars": [],
    "runtime_assets": [],
    "secrets": null,
    "type": "pipe"
  }
}
type
description Top-level attribute that specifies the sensuctl create resource type. Mutators should always be type Mutator.
required Required for mutator definitions in wrapped-json or yaml format for use with sensuctl create.
type String
example
type: Mutator
{
  "type": "Mutator"
}

Metadata attributes

annotations
description Non-identifying metadata to include with event data that you can access with event filters. You can use annotations to add data that’s meaningful to people or external tools that interact with Sensu.

In contrast to labels, you cannot use annotations in API response filtering, sensuctl response filtering, or web UI views.
required false
type Map of key-value pairs. Keys and values can be any valid UTF-8 string.
default null
example
annotations:
  managed-by: ops
  playbook: www.example.url
{
  "annotations": {
    "managed-by": "ops",
    "playbook": "www.example.url"
  }
}
created_by
description Username of the Sensu user who created the mutator or last updated the mutator. Sensu automatically populates the created_by field when the mutator is created or updated.
required false
type String
example
created_by: admin
{
  "created_by": "admin"
}
labels
description Custom attributes to include with event data that you can use for response and web UI view filtering.

If you include labels in your event data, you can filter API responses, sensuctl responses, and web UI views based on them. In other words, labels allow you to create meaningful groupings for your data.

Limit labels to metadata you need to use for response filtering. For complex, non-identifying metadata that you will not need to use in response filtering, use annotations rather than labels.
required false
type Map of key-value pairs. Keys can contain only letters, numbers, and underscores and must start with a letter. Values can be any valid UTF-8 string.
default null
example
labels:
  environment: development
  region: us-west-2
{
  "labels": {
    "environment": "development",
    "region": "us-west-2"
  }
}
name
description Unique string used to identify the mutator. Mutator names cannot contain special characters or spaces (validated with Go regex \A[\w\.\-]+\z). Each mutator must have a unique name within its namespace.
required true
type String
example
name: example-mutator
{
  "name": "example-mutator"
}
namespace
description Sensu RBAC namespace that the mutator belongs to.
required false
type String
default default
example
namespace: production
{
  "namespace": "production"
}

Spec attributes

command
description Mutator command to be executed by the Sensu backend.

NOTE: JavaScript mutators require the eval attribute instead of the command attribute.

required true, for pipe mutators
type String
example
command: /etc/sensu/plugins/mutated.go
{
  "command": "/etc/sensu/plugins/mutated.go"
}

env_vars
description Array of environment variables to use with command or eval expression execution.
required false
type Array
example
env_vars:
- APP_VERSION=2.5.0
{
  "env_vars": [
    "APP_VERSION=2.5.0"
  ]
}

As of Sensu Go 6.5.2, for JavaScript mutators, you can list any environment variables that are available in your environment in addition to defining environment variables:

env_vars:
- APP_VERSION=2.5.0
- SHELL
{
  "env_vars": [
    "APP_VERSION=2.5.0",
    "SHELL"
  ]
}

eval
description ECMAScript 5 (JavaScript) expression to be executed by the Sensu backend.

NOTE: Pipe mutators require the command attribute instead of the eval attribute.

required true, for JavaScript mutators
type String
example
eval: 'return JSON.stringify({"some stuff": "is here"});'
{
  "eval": "return JSON.stringify({\"some info\": \"is here\"});"
}
runtime_assets
description Array of Sensu dynamic runtime assets (by their names) required at runtime for execution of the command.
required false
type Array
example
runtime_assets:
- metric-mutator
{
  "runtime_assets": [
    "metric-mutator"
  ]
}
secrets
description Array of the name/secret pairs to use with command execution.
required false
type Array
example
secrets:
- name: ANSIBLE_HOST
  secret: sensu-ansible-host
- name: ANSIBLE_TOKEN
  secret: sensu-ansible-token
{
  "secrets": [
    {
      "name": "ANSIBLE_HOST",
      "secret": "sensu-ansible-host"
    },
    {
      "name": "ANSIBLE_TOKEN",
      "secret": "sensu-ansible-token"
    }
  ]
}
timeout
description Mutator execution duration timeout (hard stop). In seconds.

WARNING: The timeout attribute is available for JavaScript mutators but may not work properly if the mutator is in a loop.

required false
type integer
example
timeout: 30
{
  "timeout": 30
}
type
description Mutator type.

NOTE: Make sure to specify the type is javascript when you create a JavaScript mutator. If you do not specify the type, Sensu uses pipe as the default, expects a command attribute in the mutator definition, and ignores any eval attribute you provide.

required false
type String
default pipe
allowed values pipe and javascript
example
type: pipe
{
  "type": "pipe"
}

secrets attributes

name
description Name of the secret defined in the executable command. Becomes the environment variable presented to the mutator. Read Use secrets management in Sensu for more information.
required true
type String
example
name: ANSIBLE_HOST
{
  "name": "ANSIBLE_HOST"
}
secret
description Name of the Sensu secret resource that defines how to retrieve the secret.
required true
type String
example
secret: sensu-ansible-host
{
  "secret": "sensu-ansible-host"
}

Use secrets management in a mutator

Learn more about secrets management for your Sensu configuration in the secrets and secrets providers references.

---
type: Mutator 
api_version: core/v2 
metadata:
  name: ansible-tower
  namespace: ops
spec: 
  command: sensu-ansible-mutator -h $ANSIBLE_HOST -t $ANSIBLE_TOKEN
  secrets:
  - name: ANSIBLE_HOST
    secret: sensu-ansible-host
  - name: ANSIBLE_TOKEN
    secret: sensu-ansible-token
{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "ansible-tower",
    "namespace": "ops"
  },
  "spec": {
    "command": "sensu-ansible-mutator -h $ANSIBLE_HOST -t $ANSIBLE_TOKEN",
    "secrets": [
      {
        "name": "ANSIBLE_HOST",
        "secret": "sensu-ansible-host"
      },
      {
        "name": "ANSIBLE_TOKEN",
        "secret": "sensu-ansible-token"
      }
    ]
  }
}

Add new event attributes with JavaScript mutators

Use a JavaScript mutator to rewrite events with a new attribute added.

This example adds a new “organization” attribute to events at the top level, with a value of sec_ops:

---
type: Mutator
api_version: core/v2
metadata:
  name: add_org_sec_ops
spec:
  eval: >-
    data = JSON.parse(JSON.stringify(event)); data['organization'] = 'sec_ops';
    return JSON.stringify(data)    
  type: javascript
{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "created_by": "admin",
    "name": "add_org_sec_ops"
  },
  "spec": {
    "eval": "data = JSON.parse(JSON.stringify(event)); data['organization'] = 'sec_ops'; return JSON.stringify(data)",
    "type": "javascript"
  }
}

Combine existing attributes with JavaScript mutators

Use a JavaScript mutator to create a new attribute from a combination of multiple existing attributes and add the new attribute to events.

This example combines the event namespace and the name of the check that generated the event into a single new attribute, origination:

---
type: Mutator
api_version: core/v2
metadata:
  name: add_origination_attribute
spec:
  eval: >-
    data = JSON.parse(JSON.stringify(event)); data.origination =
    data.metadata.namespace + data.check.metadata.name; return
    JSON.stringify(data)    
  type: javascript
{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "add_origination_attribute"
  },
  "spec": {
    "eval": "data = JSON.parse(JSON.stringify(event)); data.origination = data.metadata.namespace + data.check.metadata.name; return JSON.stringify(data)",
    "type": "javascript"
  }
}