Silencing

• Specification
• Examples
  • Silence all checks on a specific entity
  • Silence a specific check on a specific entity
  • Silence all checks on entities with a specific subscription
  • Silence a specific check on entities with a specific subscription
  • Silence a specific check on every entity
  • Deleting silencing entries

How does silencing work?

Silencing entries are created on an ad-hoc basis via sensuctl. When silencing entries are successfully created, they are assigned a name in the format $SUBSCRIPTION:$CHECK, where $SUBSCRIPTION is the name of a Sensu entity subscription and $CHECK is the name of a Sensu check. Silencing entries can be used to silence checks on specific entities by taking advantage of per-entity subscriptions, for example: entity:$ENTITY_NAME. When the check name and/or subscription described in a silencing entry match an event and a handler use the not_silenced built-in filter, this handler will not be executed.

These silencing entries are persisted in the Sensu data store. When the Sensu server processes subsequent check results, matching silencing entries are retrieved from the store. If one or more matching entries exist, the event is updated with a list of silenced entry names. The presence of silencing entries indicates that the event is silenced.

When creating a silencing entry, a combination of check and subscription can be specified, but only one or the other is strictly required.

For example, when a silencing entry is created specifying only a check, its name will contain an asterisk (or wildcard) in the $SUBSCRIPTION position. This indicates that any event with a matching check name will be marked as silenced, regardless of the originating entities’ subscriptions.

Conversely, a silencing entry which specifies only a subscription will have a name with an asterisk in the $CHECK position. This indicates that any event where the originating entities’ subscriptions match the subscription specified in the entry will be marked as silenced, regardless of the check name.

Silencing specification

Silenced entry names

Silencing entries must contain either a subscription or check name, and are identified by the combination of $SUBSCRIPTION:$CHECK. If a check or subscription is not provided, it will be substituted with a wildcard (asterisk): $SUBSCRIPTION:* or *:$CHECK.

Top-level attributes

"type": "Silenced"

"api_version": "core/v2"

"metadata": {
  "name": "appserver:mysql_status",
  "namespace": "default",
  "labels": {
    "region": "us-west-1"
  }

"spec": {
  "expire": -1,
  "expire_on_resolve": false,
  "creator": "admin",
  "reason": null,
  "check": null,
  "subscription": "entity:i-424242",
  "begin": 1542671205
}

Spec attributes

"check": "haproxy_status"

"subscription": "entity:i-424242"

"begin": 1512512023

"expire": 3600

"expire_on_resolve": true

"creator": "Application Deploy Tool 5.0"

"reason": "rebooting the world"

Metadata attributes

"name": "appserver:mysql_status"

"namespace": "production"

"labels": {
  "environment": "development",
  "region": "us-west-2"
}

 "annotations": {
  "managed-by": "ops",
  "slack-channel": "#monitoring",
  "playbook": "www.example.url"
}

Examples

Silence all checks on a specific entity

Assume a Sensu entity i-424242 which we wish to silence any alerts on. We’ll do this by taking advantage of per-entity subscriptions:

{
  "type": "Silenced",
  "api_version": "core/v2",
  "metadata": {
    "name": "entity:i-424242:*",
    "namespace": "default",
    "labels": null,
    "annotations": null
  },
  "spec": {
    "expire": -1,
    "expire_on_resolve": false,
    "creator": "admin",
    "reason": null,
    "check": null,
    "subscription": "entity:i-424242",
    "begin": 1542671205
  }
}

Silence a specific check on a specific entity

Following on the previous example, silence a check named check_ntp on entity i-424242, ensuring the entry is deleted once the underlying issue has been resolved:

{
  "subscription": "entity:i-424242", 
  "check": "check_ntp", 
  "expire_on_resolve": true 
}

The optional expire_on_resolve attribute used here indicates that when the server processes a matching check from the specified entity with status OK, this silencing entry will automatically be removed.

When used in combination with other attributes (like creator and reason), this provides Sensu operators with a method of acknowledging that they have received an alert, suppressing additional notifications, and automatically clearing the silencing entry when the check status returns to normal.

Silence all checks on entities with a specific subscription

In this case, we’ll completely silence any entities subscribed to appserver. Just as in the example of silencing all checks on a specific entity, we’ll create a silencing entry specifying only the appserver subscription:

{
  "subscription": "appserver" 
}

Silence a specific check on entities with a specific subscription

Assume a check mysql_status which we wish to silence, running on Sensu entities with the subscription appserver:

{
  "subscription": "appserver", 
  "check": "mysql_status"
}

Silence a specific check on every entity

To silence the check mysql_status on every entity in our infrastructure, regardless of subscriptions, we only need to provide the check name:

{
  "check": "mysql_status"
}

Deleting silencing entries

To delete a silencing entry, you will need to provide its name. Subscription only silencing entry names will be similar to this:

{
  "name": "appserver:*"
}

Check only silencing entry names will be similar to this:

{
  "name": "*:mysql_status"
}