Back up and recover resources with sensuctl

PRO TIP: For disaster recovery instructions, read Restore your Sensu configuration for disaster recovery.

The sensuctl dump command allows you to export your resources to standard out (stdout) or to a file. You can export all resources or a subset of them based on a list of resource types. The dump command supports exporting in wrapped-json and yaml.

For example, to export all resources for the current namespace to a file named my-resources.yml or my-resources.json in yaml or wrapped-json format:

sensuctl dump all --format yaml --file my-resources.yml
sensuctl dump all --format wrapped-json --file my-resources.json

You can restore exported resources with sensuctl create.

NOTE: The sensuctl dump command does not export user passwords — you must add the password_hash or password attribute to any exported users resources before restoring them with sensuctl create.

In addition, sensuctl create does not restore API keys from a sensuctl dump backup, although you can use your backup as a reference for granting new API keys.

Because users and API keys require these additional steps to restore with sensuctl create, you might prefer to use the etcd snapshot and restore process as your primary backup and restore method. Take regular etcd snapshots and make regular sensuctl dump backups for extra reassurance.

Back up before a Sensu version upgrade

You should create a backup before you upgrade to a new version of Sensu. Here’s the step-by-step process:

  1. Create a backup folder. 

    mkdir backup

  2. Create a backup of the entire cluster, except entities, events, and role-based access control (RBAC) resources, for all namespaces. 

    sensuctl dump all \
--all-namespaces \
--omit core/v2.Entity,core/v2.Event,core/v2.APIKey,core/v2.User,core/v2.Role,core/v2.RoleBinding,core/v2.ClusterRole,core/v2.ClusterRoleBinding \
--format yaml \
--file backup/config.yml
    sensuctl dump all \
--all-namespaces \
--omit core/v2.Entity,core/v2.Event,core/v2.APIKey,core/v2.User,core/v2.Role,core/v2.RoleBinding,core/v2.ClusterRole,core/v2.ClusterRoleBinding \
--format wrapped-json \
--file backup/config.json

  3. Export your RBAC resources, except API keys and users, for all namespaces. 

    sensuctl dump core/v2.Role,core/v2.RoleBinding,core/v2.ClusterRole,core/v2.ClusterRoleBinding \
--all-namespaces \
--format yaml \
--file backup/rbac.yml
    sensuctl dump core/v2.Role,core/v2.RoleBinding,core/v2.ClusterRole,core/v2.ClusterRoleBinding \
--all-namespaces \
--format wrapped-json \
--file backup/rbac.json

  4. Export your API keys and users resources for all namespaces. 

    sensuctl dump core/v2.APIKey,core/v2.User \
--all-namespaces \
--format yaml \
--file backup/cannotrestore.yml
    sensuctl dump core/v2.APIKey,core/v2.User \
--all-namespaces \
--format wrapped-json \
--file backup/cannotrestore.json

    NOTE: Passwords are not included when you export users. You must add the password_hash or password attribute to any exported users resources before you can use them with sensuctl create.

    Because users require this additional configuration and API keys cannot be restored from a sensuctl dump backup, consider exporting your API keys and users to a different folder than backup.

  5. Export your entity resources for all namespaces (if desired). 

    sensuctl dump core/v2.Entity \
--all-namespaces \
--format yaml \
--file backup/inventory.yml
    sensuctl dump core/v2.Entity \
--all-namespaces \
--format wrapped-json \
--file backup/inventory.json

    NOTE: If you do not export your entities, proxy check requests will not be scheduled for the excluded proxy entities.

Back up to populate new namespaces

You can create a backup copy of your existing resources with their namespaces stripped from the record. This backup allows you to replicate resources across namespaces without manual editing.

To create a backup of your resources that you can replicate in new namespaces:

  1. Create a backup folder. 

    mkdir backup

  2. Back up your pipeline resources for all namespaces, stripping namespaces so that your resources are portable for reuse in any namespace. 

    sensuctl dump core/v2.Asset,core/v2.CheckConfig,core/v2.HookConfig,core/v2.EventFilter,core/v2.Mutator,core/v2.Handler,core/v2.Silenced,secrets/v1.Secret,secrets/v1.Provider \
--all-namespaces \
--format yaml | grep -v "^\s*namespace:" > backup/pipelines.yml
    sensuctl dump core/v2.Asset,core/v2.CheckConfig,core/v2.HookConfig,core/v2.EventFilter,core/v2.Mutator,core/v2.Handler,core/v2.Silenced,secrets/v1.Secret,secrets/v1.Provider \
--all-namespaces \
--format wrapped-json | grep -v "^\s*namespace:" > backup/pipelines.json

Restore resources from backup

When you are ready to restore your exported resources, use sensuctl create.

To restore everything you exported all at once, run:

sensuctl create -r -f backup/

To restore a subset of your exported resources (in this example, your RBAC resources), run:

sensuctl create -f backup/rbac.yml
sensuctl create -f backup/rbac.json

NOTE: When you export users, required password attributes are not included. You must add a password_hash or password to users resources before restoring them with the sensuctl create command.

You can’t restore API keys or users from a sensuctl dump backup. API keys must be reissued, but you can use your backup as a reference for granting new API keys to replace the exported keys.

Supported resource types

Use sensuctl describe-type all to retrieve the list of supported sensuctl dump resource types.

NOTE: Short names are only supported for core/v2 resources.

sensuctl describe-type all

The response will list the names and other details for the supported resource types:

         Fully Qualified Name               Short Name           API Version               Type             Namespaced  
────────────────────────────────────── ───────────────────── ─────────────────── ───────────────────────── ─────────────
  authentication/v2.Provider                                  authentication/v2   Provider                  false
  licensing/v2.LicenseFile                                    licensing/v2        LicenseFile               false
  store/v1.PostgresConfig                                     store/v1            PostgresConfig            false
  federation/v1.EtcdReplicator                                federation/v1       EtcdReplicator            false
  federation/v1.Cluster                                       federation/v1       Cluster                   false
  secrets/v1.Secret                                           secrets/v1          Secret                    true
  secrets/v1.Provider                                         secrets/v1          Provider                  false
  searches/v1.Search                                          searches/v1         Search                    true
  web/v1.GlobalConfig                                         web/v1              GlobalConfig              false
  bsm/v1.RuleTemplate                                         bsm/v1              RuleTemplate              true
  bsm/v1.ServiceComponent                                     bsm/v1              ServiceComponent          true
  pipeline/v1.SumoLogicMetricsHandler                         pipeline/v1         SumoLogicMetricsHandler   true
  pipeline/v1.TCPStreamHandler                                pipeline/v1         TCPStreamHandler          true
  core/v2.Namespace                     namespaces            core/v2             Namespace                 false
  core/v2.ClusterRole                   clusterroles          core/v2             ClusterRole               false
  core/v2.ClusterRoleBinding            clusterrolebindings   core/v2             ClusterRoleBinding        false
  core/v2.User                          users                 core/v2             User                      false
  core/v2.APIKey                        apikeys               core/v2             APIKey                    false
  core/v2.TessenConfig                  tessen                core/v2             TessenConfig              false
  core/v2.Asset                         assets                core/v2             Asset                     true
  core/v2.CheckConfig                   checks                core/v2             CheckConfig               true
  core/v2.Entity                        entities              core/v2             Entity                    true
  core/v2.Event                         events                core/v2             Event                     true
  core/v2.EventFilter                   filters               core/v2             EventFilter               true
  core/v2.Handler                       handlers              core/v2             Handler                   true
  core/v2.HookConfig                    hooks                 core/v2             HookConfig                true
  core/v2.Mutator                       mutators              core/v2             Mutator                   true
  core/v2.Pipeline                      pipelines             core/v2             Pipeline                  true
  core/v2.Role                          roles                 core/v2             Role                      true
  core/v2.RoleBinding                   rolebindings          core/v2             RoleBinding               true
  core/v2.Silenced                      silenced              core/v2             Silenced                  true

You can also list specific resource types by fully qualified name or short name:

sensuctl describe-type core/v2.CheckConfig
sensuctl describe-type checks

To list more than one type, use a comma-separated list:

sensuctl describe-type core/v2.CheckConfig,core/v2.EventFilter,core/v2.Handler
sensuctl describe-type checks,filters,handlers

Format the sensuctl describe-type response

Add the --format flag to specify how the resources should be formatted in the sensuctl describe-type response. The default is unformatted, but you can specify either wrapped-json or yaml:

sensuctl describe-type core/v2.CheckConfig --format yaml
sensuctl describe-type core/v2.CheckConfig --format wrapped-json

Example sensuctl dump commands

To export only checks for only the current namespace to stdout in YAML or wrapped JSON format:

sensuctl dump core/v2.CheckConfig --format yaml
sensuctl dump core/v2.CheckConfig --format wrapped-json

To export only handlers and filters for only the current namespace to a file named my-handlers-and-filters in YAML or wrapped JSON format:

sensuctl dump core/v2.Handler,core/v2.EventFilter --format yaml --file my-handlers-and-filters.yml
sensuctl dump core/v2.Handler,core/v2.EventFilter --format wrapped-json --file my-handlers-and-filters.json

To export resources for all namespaces, add the --all-namespaces flag to any sensuctl dump command. For example:

sensuctl dump all --all-namespaces --format yaml --file my-resources.yml
sensuctl dump all --all-namespaces --format wrapped-json --file my-resources.json
sensuctl dump core/v2.CheckConfig --all-namespaces --format yaml
sensuctl dump core/v2.CheckConfig --all-namespaces --format wrapped-json
sensuctl dump core/v2.Handler,core/v2.EventFilter --all-namespaces --format yaml --file my-handlers-and-filters.yml
sensuctl dump core/v2.Handler,core/v2.EventFilter --all-namespaces --format wrapped-json --file my-handlers-and-filters.json

You can use fully qualified names or short names to specify resources in sensuctl dump commands. Here’s an example that uses fully qualified names:

sensuctl dump core/v2.Handler,core/v2.EventFilter --format yaml --file my-handlers-and-filters.yml
sensuctl dump core/v2.Handler,core/v2.EventFilter --format wrapped-json --file my-handlers-and-filters.json

Here’s an example that uses short names:

sensuctl dump handlers,filters --format yaml --file my-handlers-and-filters.yml
sensuctl dump handlers,filters --format wrapped-json --file my-handlers-and-filters.json

Best practices for sensuctl dump

To reduce the running time for the sensuctl dump command, omit events and export only one namespace at a time.

Omit events from your sensuctl dump command to reduce the size of the exported payload and the system resources required to export. The most important part of a backup is capturing the Sensu configuration, and even with regular backups, events are likely to be outdated by the time you restore them. If you need access to all events, send them to a database store instead of including events in routine Sensu backups.

It takes longer to export resources from all namespaces at once than the resources from one namespace, especially as the number of resources in each namespace grows. To export resources more quickly, export a single namespace at a time.