Rules overview
Rules trigger actions when certain conditions are met. Rules evaluate conditions based on monitoring metrics or logs.
Some common scenarios of rules on metrics are:
- Notify me when a production machine is overloaded.
- Restart a service when it is not responding.
- Delete log files when the free disk space is below 10%.
Some examples of rules on logs are:
- Notify me when machines are created or destroyed in my production cloud.
- Destroy a machine when post-deployment steps fail.
- Open a ticket in my issue tracker when provisioning fails.
Rules on metrics and on logs can be configured centrally through the rules section.
Configuring rules
Before you begin with rules, please keep in mind that:
- Rules can be set and edited only by members of the Owners team. Check if your email is included there from the Teams section.
- If you would like to set rules on monitoring metrics, you need to have monitoring enabled on the relevant machines.
A rule is defined in four steps:
- apply on - The first step is to choose the resource type you would like to check. Rules can be applied either to all of them, a single one or a subset with a certain tag.
- check every - The second step is to choose how often you would like to check, up to a maximum of once per minute.
- if - The third step is to set the actual condition to check. Conditions are evaluated based on monitoring metrics or log entries. Once you select "metric" or "log", the form will expand and ask for additional input.
- then - In the last step you define the action that will be triggered when the condition is met.
When it comes to actions, Mist can do any of the following:
- alert - Mist will email an entire team, specific team members and/or a custom email address. In all cases you are able to configure the alert level and include a custom message in the email body. The custom email option is useful for sending alerts to pager services, e.g. PagerDuty etc, that handle pager policies.
- webhook - Mist will hit the webhook you provide. This is useful for feeding data to other systems, e.g. for error log aggregation. It can also be used for sending notifications on Slack. When configuring your webhook action, you must provide the relevant URL and HTTP method. Optionally, you can also set query string parameters, HTTP headers, request and JSON body.
- reboot - This option applies only on machines. When triggered, Mist will send a reboot call to the relevant machines so the services running there will not be available until the process is completed. For this reason, the reboot action requires caution and is recommended for systems that are either highly available or not mission critical.
- destroy - This option applies only on machines. When triggered, Mist will attempt to destroy the relevant machines. As in the reboot action above, this option should be treated with caution. It is mostly relevant in cases where you would like to be proactive with your spending by destroying machines that are not utilized, e.g. someone in your development team forgot an XL instance running without actually using it.
- run - This option applies only on machines. When triggered, Mist will connect to relevant machines over SSH and execute the script you defined. For example, you could be tracking memory usage of your web server and restart it gracefully whenever it goes over a certain threshold. Before using this option and because of the SSH connection requirement, please make sure the proper keys are available in the Keys section and they are correctly associated with the proper machines.
As a final note, rules can be managed over Mist’s RESTful API. You find more details regarding these API calls here.