Notifications and alerting

How alerting works

A notification channel tells RMON where to send an alert. A check uses only the channels selected in that check's settings.

1. The check detects a failure or a configured threshold violation.
2. RMON applies the retry setting before treating the result as an incident.
3. An alert is sent to each channel assigned to the check.
4. When the check becomes healthy again, RMON updates the state and sends or resolves the recovery event where the channel supports it.

Channels are shared settings. Create a channel once, test it, and then select it for any relevant checks in the same group.

Available channels

• Telegram: bot token and destination channel or chat.
• Slack: application token and Slack channel.
• PagerDuty: integration key and a recognizable integration name.
• Mattermost: incoming webhook and channel name.
• IncidentRelay: token and destination URL.
• Email: one or more recipient addresses and a recognizable list name.

For provider-specific preparation, see Telegram, Slack, and PagerDuty.

Tokens, webhooks, integration keys, and destination URLs are credentials. Store them only in the channel fields and rotate them if they are exposed.

Create and test a channel

1. Open Monitoring → Channels.
2. Find the required provider and click + Add.
3. Enter the credential or webhook and a clear channel name. If groups are available to you, select the group that should own the channel.
4. Save the channel.
5. Click Test on the new row and confirm that the test message reaches the expected destination.

Use names such as Operations Telegram, Critical PagerDuty, or Customer API Email. Clear names prevent users from selecting the wrong destination when editing a check.

You can edit the fields in the channel list, copy a channel's settings, or delete an unused channel. Test again after changing a token, webhook, destination, or mail configuration.

Assign a channel to a check

1. Create a new check or edit an existing one.
2. In the notification section, choose a saved destination for Telegram, Slack, PagerDuty, Mattermost, IncidentRelay, and/or Email.
3. Leave a provider set to Disabled when that check must not notify it.
4. Select the incident Priority and add a Runbook link if responders have an instruction page.
5. Save the check and verify the next result on the dashboard.

A check can notify several providers at the same time. A provider can have several saved destinations, but one destination per provider is selected in an individual check.

Retries, thresholds, and priority

• Retries: the maximum number of retry attempts before the service is marked down. Increase this carefully for services with occasional short network interruptions.
• Threshold timeout: the response time after which a reachable service is reported as too slow. Use 0 when no response-time threshold is required.
• Priority: Info, Warning, Error, or Critical. Choose the level according to customer and operational impact.
• Runbook: a link included with the alert so responders can open the correct recovery instructions.
• Expiration date: automatically disables a temporary check at the specified UTC date and time.

Start with settings that match the service objective, then review alert history. Too few retries can create noise; too many retries can delay notification of a real incident.

Troubleshooting

• The channel test fails: verify the credential, destination, provider permissions, and outbound access from RMON.
• Email does not arrive: ask an administrator to verify the mail settings, then check spam filtering and the recipient list.
• The test works but a check sends nothing: edit the check and confirm that the intended channel is selected instead of Disabled.
• The wrong team receives alerts: review the selected destination, channel name, and owning group.
• Alerts arrive too often: review retries and the response-time threshold, then confirm that the expected result for the check is correct.
• No recovery is visible: first confirm that the latest check result is healthy; then test the channel and review the alert destination.