Use a Combination of Alerting Methods for the Best Result
Monitoring tools offer a very limited way to get alerts. Usually, only by email. But nobody wants to sit in front of their inbox waiting for an email while they are on-call, just incase something happens.
Through a series of blog posts we have made the case for each type of alert: phone, SMS, email or push notifications via the PagerDuty mobile app. While each of these notifications types have their pros and cons, we have found that a combination of more than one tends to be the most effective.
Straight from the experts here at PagerDuty we are excited to share some examples of how our PagerDuty team sets up our personal notifications. As a team we make sure that we have fairly tight notification rules, as escalation policies will override personal notification settings.We usually escalate incidents to the secondary on-call person in 5 minutes to create a sense of urgency for the primary owner.
The Boom-Boom-Boom In My Pants Method
To take this approach, set all notification types to occur with a zero delay. Your phone will be blowing up with push notifications, phone calls, text messages and emails that you will definitely know something is going on! Repeat every two minutes.
The Minuteman Method
For this approach, start with a combination of an email and push notification. After 1-minute add an SMS alert. Then after another minute add a voice alert. Repeat this every minute until you have acknowledged the incident or it has been re-routed to the next person in your escalation policy.
The Otherwise Silent Method
Start by receiving a combination of an email and push notification. These tend to be the quietest of the alerting types. Then set your notification rules to send a phone call after 3 minutes. Make sure that your phone is configured to ignore silent mode for PagerDuty phone numbers.
The Let My Wife (Or Husband) Sleep Method
Because they are less disturbing to those around you, start by receiving a push notification for your incidents. Then slow increase the intensity of your alerts moving to SMS then, finally, a loud phone call. Promptly apologize to your significant other if you didn’t catch the first alerts to keep the magic alive.
The Trigger Happy Method
Start with a Push Notification, SMS and Email with a zero delay. Then 1 minute later try a phone call. Repeat every minute, as necessary.
There are endless possibilities for how you can set up your personal notification rules. What’s important is finding the method that works best for you to make sure that you are responding to alerts quickly.
How do you set up your notification rules? Tell us in the comments below, and give your method a wacky name.