For a while now, we’ve had a feature to send out alerts as short emails for our customers who send emails to pagers but we’ve selectively enabled it for some accounts. Based on customer demand, we’re going to turn that on for all accounts now so if you’re sending emails to a pager (or an SMS gateway*) you can enable short emails and we’ll send the same text we’d send via an SMS:

Sends:

* Just a gentle reminder: although we don’t stop you from sending emails to an SMS gateway, we can’t guarantee delivery. SMSes sent this way are not guaranteed to be delivered quickly or at all.

The new on-call scheduling tool is here. We migrated all accounts to it prior to the holiday break. Some of you have played with it. Some of you love it. Some of you don’t love it as much, but still like it a lot. I’m happy with both of those result types.

At a high level, the new PagerDuty on-call scheduler is a toolkit that allows you to create pretty much any (recurring) on-call rotation you want. Say you’re the owner of a web-based startup and you want to be on-call all the time to get notified when your site goes down. You can do that. (You could do it before, but you know… I’m being explicit.) Say you want to create a DB Secondary Rotation where you have in-house DBAs on-call weekly from 8:00am to 8:00pm, and then a daily rotating night support staff in an entirely different continent on-call overnight. You can do that. (We call that a “Follow the Sun” schedule.) Say you want to create a Frontend Dev Primary Rotation that has three developers who rotate daily, but not on weekends. You can do that. (We don’t recommend having a dead-zone where nobody is on-call, but it is technically possible.) Say you want to put your Grandma on-call for 30 minutes once a week at 7:30am on Wednesday. You can do that. (Though I don’t know why you would.)

How do you achieve all these wacky calendars designed to torture your loved ones at random intervals? Think Photoshop. Think digital image layers. What you do is basically superimpose different rotations on top of each other, leaving gaps in the higher priority layers to allow lower priority ones to “show through.” The layer gaps are like the transparent parts of an image layer that allow you to compose something more interesting.

So that’s all well and good, and actually the important part. What is less important, but more exciting to me (because I’m a dork) is all the behind the scenes magic and the actual design and usability that plays out on the front stage. I’m not going to go into that so much though. I encourage you to play with the new tool and see how it feels. Do you like the feel of smooth responsive Javascript under your mouse clicks? How do you like the non-blocking page loads and the on-the-fly calendar previewing which responds to all of your actions? These are the sorts of things we are going to push for more and more as we continue to develop the PagerDuty product.

With this project (and another one which will be announced very soon), we are starting a rollout of a site restyling and modernization of our app stack. For those of you who are up to date on your design blogs, yes, Twitter’s Bootstrap Framework does lay some basic foundation for our new layout and basic page elements. We feel that it’s best to stick with well known visual paradigms so you can intuitively jump into our (somewhat complicated) application quickly, get productive, and get your systems monitored properly without breaking too much of a sweat.

I better cut this off now as it’s getting long-winded. But, rest assured, this is just the start of even more greatness. We have been getting some great feedback during our Beta period for this feature, and we listen to your suggestions very intently. There is plenty more where this came from!

Oh, and we’ve also added iCal and Webcal integration for the calendar; stay tuned for another blog post with the full details.

This is very much a work in progress, and we’ll continue adding documentation — I’ll be watching which pages people ask for help with and prioritize them, and you can continue to email us at support@pagerduty.com

Now, you can either download your calendar for the next month, or use a secret URL that’s unique to your user profile to integrate your on-call schedule with whatever calendar app you use.  It’s read-only right now, but there are two things that I’m really excited about:

• If you share your secret URL with your team, they can add your schedule to their calendar, so managers can see the whole team at once in their calendar.
• If you use a program that gives you reminders, you can be notified before you go on call, which is a popular ask on our feedback forum

The best part is that when you get paged via phone or SMS for an issue detected by Keynote, the page will contain the full error details: the name of the affected website/webapp, the severity of the issue (Warning or Critical), and whether the issue is performance or availability related.

About Keynote

Keynote is a global leader in Internet and mobile cloud monitoring. With Keynote, you can quickly diagnose website performance problems at the application, transaction, and infrastructure levels in real-time. Their measurement infrastructure consists of over 3000 measurement computers and mobile devices in over 275 locations around the world — wow!

Integration Details

The integration between Keynote and PagerDuty is very simple to set up: it shouldn’t take more than a couple of minutes. On the PagerDuty side, we’ve created a new service type specifically designed for Keynote. It accepts and intelligently parses Keynote email alerts and triggers new incidents when Keynote detects website performance or availability issues. It can also automatically resolve open incidents when Keynote detects that the underlying problems have been fixed. Finally, the new service type allows you to set up alert filters based on the severity of the problem. For instance, you can configure it to only alert on Critical issues and ignore Warnings.

On the Keynote side, you simply need to log into your MyKeynote account and configure your alarms to send emails for both Warning and Critical issues to your PagerDuty-generated email address.

Here is the full Keynote / PagerDuty integration guide, which includes step-by-step instructions on setting up the new integration.

This feature, when enabled for Generic Email Services, ensures that all incoming email with the same subject line get appended to the log of an existing open incident with the same subject, if one exists, instead of creating a new incident.  So only one incident with that subject line will get triggered at once.  (If you are familiar with our integration API, it is the same concept as the “incident_key”.)

This feature has been part of PagerDuty’s Generic Email Services for a long time, but we’ve noticed that many are not taking advantage of it since the default setting has us opening a new incident for each incoming trigger email.  Going forward, for newly created email services, we’re going to be changing the default instead to open a new incident for each incoming trigger email subject, so be warned.

To change the setting (and only get paged once per problem by PagerDuty), simply edit your Generic Email Services and tick “Open a new incident for each new trigger email subject” under the “Incident creation” settings.

You might be saying to yourself:  ”well, that persistant barrage of PagerDuty phone calls certainly drives me insane when large-scale problems happen with my systems, but if I collapsed them into only one phone call, what happens if that phone call is missed?”

A good Escalation Policy is what you are looking for here.  If the alert is missed, it will move onto the next person in line.  Just make sure that your service’s escalation policy loops back to the beginning when it reaches the end of its escalation rules.  Just edit the policy associated with the service in question and tick the checkbox labeled “If an incident runs out of escalation rules, loop back to rule 1.”

Happy paging!

Pingdom users can now take advantage of PagerDuty’s comprehensive alerting and scheduling functionality. Users can receive Pingdom alerts by phone, SMS, and email, and can acknowledge, escalate or resolve incidents on the go. PagerDuty also adds incident tracking to Pingdom and helps support teams set up fair on-call duty schedules.

About Pingdom

Want to be the first to know when your website goes down? Pingdom tracks uptime, verifying the availability of your website from multiple locations across the world. Should something go wrong, Pingdom also offers a variety of tools to help you get to the root of the problem.

Integration Details

On the Pingdom side, simply configure downtime notifications to be sent by email to a PagerDuty-provided email address.

On the PagerDuty side, we’ve created a new service type specifically for Pingdom. It accepts and parses the alert emails to create incidents, and, just as importantly, automatically resolves them should your website come back on line.

To read more about how Pingdom can help you keep tabs on your website’s uptime, please visit their website. To learn just how easy it is to add the benefits of Pingdom monitoring to your PagerDuty account, check out the Pingdom Integration Guide.

Well, now you can! PagerDuty is happy to announce the release of a couple new REST APIs for querying your current and past incidents, as well as your various on-call schedules.

The new Incidents API allows you to query your incident data and integrate it into your custom operations dashboards. It also allows you to produce tailor-made summaries and incident reports for your ops teams on a weekly or monthly basis, or whatever time frame you like.

The new on-call Schedule API allows you to programmatically query on-call scheduling data. Using this API, you can integrate your on-call schedules with your existing calendar applications, user directories, and messaging systems such as Campfire, Hipchat or IRC. We hope that this API will help you bring better visibility into who is on-call in your organization, which can’t hurt in terms of ensuring that no problems or alerts go unattended even for short periods of time.

These two new APIs are currently read-only, but we hope to eventually broaden them so that you’ll be able to, say, create and edit your schedules via the same API. We also hope to eventually surface other REST resources (like user profiles, escalation policies, etc), via other APIs in the future as well.

For more information on our new REST APIs, or our existing monitoring integration API, please see our documentation.

As part of this new partnership, we have integrated PagerDuty with Red Gate SQL Monitor. The new integration allows SQL Monitor users to take advantage of PagerDuty’s sophisticated alerting functionality: Users can receive SQL Monitor alerts via phone call, SMS or email, set up automatic alert escalation, and handle high-severity database problems on the go from a mobile phone. PD also adds incident tracking to SQL Monitor and helps support teams set up fair on-call duty schedules.

About SQL Monitor

Ever wish you had better visibility into how your MS SQL Server databases were performing? SQL Monitor helps you better understand what’s going on with your databases by providing comprehensive monitoring and alerting for machines, clusters, servers and databases. It also offers rich reporting to help you get to the root cause of any operational issue.

Integration Details

The new 2.3 release of SQL Monitor will allow you to forward database problem alerts directly to PagerDuty. On the SQL Monitor side, the setup process couldn’t be easier: simply configure email alerts to go to a PagerDuty-provided email address.

On the PagerDuty side, we have created a new service type specifically designed for SQL Monitor. It accepts and intelligently parses SQL Monitor email alerts and will trigger new incidents when SQL Monitor detects database problems. It can also automatically resolve open incidents when SQL Monitor detects that the underlying problems have been fixed. Finally, the new service type allows you to set up alert filters based on the severity of the problem. For instance, you can configure it to only alert on high severity db issues.

To learn more about how Red Gate SQL Monitor can help you monitor the health and performance of your SQL Server databases, check out the product page here. Be sure to also take a look at our SQL Monitor integration guide to see just how easy it is to have SQL Monitor alerts delivered to PagerDuty.

The Incidents Table is an integral part of tracking and resolving your service issues and downtimes in your PagerDuty account, and it provides smooth UI interactions through heavy use of AJAX. Even though AJAX is a great method for enhancing web applications and providing a great user experience, it unfortunately breaks the browser history stack and bookmarking capabilities. This issue becomes annoying very quickly as you navigate around your account, try to send links to your coworkers, or bookmark a specific sort order, page, or filter in the table.

The fix is a little hack using the browser’s anchor tag for deep linking. Since the anchor tag is used to jump to resources within a page, it won’t reload the page when changed. Different browsers handle anchor change events differently or not at all, and so it becomes quite involved fixing this for different browsers. jQuery Address is a great plugin that handles the different cases, providing a callback for the change event. We’ve added deep linking to the incidents table. The browser will now remember all your interactions with the table as you move throughout your account or recall your bookmarks.

Hopefully this will make it even easier to navigate your incidents. As always, your comments and feedback are greatly appreciated.