PagerDuty Blog

Foundations of a Successful DevOps Team

This is the final post of our series about transitioning to a DevOps culture (for now). To start from the beginning check out, Why You Should Establish a DevOps Culture.

When we talk about DevOps we often defer to a conversation around collaboration and culture. One of the most important aspects of taking a DevOps approach in your organization are the tools and procedures that drive this movement. Even more importantly are your relationships with your tools and culture to create a foundation of self-service, prioritization and people.

Developer Self-Service with Infrastructure Automation

In a DevOps model, developers are empowered to do a lot on their own where previously they would have to rely on an operations team. To effectively transform into a DevOps culture, operations engineers help developers by creating tools that arm developers to solve problems on their own.

These self-service tools maintain consistency within your products’ performance, functional and physical attributes. Additionally, keeping your requirements, design and operational information in check through your product’s life cycle. Tools like Chef (which we use at PagerDuty), allow us to treat our infrastructure as code to enforce consistency across our test, development and product environments via version control, automated testing and peer reviews.

By providing developers with the tools they need to put out their own fires, more time can be spent improving your product, services and processes.

Establishing Prioritization for What You Monitor

Tracking what customers care about will help you prioritize your monitoring metrics. At PagerDuty, we prioritize our ability to accept events and send alerts to our customers above all other systems and processes. We take reliability seriously and know how much our customers depend on us to receive alerts. If they don’t get alerts they may not recognize an error in their systems, which may cause extended outages. Because we monitor and prioritize metrics customers care about, there is a sense of urgency whenever one of our monitoring tools alerts us of an incident.

Our engineers won’t delay fixing an alerting issue because they’re in the process of attending to a non-customer facing incident. By establishing priority we can quickly shift gears to make sure our service is available to you at all time.

We recommend that you figure out what part of your product and services matter most to your customers and focus your monitoring and alerting in these areas. You will find that your satisfaction rate will increase with less disruption of the services your customers care about most.

Connect People to Each Other and Your Systems

Being empowered and working together is necessary for the adoption of a DevOps culture. Tools are meant to aid and enforce our relationships with each other. For example. GitHub allows your team not only to store code, but to collaborate and share a central source of knowledge and provide version control in case something goes wrong and needs to be rolled back.

At PagerDuty, we strive to be the interconnecting layer between your tools and people so you can respond to incidents faster and reduce your mean time to repair. We hope that our service creates a sense of accountability for your team members and allow teams to work together for a unified cause (i.e. fixing a critical incident). You will not have a successful business or working product without people.

For people to effectively impact your business they need have the ability to work when they are needed. Getting people on-call to keep them aware of the systems they are accountable for is an effective way to start.

“Getting devs on-call means we are targeting the appropriate teams with actionable alerts.” – Eduardo Saito, Director of Engineering at GREE

Those who haven’t been on-call before may be resistant, but they will soon see the value of being on-call as everyone aligns their pains with the pains of their customers.

We’ve spoken quite a bit lately on how to transition to a DevOps model in your organization. If you are just starting out remember that a culture of collaboration is key. You cannot operate a DevOps model with siloed departments. This may be the biggest hurdle for your company. But if you focus your energy on empowering your team through self-service, using tools that connect people to their systems and identifying a prioritization among your common goals you will be well on your way.

Update 4/10/14 – Continuing reading the transitioning to DevOps series: