Does the Rise of DevOps Means the Fall of Engineering Productivity?
Practical advice from top tech leaders like Buzzfeed CTO, Peter Wang, Instagram VP Engineering, Maria Zhang, and Sisense SVP Engineering, Leon Gendler.
I recently sat down with nearly 70 members of our Advisory board, which includes some of the most influential technology leaders in the world, to get their perspectives on the rise of DevOps. We talked about the benefits and challenges it brings, such as maintaining engineering productivity, and they gave some great tips to optimize for success.
How has the DevOps world evolved?
As the complexity of technology and the primacy of software increased in almost every industry across the globe, the role of an engineer within the software development space has changed in sync. Just a few years ago, teams were siloed in their own processes and environment. Developers, programmers, operations, admins — everybody — took care of their own processes, with little information-sharing taking place even though the ultimate goal at the end of the day was shared: to keep up with customer demand.
NewRelic accurately describes the problems with a pre-DevOps world: “On the one hand, business users demand change—new features, new services, new revenue streams—as quickly as possible. At the same time, they also want a system that is stable. Developers are willing to push out software faster and faster. Operations teams, on the other hand, know that rapid-fire changes without proper safeguards could destabilize the system. Companies feel like they have to choose between delivering changes quickly and dealing with an unstable production environment.”
Hence, the rise of DevOps: a cultural change that unified development and operations teams, enabling organizations to maintain alignment, quality, and speed. As a result, Engineers have been given more responsibility than ever before. Whether it’s related to shift-left, or simply delivering high quality software on our increasingly complex infrastructure environments, it’s clear that today, organizations are relying on Engineering functions more than ever.
So, DevOps is first and foremost, a cultural change, and to best enable it, today’s Tech Leaders must empower their Engineers with new best practices and tools in order to maintain productivity.
What do Glilot’s Technology Advisors recommend?
Best Practice #1: Ensure Everyone Does Their Part!
As the role of the Engineer grows, so does the responsibility. According to software giant Atlassian “DevOps culture is all about a shared understanding between development and operations teams, and a shared responsibility for the software they build. That means increasing transparency, communication, and collaboration across development, IT/operations, and “the business” ”. The clearer this understanding is, the better equipped engineers are to take greater responsibility for the products they are tasked with delivering.
Maria Zhang, VP Engineering at Instagram (and former VP Engineering at LinkedIn) says, “Engineers have an increased responsibility that if something goes wrong it can be detrimental to the organization. The key is to keep everyone in the loop. Educating members of the engineering team is important if you want to give autonomy in order to not break systems”.
I couldn’t agree more that continuous education and learning is a great way to ensure this mindset, such that Engineers feel this is an opportunity for growth and learning, together.
Best Practice #2: Utilize existing and innovative solutions to combat challenges such as dependency management.
As our systems have become increasingly complex as well as reliant on the other, there is always a question of whether our existing tools and processes can keep up. When looking at the software application lifecycle today, it’s clear that each phase – Plan, Develop, Deliver, Operate – relies on the next (check out this great visual from Microsoft).
Maria explained the importance of using both legacy tools and new tools to combat dependency management issues. She says, “With microservice-type infrastructure and fully-distributed systems serving hundreds of millions of users on a global scale, there is a lot of complexity behind the scenes with how the services intertwine. Dependency management became a huge focus area – especially the need to isolate the root cause of issues to prevent cascade-type failures to make systems more resilient. You might make a minor change in one of the hundreds or thousands of microservices, but when a dependency cascades, it lights up. The overall approach is a combination of looking at existing technologies plus working with innovative solutions.”
At Glilot, we are always in favor of using innovation during times of change. Lightlytics (www.lightlytics.com), for example, enables DevOps teams to continuously verify any changes made to the cloud environment and deploy with confidence. Being able to dynamically correlate the dependencies between services and infrastructure by simulating the environment in code is critical, especially given the continuous increase of our systems’ complexity, like with IaC, that Maria mentioned above.
Take another example. Lightrun (www.lightrun.com), which shifts observability left with its developer-first observability platform. It natively integrates with existing editors and IDEs to allow developers to understand, observe, and debug live applications while running.
There are some other great examples of observability tools to monitor, identify, and fix issues, such as Elastic, Datadog, Dynatrace, Splunk, among others.
Best Practice #3: Push autonomy to the developers, with proper guidance.
At the end of the day, Tech leaders want their Engineers to feel empowered. In what areas can leaders push autonomy? The shift-left movement was brought up as an example.
Leon Gendler, SVP Engineering at Sisense, suggests that a synthesis of both autonomy and guidance can be established, allowing engineers to flourish, while ensuring security (or the like) remains a primary focus; “From a security standpoint, it’s interesting to see how to push autonomy to our developers, but still there is a lot of concern (especially with open-source).”
He gave an example of how to mitigate potential dependency or collaboration gaps, “At Sisense we focus on two things in order to ease the security work of the developers: First, we introduce code-scanning tools or static code analysis tools to the pipeline in order to detect vulnerabilities in open-source components during development time and manage risks accordingly. Second, we also include a Product Security Expert as part of every design review and relevant code changes, which helps to harmonize the work between security and development personnel, and close any security gaps that might arise.”
The notion of enabling autonomy amongst developers was common around the Advisors’ table, with the idea that their freedom would bring about creative solutions to problems and ultimately, drive the success of the organization as a whole.
Best Practice #4: Increase automation to ease cross-functional collaboration.
We asked Peter Wang, the CTO of Buzzfeed his thoughts on the matter. He said, “The need for automation is greater than ever.” When we asked where automation was accelerated at Buzzfeed, he said, “The value chain that has been most automated at Buzzfeed is continuous deployment (more than testing) and communication.”
With so many different teams, there is a real urgency to ease communication. Peter said, “In fact, we have 13+ teams across various product areas; Each team has their own approach regarding roadmaps, and are working to standardize these roadmaps, which can be a painful process.. I’m a big believer in using Slackbots to automate messages, from deal wins, goal tracking, analytics reporting, infra alerts, etc. to centralize cross-functional communication.”
Along with Slackbots, some other great cross-functional collaboration tools today are Microsoft Teams, or NewRelic. These tools, and task-management tools like Trello or Monday, can definitely help simplify workflows.
Peter also emphasized that when introducing automation tools, it’s critical to set protocols in place. He said, “Having tools alone doesn’t guarantee anything – the importance is having protocols for who and what needs to be done during what scenario, especially for when there are issues in production.” This coupling of automation with clear protocols seems critical for maintaining productivity.
The move to DevOps is a transformational opportunity to foster collaboration, break down silos, and deliver better products faster than ever before. However, it has to be done right. Our Technology Advisors have given us great food-for-thought on how to optimize engineering productivity in today’s world by ensuring responsibility across the table, utilizing existing technologies as well as innovation, enabling autonomy amongst developers, and introducing not just automation tools, but the right protocols. With these four best-practices, organizations looking to maintain Engineering productivity can ensure success.
With 2020 behind us, we at Glilot are looking to employ these practices and more as we continue to support our companies.
A big thank you to Peter Wang, Maria Zhang, Leon Gendler, and the rest of our Advisors who participated in this fruitful roundtable discussion and lent their expertise on how leaders can combat the changes in today’s DevOps space to come out stronger on the other side.
For more information, please contact Kobi Samboursky, Founding Managing Partner, Glilot Capital Partners