Name: 7 Surprising Habits of High Velocity Engineering Organizations
Uploaded: 2026-02-05T17:55:07.117Z
Duration: 36 min 17 s
Description: 7 Surprising Habits of High Velocity Engineering Organizations

Transcript for "7 Surprising Habits of High Velocity Engineering Organizations": Excellent. Alright. I'll just get started. You know, I'm Ricardo Reyes. I am a senior solutions engineer with with the public sector partnership team. My background is has been primarily a, you know, software engineer. I spent fifteen years in the federal government implementing enterprise solutions. I live in the DC area. But, obviously, you know, over time, I moved into the start up world for the last eight years or so. I've been working with start ups where they built in emerging technology that goes into the market for the federal government. And so I'll talk to you today about the seven surprising habits of high velocity engineering organizations. In today's agenda, we're gonna talk about high velocity engineering, the challenges that erode engineering velocity, the frameworks that measure and improve developer experience, and talk about how we can help and give you an opportunity to ask questions. So what is high velocity engineering? You know, it is possible, but usually what we hear is, like, there's myth. Right? It means that you have to there's chaos. There's high, you know, engineers or you have higher engineers. You have to increase the number of developers that can help you with that, you have to cut corners or break things because, obviously, you have to move faster. And you can buy it off the shelf, right, instead of building yourself. But, usually, what it means is that, you know, you want to have velocity engineering by eliminating friction in your operation. If you're a software factor engineering company and you're trying to build software every day and you wanna scale it faster, you want to figure out those friction points and eliminate that by automating it through different automation tools that exist in the market today and developing systems that build for they're built for complexity. Right? It's not about working harder. It's about working smarter, using the tools, and improving that process. Why does it matter? It matters because high velocity engineering accelerates innovation, you know, while strengthening resiliency and reducing organizational risk. There's a lot of risk out there when you move fast, but we have to look at those elements and figure out how do we actually improve that process by controlling it. And the challenges that erode engineering velocity, you know, we talk about that because Chainguard conducted a survey of 1,200 engineers and leaders in this space to examine the state of the modern developer experience. We wanted to focus and study what can we do today to help improve those practices. And so one thing that we discovered was the engineering toil trap, maintenance over innovation. You know, 79% of the engineers pointed to code maintenance as a major drain in their time, meaning that engineers report spending more than only 16% of their week working on writing new code or writing code, building features, right, despite that being their primary, more rewarding, fulfilling task. And then 72% of engineers said that the demand of their time make it difficult to find space for building new features. You know, I can relate to this. You know, I spent a lot of time in my career implementing technology for the federal government and running technology companies where you can see that, you know, the maintenance become a problem. I spent six years at home and security where we built, you know, a lot of technology CMS systems that were built with open source. And, typically, when you deal with open source, you're gonna deal with a lot of that maintenance toil. Right? You have to continue to update components that are, you know, often being maintained by upstream maintainers, and you have to make them current. And when that happens, it becomes a challenge for your operation because you're prioritizing, you know, maintenance over feature development or getting new product developed. If you're doing that development activity, your goal is to figure out how do I optimize that. And usually, it becomes at a cost, and the cost is usually prioritizing maintenance because you don't wanna have an issue in your application that breaks when you're writing it in production. So, certainly, that's something that I can relate to. That's definitely a pain that we can see across many different engineering practices that I feel like you may also encounter. The question is how do you handle that? How do you improve that process and make those engineers happy? Because sometimes you may have turnover. Right? You may have engineers who are like, I don't wanna do maintenance. I wanna write new features. I wanna learn. I wanna be able to build new capability. I wasn't hired to write, you know, maintain code. That was not my purpose on this job. So you may have turnover. I've placed that as a leader in the past where I led a large team of engineers doing this and they would leave because they wouldn't be happy with or satisfied with the accrual that they had. So one of the key things as a leader, if you're a leader in the practice, is to make sure that you keep that in mind. Right? To keep that in mind of that relationship with a developer while dealing with this engineering toil, that you wanna keep them satisfied, keep them happy in terms of learning and adapting to new capabilities and technology. Another key factor is security as a blocker. Right? The cost of patching as a technical debt. 40% of engineers spend significant amount of time on upgrades. I think we all do that. It breaks changes. You know? The patching is a very painful experience when you have to update the the version, right, on technology, especially on open source technology components. And the vulnerability resolution. If you work with security, you're gonna face that as the biggest risk in operation or running a system, an application you're writing, or implementing or going to market with. You can't have flaws in your code. So the abilities have to be addressed, and that's a big blocker. And then 60% of the respondents frequently, you know, came across technical debt that impacts their ability to deliver work effectively. And I can speak to this in my experience, you know, I. E. After I spent fifteen years in the federal government building enterprise solutions, I went and formed a tech startup company with my friend of mine. And we spent three years building that technology, hiring a team, and outsourcing the work to help us with the operation of that, run an application, build the technology faster, right, get into MVP. But what we didn't realize is that over those three years, technology would change. For example, I was using AngularJS, we cache TV, you know, and they became deprecated. Those technologies were no longer, you know, being maintained by upstream. And I had to then take the responsibility to figure out how do I upgrade to the new version of those Angular, right, Angular two point o. And that became a very costly operation because I couldn't do it. I had to prioritize building a new feature that I already implemented with a customer base, or do I want to focus on just upgrade for six months and rewriting the entire code base? And that basically sum summarized the business. After three years and, you know, millions of dollars, let's say, we lost the business because we couldn't actually resolve the security issues. Our customers were not gonna, you know, compromise their data and operate with our system if our system wasn't up to par with their security requirements. Something that I've learned, right, you can't see my scars, but I actually have this pain and experience the lessons that I've learned by dealing with security as a major issue. Fermentation, right, tools for all. It may not yet be, you know, integrated properly with the team that is managing those tools. So I think what you have to consider is, you know, fragmentation is a problem. Right? Tools is problem because I've faced that myself. When you're switching between different tool sets, you have to then, you know, allow your workspace. And that alone could take some time. Go figure figure out where you left off as a programmer, right, developer. And that's just downtime. That's not counted on your hours that you have to charge for some projects. Right? And so I think that this is something that needs to be considered as a fact and estimated properly in your team. So again, this is something that we discover as a major pain in the operation of a software factor a software company that's building software and implementing technology that deals with, like, you know, modern application development. Alright. But there's ways to improve this, right, and measure velocity, and this is what we're gonna talk about next. And one of the things that we see across the board of organizations that are doing it well is that they fight the toil with automation. I think you all know that automation is safe to help, not to just, you know, compromise our data or intelligence, the information that we have. You know, we want to be able to build, you know, systems that are through pipelines, automate that through compliance checks, scanning and testing, and infrastructure provisioning and deployments. That helps with automation, with an operation of a software company. If you're doing a software engineering today, you want to have these processes. You want to be able to automate activities that are always repeatable. Right? Because if you're not doing it for to automation, you're gonna have to do it manually. And manually means more time. It means more cost and more delays and not efficient. Right? You can't accelerate as fast as you want because you're gonna have to have moved things, you know, based on your manual steps. So how do we get there? You know, to get there, you wanna, you know, create workflows and processes that hone in steps that seem second nature. You know, for me, it would be like, if you're gonna do testing, perhaps consider rewriting your test case through automation. Don't do it manually. Right? Because you may not require a resource to do that and do functional testing, and that's gonna be very expensive and timing consuming. So I think creating an automation process is key for testing, in my opinion. And, also, like, one of the things that I did effectively well was building pipelines. You know? If you're gonna do desktop ops, you have to build the proper pipeline. You you know, figure out where you are today and automate some of the activities so that you don't have to move it manually. And I think most organizations do that today, but you have to figure out how complex you wanna get. And then identify areas of friction, right, bottlenecking and processes that basically automate the manual process like we've been saying. Figure out anything that deals with scanning, infrastructure provisioning. Oftentimes, a lot of organizations may separate the, you know, desktop activity from the engineering practices, and so you need to figure out how do I set up a production environment without having to be delayed where I have to ask permission to set up a, you know, a cloud infrastructure. And that usually delays activity, then it goes and automate the process. So I also create a short term and long term road map. To me, it's like that's key. Because if you only focus on the current state and not the long term state, you may avoid implementing a solution that's gonna work long term. Again, one of the things that I've seen here in the federal government and that I'm working with on this daily basis is that every organization is a different maturity level. Not everybody is going to be running or flying, right? You have to meet them, out where you are today, take the steps to help improve the desktops, improve the operation through automation, leverage the tools that exist in the market today to help you develop, you know, procedures that help you get to market faster or get the code scanning properly, get the artifacts you need to validate activities in your code base, or perhaps look at the actual automation test results and automate some of that aspect of the data and present those tickets to the development team so you know exactly what things are working and were not working properly. But, again, effectively, the goal is to get to full automation as much as you can so you can have a streamlined process and get to market faster or get your product to release more often. And, again, we also wanted to build more secure by default systems. VOSS, again, I need some water here. Let's see if there's any questions online so far. Let me pause there. I don't see any questions, but I wanted to give you the opportunity to ask questions throughout the presentation as well. Again, we wanna look at security by default systems. Right? Oftentimes, we forget that when developers write code today, they're writing code by leveraging open source technology. And usually, what that means is that most of the code that you're using in your code base is not written by the actual developers who are you hire or your team that's actually implementing the application or yourself. You're looking at open source components, you know, that you leverage from, you know, let's say, Python, Java, and Node. Right? If you're building one of those ecosystems, you're gonna probably reuse libraries and components that are necessary for your application so you don't have to rewrite it from scratch. Typically, what that means is that you're gonna keep a software that's gonna make sure it's secure, but also minimize the open source artifacts, reduce that attack surface that you may encounter from open source, and then harden the built environments because you don't wanna give root access to a lot of those tools and services that you're running in production. It removes packages, package managers, for example, if you're running containers. And then, obviously, you know, you wanna make sure you know how to mitigate zero you know, pseudo vulnerabilities in general. You know, continuously update container images so that you don't have CVEs. Because if you have CVEs, that compromises your operation in your system, and you may be impacted, you know, both from a functional perspective and actual financial or legal. So how do we get there? Right? I think the key is to choose technology and tools that are secured by default, right, and create a process and workflow for application and cloud security with the teams that you have in mind to ensure that you are meeting those requirements. Knowing the requirements upfront is key. You know, you have to partner with your Cisco and security officer to ensure that you know exactly what the requirements are gonna be before you start writing a line of code, in my opinion. This is something that I faced in my career at home security where, you know, my first priority was as a leader, as a lead engineer, was like, hey. I need to work with a CISO, a security officer, and say, hey. What are your requirements for this, you know, critical environment? Is that a high moderate system? And I'm like, okay. Do I need to meet all the standards? And what are the requirements to ensure that I can start building the control gates and the security controls that are needed for that application. Right? So those are the things that you need to sort of uncover before you write the code and document those requirements as you would design your application or your system, and then also choose the right technology to help you get there. So, again, meet the short term and long term goals by identifying those early and figuring out what technology works best for you. Alright. Measure what matters. Right? This is where you look at the actual DevOps research and assessment metrics that was originally coined by Google, but it's become the industry standard now. This is gonna measure, like, your surface delivery performance, figure out how fast can I get my code to production, you know, and also look at the actual deployment frequency, in the meantime, to recover, right, and the change failure rate? This is where I think it's critical to get that prioritization done, understand how this is helping helping you as an organization identify data points that can help you improve your existing process and adapt and perhaps implement and iterate changes in the process. How do we get there? Align KPIs, you know, and actually support those metrics by working with leaders to implement these efficiencies. You know, either look at our tools and minimize the manual work and also, you know, evangelize metrics internally. Right? Translate velocity and compliance data into stakeholder priorities, like roadmaps and feed in AR growth. I mean, this is more like figure out how fast can I get my software to production without impact and operational cost so that we can get a product or feature out faster and learn from that and optimize that metric data to help you in your leadership and your teams in product management, for example, to figure out what the bottlenecks are so you can optimize that process? Again, I can see that in the past. You know, for me, it's about deploying more efficiently because sometimes you don't want to delay deployment or release a software that's expected based on the timeline and objectives, especially if you have a deadline to a customer and you wanna get that out faster and figure out what those pain points are. Then we go into reducing cognitive load, right, with standardization. You know, high velocity organizations, you know, abstract complexity and make that path to production smooth and simple. And typically you want to look at the pave the path. You know, developer evangelization is important. You need to streamline the development experience and onboarding and documentation and awareness. Something that, you know, Chainguard does really well here is that, you know, we're a high performing company. We have one of the best engineers in the market, and all engineers are really self sustained, very efficient with their time, and they build because we have everything streamlined like we're talking about. The cognitive load is reduced. It's standardized by looking at the actual developer eventualization process with the devrels team and we streamline by documenting everything. And onboarding is quite seamless. So again, you want to look at how do we get there. Right? You wanna look at the internal development platforms, you know, capabilities and start looking at that narrow that use case and the paved path, figure out how do I use tools so I don't hate this process of adoption, onboarding. And also figuring out teams, you know, that can actually help develop best use cases, evangelize that across the board so you can say, hey. I have this use case now. I can actually get to that particular goal that I have, and my onboarding process is gonna be seamless. Right? If I hire a new developer today, how much is that gonna cost me in terms of time and resource to be able to train and onboard properly before they can be effective? And, obviously, you don't wanna dedicate a lot of cycles on that, so your goal is to be able to, like, create a seamless process to help you with the actual onboarding, to help you with the actual development task. And if you have a process already developed with documentation that's that's visible and transparent to them, they could be self sufficient. At Chainguard, we have the edu.chainguard.dev, and that website basically has everything documented in terms of how our products work, how our systems operate, and when our limitations are. Next, embrace small and frequent releases, right? A high velocity release strategy drives continuous innovation and creates unstoppable momentum. What does that mean? I mean, it means lower the pressure, you know, smaller failures, frequent wins, and constant customer value. To me to me, how do we get there? Let me ask answer that question first. You know, I shared my experience and how we actually get achieve that. Obviously, you wanna launch a an IDP focused use case to gain early buy in. Right? Once you get an early buy in, then that means people are gonna follow that same process. And and so the key is, like, how do you lower the pressure and how do you, you know, minimize the failures? And that's important. And frequent wins as well and constant customer value. You have to get the customer feedback to be able to iterate and improve your product. If you don't get that, you're not gonna be able to So it modulates the value and incentivize adoption, right, with strong documentation and, again, expand the use cases to drive broader adoption and reduce developer friction. So I can speak to this in terms of, like, how do I actually improve this? You know, I spent a lot of time in the where I would spend I started writing, it was more like a waterfall approach with a nudge. So when you're doing waterfall, you definitely have to plan ahead, design everything, document everything, and implement a solution, and you have to freeze your pill, and you have to release your production. So you have to create a project. And that project plan And by the time you get there, if it didn't work, then you meet the customer's business needs, that's a big picture. Right? You don't wanna face that. So what I prefer is agile. You know, you even with agile, you have to make sure that you can have more frequent updates. In my startup, when I ran it, one of the challenges that I faced was, okay. This is the customer requirement. I gotta be able to develop this requirement and complete it as a functional, you know, feature. Right? You create that feature, and then you wanna release that as fast as possible so that you can give you feedback. Because if they don't give you feedback, it doesn't meet the mark, then you have to go back to the drawing board. You have to rewrite the entire code base. And that's just cost and time that you can't get back. So those are the things that I'm seeing here that are important in your operation and how you review your engineering practice. I'll pause there just if there's more questions while I increase some water. Let me see. Alright. No questions so far. So, again, embrace small frequent releases. Right? Everybody is happy when you're everything's working, and you have more features out, and you have more deployments to production, and nothing's breaking. And that means customers are happy because you have a new feature up and released, and they can actually play with it and use it. So create a culture of learning. I think this is important as well. Build teams' confidence. Right? Create a resiliency to adapt quickly and innovate fearlessly because blameless you know, you have to can't blame people when things go wrong. And if you do, you're gonna have a challenge managing personalities. Right? Managing teams that are not be happy, you're gonna have to, like, deal with that sort of relationship problem. And you have to focus on the code, focus on the application, figure out what happened if you're doing a sprint retro meeting, for example. You have to identify what worked, what didn't work. So you want to look at the postmortem without blaming, making sure people are honest and they don't take it personal. Make sure that they are open and critical about what really transpired in that favor, that system, and openly share what their experience was or what they observed that didn't work. And, obviously, figure out experimentation, you know, determine exactly what's working and not working so you can document that and learn from it. So how do we get there? Right? You know, you have to have start from the top of the leadership. You gotta go to the CTO. Right? The leaders who prioritize the, you know, postmortem and elevate the feedback consistently. You have to make sure they give you the feedback, honest feedback without blame, And you have to understand exactly what in your operation is working and what's not working and recognize those who embody the values and improve the process because you have to, like, ensure that people are not blamed, in particular, people that are new to the team. Imagine if you're a developer and you just joined that team, you know, you something else caused that you've wrote as a code. Are you gonna cover your, cover your cracks, or are you gonna be honest with what happened? Clearly, this is something you have to, like, allow them to learn from and allow them to not be, you know, blaming their their lack of skills and say, well, the lack of their mistake that they made because you wanna make sure they expand and learn from that experience. Again, another thing you wanna do is align engineering with business outcomes. You know, high velocity isn't about moving fast, right, in circles. It's about moving fast in the right direction. Successful organizations design engineering priorities with business goals. You know, figure out the customer value. What is that product or feature you're building that's gonna give that customer some actual interest for them to use it or buy that product feature, right, or expand that value to that software they're buying from you because they they can't just, you know, receive a value without understanding what that is. Look at the revenue. You know? Figure out if that's actually working out, and you're going in the right direction as a business leader, as a business owner, or as a operational manager. Right? And also look at strategically in event as an advantage. And how do we get there? You know? Leaders reinforce each team's contribution to the organization's priorities. Right? Track and share KPIs. You know? Keep performance indicator, right, that their business is an impact over time. Showcase team's contribution, their challenges to maintain momentum. And to me, this has something a personal, I would say. Yesterday evening, my friend whom I did the business with in the past called me, and he's like, no. I've known him for thirty years. He shared with me that he is there's a lot of challenges at his company. Right? He realized that the KPIs weren't documented. There was no actual, you know, key measurable outcomes that they needed to determine properly because leadership wasn't documenting. And he's the AI director of a fintech company that he leads. And what he shared with me is, like, you know, he got frustrated to the point that he had a meeting with everybody, including the founders and the other leaders to discuss his point point of view because there wasn't they were blaming him, making him out of scapegoat for the failures. And, obviously, I can imagine the stress that he was going through he's a lead one of leaders in the organization who feels like he's being outcast. He's not being recognized for the work he's done or that the fact that he's not, you know, being valued. And I think that's the key factor here to document the KPIs, right, and track them, make sure that the goals are being tracked properly and documented so that you know exactly what the objectives are from a business perspective. Alright. Now that I've talked about those findings from our survey and our key to success, I'll talk about how Chainguard can help. You know, Chainguard eliminates the engineering toil that we've been talking about. We eliminate the supply chain risk and necessary context switching because we've built trusted, compatible, open source technology on a daily basis. You know, we rebuild it from source. We hardened container images and removed malware from open source by some of the best experts in the field, you know, that know secure by default. And, also, we optimized zero CVEs and minimal attack surface because we we remove unnecessary components from packages, you know, from the container image, we remove package managers, we remove shells and no root access. So we have two products that we offer today, container images, over 2,000, hard images that are open source on a daily basis, and also our free Python, Java, and JavaScript. Here's a list of the actual containers. We provide base images, application images. So you're if running an application from scratch, let's say you're gonna use Python, hey. We provided Python with zero CVEs with an SLA of seven days for criticals and fourteen days for high, medium, and low. And we also provide application images, like, you know, if you're doing databases like MariaDB or Postgres, and also analytics tools like Grafana and so forth, even CI pipeline tools. And we provide FIPS and STIGS as well, and AI GPU compute container images. And so, again, our catalog is quite extensive, and you can find, you know, all the artifacts you need that meet the requirements for, you know, government entities that are now requiring s bombs, you know, a digital signature and attestation in Providence. So we also like I said before, we do have machine gun libraries where we remove the malware altogether because we build it from source, and we provide that build process to secure. And, also, we are patching Python CVEs at the moment. Overall, based on the topic today, we actually automate the the entire process through automation that we built here in software factory, and that does continuously every day. We look at those changes that are happening in the space for resource, and we remediate CVEs and rebuild that software, and we do testing, integration testing, and functional testing to ensure nothing breaks. And we rebuild secure by default systems, right, end to end with, again, cryptographic signatures for a lot of the data that you're looking at, your system, your artifacts, and we've reduced cognitive load. So you don't have to keep switching back and forth if you're doing testing on remediating CVEs that are open source. And that, again, is the webinar today. It's thirty minutes, and I wanted to give you an opportunity to ask questions. I'm gonna pause sharing here and see if they have questions. Alright. There's a question from Luxome. Why is it necessarily to minimize open source artifacts It is not against the file for reusability from open source community. I think I understand your questions of why is it necessary to minimize the open source artifacts is because the way technology is built today, especially in containers. Right? A lot of the containers you find are in upstream communities. They're created so that every component that's necessary to make that software is available to you. But when you look at a production workload, what you want to do is minimize that attack surface because the more software you have in production, the more risk you present to your system in your operation. So by reducing the artifacts and only keeping necessary components, you're actually only utilizing the necessary, let's say, packages and tools that are needed to run your application at runtime. You don't necessarily have that big file that's really running your desktop apps, for example. And so the key factor is, you know, by minimizing the the actual size, you're only reducing the attack surface, and you're only keeping it and maintaining it more streamlined. Right? Because you're not gonna have to patch so many more components. So that's one of the key values there as to why many organizations do that by hardening images and removing the necessary components that are needed for the application. Is there another question? Do you have any any metrics around other benefits such as build time reduction or even performance improvements from using linear images? I'm trying to process that question. Give me a second. It says, do you have any metrics around benefits of such as build time, production? I mean, to me, it's about security, right, and maintain maintenance. It's similar to what I said earlier. You know, if you focus primarily on the benefits of build time, right, you're also looking at the actual overall management requirements or operation of the application because you have to figure out if I have a CVE and a component x y, I have to go and patch it on all those different components. And that, to me, is about a cost. It's about the time that a resource engineer has to go and evaluate that, and you have to determine, am I really dedicating my time to maintain that code, patch that code, figure out what the CVE status is, and figure out the justification of that CVE because you have to get security buy in to ensure that you're not impacted. Right? And if you don't do that, they can't trust that the software could be go out of production, get that sign off, and the federal government, you have to get something called a authority to operate, ATO. And for that, you have to get signatures from security to say, hey. I looked into this vulnerability. I looked at this system that has a flaw, and I can share that, you know, with this confidence because we engineering already patched it, they can say, I provide a CV certification that's been corrected, and they can look at all those artifacts and look at all those flaws and say, I can confidently say that it's been corrected by the engineering team who manages this code and releases the production. So, I mean, there's to me, it's security and maintenance and operational cost to that effect. Right? Because we don't have that operational cost in mind, then, obviously, you're gonna continue to be in that doom loop where you're gonna be continually patched CVEs for software that you're not even using in production, and that's the key. If there's no component that's being used at runtime, why would you even include it in your code base in the first place? Because that, again, also adds additional cost. If you're running it on the cloud, the bigger the the artifact, the bigger the file, the bigger the resource you're running, that costs money for hosting for you know, that sort of determines the cost on a monthly basis. If you're running a technology company, you wanna streamline that cost. So there's some other variables there as to why you wanna consider that as well. Are there any questions? I think you can ask more questions by using your microphone as well. I think I've covered the questions from the chat that were presented. Sam, did I answer your question? Did I answer your question? I wanna make sure that I properly answered it what you were asking. Awesome. Okay. Thank you, Sam. And if you're writing applications today, I'm curious to know, you know, are you using open source? And if so, how do you deal with that risk? Do that do you mitigate yourself, or do you work with security teams who actually have to write the code? Because that's usually the challenge. Right? Figuring out who's gonna be responsible for maintaining that code that's open source, figuring out the maintenance aspect of the CDEs, and addressing those CDEs head on because you have to patch. You can't keep it open. Otherwise, your system will compromise. But, again, that was our topic today, and I appreciate your time. And thank you for joining us. And there'll be more webinars in the future, and I hope you can join us. Awesome. Thank you.