High-performing engineering teams and the Holy Grail

Metrics, High-performing teams, and the Holy Grail

Jeremy Meiss Director, DevRel & Community

So back to the tech industry….

CI/CD Benchmarks for high-performing teams Duration Mean time to recovery Success rate Throughput

So what does the data say?

Duration the foundation of software engineering velocity, measures the average time in minutes required to move a unit of work through your pipeline

Duration Benchmark <=10 minute builds “a good rule of thumb is to keep your builds to no more than ten minutes. Many developers who use CI follow the practice of not moving on to the next task until their most recent checkin integrates successfully. Therefore, builds taking longer than ten minutes can interrupt their flow.” — Paul M. Duvall (2007). Continuous Integration: Improving Software Quality and Reducing Risk

Duration: What the data shows Benchmark: 5-10mins

Improving test coverage Add unit, integration, UI, and end-to-end testing across all app layers Incorporate code coverage tools into pipelines to identify inadequate testing Include static and dynamic security scans to catch vulnerabilities Incorporate TDD practices by writing tests during design phase

Mean time to Recovery the average time required to go from a failed build signal to a successful pipeline run

Mean time to recovery is indicative of resilience

“A key part of doing a continuous build is that if the mainline build fails, it needs to be fixed right away. The whole point of working with CI is that you’re always developing on a known stable base.” — Fowler, Martin. “Continuous Integration.” Web blog post. MartinFowler.com. 1 May 2006. Web.

MTTR Benchmark <=60min MTTR on default branches

MTTR: What the data shows Benchmark: 60mins

Treat your default branch as the lifeblood of your project

Getting to faster recovery times Treat your default branch as the lifeblood of your project Set up instant alerts for failed builds using services like Slack, Twilio, or Pagerduty. Write clear, informative error messages for your tests that allow you to quickly diagnose the problem and focus your efforts in the right place. SSH into the failed build machine to debug in the remote test environment. Doing so gives you access to valuable troubleshooting resources, including log files, running processes, and directory paths.

Success Rate number of passing runs divided by the total number of runs over a period of time

Success Rate Benchmark 90%+ Success rate on default branches

Success rate: What the data shows Benchmark: 90%+ on default

Throughput average number of workflow runs that an organization completes on a given project per day

So what Throughput is ideal?

Throughput Benchmark It depends.

Throughput: What the data shows Benchmark: at the speed of your business

High-Performing Teams in 2023

Platform teams and their impact

The impact of Platform Teams Duration MTTR Success rate Throughput Identify and eliminate Set up effective monitoring Look at MTTR and shorten Map goals to reality of impediments to developer and alerting systems, and recovery time first internal and external business velocity track recovery time Set a baseline success rate, situations Set guardrails and enforce Config- and Infrastructure-as- then aim for continuous Capture a baseline, monitor quality standards across Code tools limit potential for improvement for deviations projects misconfig errors Be mindful of patterns and Alleviate as much developer Standardize test suites and CI Actively monitor, streamline, influence of external factors cognitive load from day-to- pipeline configs, i.e. shareable and parallelize pipelines config templates and policies across the org day work

Thank You. timeline.jerdog.me IAmJerdog jerdog /in/jeremymeiss For feedback and swag: circle.ci/jeremy @jerdog@hachyderm.io