Interesting post Darragh and love to seeing the AI adoption 🚀... A few questions:
- Agents reproduce whatever patterns exist in the codebase, good or bad. At 93.6% agent-driven PRs, bad patterns can spread faster than any human team could. How are you thinking about preventing that?
- For non-R&D folks Agentic coding is wild. Do you see that path eventually leading to non-engineers shipping to production?
re: reproducing good/bad patterns. yes - this requires you to be very deliberate - to be opinionated about what good looks like in various contexts - and make sure that both code generation and code review are respecting your wishes.
re: non engineers shipping to production - already happening - e.g. PM/Design shipping changes (e.g. small tweaks) but also building/shipping working prototypes/experiments. also folks outside R&D e.g. support ops team iterating on internal APIs used to power our instance of Fin, or a fun recent example, our Director of Tax shipping changes to our billing system and core product https://www.linkedin.com/feed/update/urn:li:activity:7451009289873395712/
Darragh, really appreciate the transparency! Willing to share your story at our next AUTONOMOUS summit end of May? We typically get around 15,000 participants, and this time we're exactly focused on this type of fundamental transformation stories. It would be great to have you!
Curious on how you're measuring token spend. I'd like to do similar graphs with my team's data, but finding the Claude analytics data a bit more miss than hit. FWIW we're on the plan instead of on-demand billing
For #2, why not compare to the previous years Dec-Mar? Those charts just look like typical velocity coming out of the new year.
For #3, what is the X axis? It’s honestly hard to extrapolate from this because a lot of key information is missing. Also, what does agent-driven mean in a PR if only 19% are approved by agents?
fair point on time range of comparison - I picked the period aligned with our steep ramp in PR throughput - and this wasn't the pattern we saw e.g. last year, yes we'd get some months where it'd blip up maybe 10-50% then dip down - the start of this year has been unprecedented, and all signs suggest the trend continues.
re: #3 - do you mean the code quality? x-axis is time/weeks - y-axis, is a aggregated view based on code analysis of contibutions that make the code base less complex (positive bars) or more complex (negative bars).
Agent driven means engineer is prompting Claude Code (in our case) with the context of the problem to solve, and the agent is writing all the code, submitting PR etc. 93.6% of PRs are like this.
19% AI approved, are PRs that are reviewed and approved without the need for a human reviewer.
Interesting post Darragh and love to seeing the AI adoption 🚀... A few questions:
- Agents reproduce whatever patterns exist in the codebase, good or bad. At 93.6% agent-driven PRs, bad patterns can spread faster than any human team could. How are you thinking about preventing that?
- For non-R&D folks Agentic coding is wild. Do you see that path eventually leading to non-engineers shipping to production?
Excited for the next post 👏👏👏
Thanks Bruno.
re: reproducing good/bad patterns. yes - this requires you to be very deliberate - to be opinionated about what good looks like in various contexts - and make sure that both code generation and code review are respecting your wishes.
re: non engineers shipping to production - already happening - e.g. PM/Design shipping changes (e.g. small tweaks) but also building/shipping working prototypes/experiments. also folks outside R&D e.g. support ops team iterating on internal APIs used to power our instance of Fin, or a fun recent example, our Director of Tax shipping changes to our billing system and core product https://www.linkedin.com/feed/update/urn:li:activity:7451009289873395712/
Darragh, really appreciate the transparency! Willing to share your story at our next AUTONOMOUS summit end of May? We typically get around 15,000 participants, and this time we're exactly focused on this type of fundamental transformation stories. It would be great to have you!
Curious on how you're measuring token spend. I'd like to do similar graphs with my team's data, but finding the Claude analytics data a bit more miss than hit. FWIW we're on the plan instead of on-demand billing
For #2, why not compare to the previous years Dec-Mar? Those charts just look like typical velocity coming out of the new year.
For #3, what is the X axis? It’s honestly hard to extrapolate from this because a lot of key information is missing. Also, what does agent-driven mean in a PR if only 19% are approved by agents?
fair point on time range of comparison - I picked the period aligned with our steep ramp in PR throughput - and this wasn't the pattern we saw e.g. last year, yes we'd get some months where it'd blip up maybe 10-50% then dip down - the start of this year has been unprecedented, and all signs suggest the trend continues.
re: #3 - do you mean the code quality? x-axis is time/weeks - y-axis, is a aggregated view based on code analysis of contibutions that make the code base less complex (positive bars) or more complex (negative bars).
Agent driven means engineer is prompting Claude Code (in our case) with the context of the problem to solve, and the agent is writing all the code, submitting PR etc. 93.6% of PRs are like this.
19% AI approved, are PRs that are reviewed and approved without the need for a human reviewer.
Yes— I meant Y axis, thank you for clarifying! Makes sense on this trend being different from past years.
Thank you for following up— that PR flow sounds agent-driven to me, very cool.