# Developer Productivity in the AI4SE Era: Why Traditional Metrics Are Failing
Measuring AI4SE-era developers by lines of code, story points, or PR counts? These metrics are not just useless — they actively encourage the wrong behavior. We need a new framework.
The Disaster of Old Metrics
| Old Metric | Why It Fails Under AI4SE |
|---|---|
| Lines of code | Agents can generate huge volumes of code — that doesn’t mean value |
| Story points | Agents change completion speed — relative estimation breaks down |
| PR count | Encourages splitting into tiny PRs to pad numbers — ignores review burden |
| Commit frequency | Agents auto-commit — no longer a signal of human initiative |
Adapting the SPACE Framework for AI4SE
| Dimension | What AI4SE Should Focus On |
|---|---|
| Satisfaction | Developer satisfaction with agent collaboration |
| Performance | End-to-end outcomes (not coding speed) |
| Activity | Volume of HITL review + decision participation |
| Communication | Spec quality + review feedback frequency |
| Efficiency | End-to-end flow efficiency from spec to merge |
The Core Shift
From “how much was produced” to “how much was verified”
- Don’t look at how much code the agent generated — look at how much passed independent review
- Don’t look at how fast PRs merged — look at how many regressions followed
- Don’t look at time spent using tools — look at the closed-loop cycle time from spec to delivery
References
- SPACE Framework (Nicole Forsgren et al.)
- DevEx: Developer Experience Framework
- DORA 2025 AI Capabilities Model