LLM coding tools have been available for ~2 years, with some developers claiming 5-10x productivity gains. However, these claims are not widespread, and many do not observe significant output increases. The tools can be tricky to use effectively and require workflow adjustments, often leading to wasted time fixing generated code. The actual productivity boost may be limited to experienced users handling small tasks or those not requiring complex coding. Most reported productivity gains may stem from biased sample reporting rather than concrete outcomes. The real-world impact of LLMs, especially in enabling substantial projects, lacks clear evidence and depends on advancements in LLM capabilities.
How Much Are LLMs Actually Boosting Real-World Programmer Productivity? — LessWrong
