"Premature optimization" doesn't mean what you think it means
Junior developers often seem want to discuss "premature optimization" with me when I bring up things such as scalability and performance. For those not familiar, here's the entire context of Knuth's quote.He was talking about gotos! People were using goto statements to improve efficiency and he felt that it was possible to get nearly same efficiency without using them too early when coding.
This is entirely different than what Hacker News, Reddit et al. have mangled this into.
What people believe he meant is that efficiency is not an important concern early in the lifespan of a program. What he actually meant is that major tradeoffs for readability using gotos for efficiency often end up not being the win you thought they'd be. Specifically. Goto statements.
There are basics towards building successful scalable systems that people missing by taking Knuth's quote at face value.
- He's saying that anti-patterns of development are not worth the tradeoff for efficiency. Not that spending time considering efficiency is not inherently valuable.
- Performance as a requirement. It is one. This is the most common thing I've seen people overlook. How is it that systems backing major websites are designed and built without any discussion towards how many requests per second that system will serve right away, in a year, or in 3 years? How is it that a latency target is not considered? That's simply gross negligence that Knuth was not advocating.
- A system that doesn't scale early won't scale later. This is probably the most overlooked thing I've seen. If you design a system that is not fast when you are throwing one one-hundredth of the traffic at it, how do you expect it to behave when you scale it up? A system should be incredibly fast with no traffic, and if it isn't, you won't scale. I've seen this too defended with "premature optimization"
Most of the time I think the "premature optimization" quote is used to defend badly designed systems that happen to follow a pattern someone would like to use instead of taking performance as a requirement into consideration. Maybe they want to use node.js, and the initial performance shows it's terrible. This is more often what I see defended by "premature optimization": not following through on the basic requirements of a scalable system in order to preserve a developer's choice of language or architecture.