Sometimes the problem is to discover what the problem is (part 1)

 "Sometimes the problem is to discover what the problem is." (Gordon Glegg, The Design of Design, 1969)

As software engineers, we often frame problems poorly. Sometimes we frame problems in a way that is inflexible, overly-literal, convenient, laden with implicit assumptions or just plain wrong. The financial impact of this can be enormous: undertaking the "critical" re-platforming project that never finishes, grinds the business to a halt and causes a company to lose faith in its engineering organization; building features that are well-intended but barely used or even worse, ripped out, because they were poorly conceived; reduced time-to-market and revenue for those capabilities that are needed; and finally, we drastically reduce our potential effectiveness as problem solvers and most of the time we don't even realize it. Like high blood pressure, poor problem-framing can be the silent killer when developing software: it can destroy value delivery and we often don't even know it's a problem. Rich Hickey makes the point succinctly: "without a doubt, most of the big problems we have with software are problems of misconception".

Generally the later it is in the development lifecycle the more expensive it is to fix something. For example, fixing a bug in production is more expensive than fixing it at design-time. Makes sense. But it might surprise you how much more expensive it is. Is it twice as large? The late Barry Boehm researched this topic and here's what he found:

Finding and fixing a software problem after delivery is often 5 to 100 times more expensive than finding and fixing it during the requirements and design phase. This is based on two publications from 1987 and 2001. But should you agilistas draw your swords, agile development does not fix this problem. Incremental delivery can lessen its impact but blind worship at the YAGNI altar and de-emphasis of the importance of design (whether intended or not) can worsen it too. Incremental delivery helps (so today's number is likely < 100) but that does not negate the overall premise here: the later we fix a problem the more expensive it is to fix.

Do you see the point? The point of the lifecycle where we have the most economic leverage is when we are framing the problem -- and that is the point where we often make some of our biggest mistakes! The impact of getting better at problem-framing can be enormous.

But what drives our tendency to frame problems poorly? What should we do differently? Great questions. Stay tuned for the next part of this multi-part series.