The problem with experts is they're too smart. The problem with the rest of us is we don't have the expertise to reign in the experts.

Consider the following conversation:

Mortal Developer: This system seems complicated. Can we do without the WhackAMole feature?
Expert: No, some clients might need WhackAMole to interact with their legacy abacus.

Now our poor mortal developer is stumped. After all, this is the expert we're talking to here -- and if he says we need WhackAMole you better get whacking.

What our developer missed is this: complexity alone is enough to reject a design. After all, you're a smart engineer with years of programming experience. Even if you're not an expert, if a system is hard for you to comprehend, it's probably too complicated. Sadly, there seems to be an "Emperor's New Clothes" phenomenon in software -- no one wants to admit something is hard to understand.

This shouldn't be surprising -- objecting to complexity might be viewed as looking stupid in an industry where intelligence is the highest virtue. Unfortunately this can do a lot of harm. Imagine if the J2EE specification had involved a developer without expertise, but with veto power. Surely the result would have been a lot simpler. This lesson may have been learned for newer versions, but once added this complexity cannot be completely removed.

For anyone that might be struggling with this right now, some humble advice:
  • Follow Alan Kay's axiom: "Simple things should be simple, complex things should be possible." -- If something complicated must be in the system, it still should not affect the simple things.
  • View the project as a constant battle against complexity -- A single complex module may seem unimportant, but it quickly compounds.
  • Understand a project end-to-end -- the project should be easily broken down into problems that are known to be solvable. (Avoid Southpark's 1. Collect Underpants. 2. ??? 3. Profit!!!).
  • A strong-willed engineer or executive can will a group down the wrong path -- it's an engineer's duty to object and prevent spiraling complexity
  • The clever hack should be a last resort -- a hack that is hard to come up with is much harder to debug or support
All of this boils down to Keep It Simple, Stupid. It turns out keeping things simple is pretty hard.

Update: This isn't to suggest that all experts fall into this trap, but there is certainly evidence suggesting that some do.