The myth of idiot-proofing

Idiot-proofing is a myth. Attempting to simplify an over-complex task is good, but be careful how you do it - beyond a certain point you aren't idiot-proofing, just idiot-enabling.

Three classes of tool

Tools (like programming languages, IDEs, applications and even physical tools) can be grouped into three loose categories: idiot-prohibiting, idiot-proof and the merely idiot-enabling.

Idiot-prohibiting tools are those which are effectively impossible to do anything useful with unless you've at least taken some steps towards learning the subject - tools like assembly language, or C, or Emacs. Jumping straight into assembly without any idea what you're doing and typing code to see what happens will never, ever generate you a useful program.

Perhaps "prohibiting" is too strong a word - rather than prohibiting idiots these tools may only discourage naive users. These idiot-discouraging tools are those with which it's possible to get some results, but which leave you in no doubt as to your level of ability - tools like perl, or the W3C XHTML validator. Sure you might be able to open a blank .html or .pl file and write a few lines of likely-looking pseudocode, but if you want to have your HTML validate (or your Perl code actually do anything useful and expected), you're soon going to be confronted with a screen-full of errors, and you're going to have to stop, study and get to grips with the details of the system. You're going to have to spend time learning how to use the tool, and along the way you'll practice the skills you need to get good at what you're doing.

The myth of idiot-proofing

Garbage in, garbage out is an axiom of computing. It's generally impossible to design a system such that it's completely impossible for someone sufficiently incompetent to screw it up - as the old saw goes: "make something idiot-proof and they'll invent a better idiot".

A natural fall-out consequence of this is that making anything completely idiot-proof is effectively impossible - any tool will always be somewhere on the "prohibiting->discouraging->enabling" scale.

Furthermore, even if your tool merely makes it harder for "idiots" to screw things up, at the same time that very feature will attract more idiots to use it.

Shaper's Law of Idiot-Proofing:

Lowering the bar to prevent people falling over it increases the average ineptitude of the people trying to cross it.

Or, more simply:

Making a task easier means (on average) the people performing it will be less competent to undertake it.

Obviously I don't have hard statistics to back this up (though there could be an interesting project for a Psychology graduate...), but it often seems the proportion of people failing at the task will stay roughly constant - all you're really doing is increasing the absolute number of people who just scrape over the bar... and who then go on to produce nasty, inefficient, buggy - or just plain broken - solutions.

Fixing the wrong problem

The trouble with trying to make some things "idiot-proof" is that you're solving the wrong problem.

For example, when people are first learning programming they typically spend a lot of their time concentrating on the syntax of the language - when you need to use parentheses, remembering when and when not to end lines with semi-colons, etc. These are all problems for the non-programmer, but they aren't the important ones.

The major difficulty in most programming tasks is understanding the problem in enough detail to solve it^[1]. Simplifying using the tools doesn't help you understand the problem better - all it does is allow people who would ordinarily never have attempted to solve a problem to try it.

Someone who's already a skilled developer won't benefit much from the simplicity, but many unskilled or ignorant people will be tempted by the extra simplicity to try and do things that are (realistically) completely beyond their abilities. By simplifying the wrong thing you permit more people to "succeed", but you don't increase the quality of the average solution (if anything, you drastically decrease it).

By analogy, spell-checkers improved the look of finished prose, but they didn't make anyone a better author. All they did was make it easier to look professional, and harder to tell the difference between someone who knew what they were talking about and a kook.

Postel's Law

The main difficulty of programming is the fact that - by default - most people simply don't think in enough detail to successfully solve a programming problem.

Humans are intelligent, flexible and empathic, and we share many assumptions with each other. We operate on a version of Postel's Law:

"Be conservative in what you emit, liberal in what you accept"

The trouble begins because precision requires effort, and Postel's Law means we're likely to be understood even if we don't try hard to express ourselves precisely. Most people communicate primarily with other humans, so to save time and effort we express ourselves in broad generalities - we don't need to specify every idea to ten decimal places, because the listener shares many of our assumptions and understands the "obvious" caveats. Moreover, such excessive precision is often boring to the listener - when I say "open the window" I don't have to specify "with your hand, using the handle, without damaging it and in such a way that we can close it again later" because the detail is assumed.

Sapir-Whorf - not a character on Star Trek

The problem arises that because we communicate in generalities, we also tend to think in generalities - precision requires effort and unnecessary effort is unpleasant, so we habitually tend to think in the most vague way we can get away with. When I ask you to close the window I'm not imagining you sliding your chair back, getting up, navigating your way between the bean-bag and the coffee table, walking over to the window, reaching out with your hand and closing the window - my mental process goes more like "window open -> ask you -> window closed".

To be clear: there's nothing inherently wrong with thinking or communicating in generalities - it simplifies and speeds our thought processes. Problems only arise when you try to communicate with something which doesn't have that shared library of context and assumptions - something like a computer. Suddenly, when that safety-net of shared experience is removed - and our communication is parsed exclusively on its own merits - we find that a lifetime of dealing in vague generalities has diminished our ability to deal with specifics.

For example, consider probably the simplest programming-style problem you'll ever encounter:

You want to make a fence twenty metres long. You have ten wooden boards, each two metres long, and you're going to the hardware shop - how many fence-posts do you need?

No programmer worth his salt should get this wrong, but most normal people will have to stop and think carefully before answering^[2]. In fact this type of error is so common to human thinking that it even has its own name - the fencepost (or off-by-one) error.

Solve the right problem

Any task involves overcoming obstacles.

Obstacles which are incidental to the task ("publishing my writing">"learning HTML") are safe to ameliorate. These obstacles are mere by-products of immature or inadequate technology, unrelated to the actual task. You can remove these without affecting the nature of the task at hand.

Obstacles which are an intrinsic part of the task are risky or even counter-productive to remove. Removing these obstacles doesn't make the task any less difficult, but it removes the experience of difficulty.

Counter-intuitively, these obstacles can actually be a good thing - when you don't know enough to judge directly, the difficulties you experience in solving a problem serve as a useful first-order approximation of your ability at the task^[3].

Despite years of trying, making programming languages look more like English hasn't helped people become better programmers^[4].

This is because learning the syntax of a programming language isn't an important part of learning to program. Issues like task-decomposition, system architecture and correct control flow are massively more important (and difficult) than remembering "if x then y" is expressed as "if(x) { y; }".

Making the syntax more familiar makes it easier to remember and reduces compile-time errors - making the task seem easier to a naive user - but it does nothing to tackle the real difficulties of programming - inadequately-understood problems or imprecisely-specified solutions.

The trouble is that the "worth" of a program is in the power and flexibility of its design, the functionality it offers and (inversely) the number of bugs it has, not in how many compiler errors it generates the first time it's compiled.

However, to a naive beginner beauty and solidity of design are effectively invisible, whereas compiler-errors are obvious and easy to count. To a beginner a program that's poorly designed but compiles first time will seem "better" than a beautifully-designed program with a couple of trivial syntax errors.

Thus to the beginner a language with a familiar syntax appears to make the entire tasks easier - not because it is easier, but because they can't accurately assess the true difficulty of the task. Moreover, by simplifying the syntax we've also taken away the one indicator of difficulty they will understand.

If a user's experiencing frustration because their fire-alarm keeps going off the solution is for the user to learn to put out the fire, not for the manufacturer to make quieter fire alarms.

This false appearance of simplicity begets over-confidence, directly working against the scepticism with the current solution which is an essential part of the improvement process^[5].

Giving a short person a stepladder is a good thing. Giving a toddler powertools isn't.

---

Footnotes

[1] This is why talented programmers will so often use exploratory programming (or, more formally, RAD) in preference to designing the entire system first on paper - because although you might have some idea how to tackle a problem, you often don't really understand it fully until you've already tried to solve it. This is also why many developers prefer to work in dynamic scripting languages like Perl or Python rather than more static languages like C or Java - scripting languages are inherently more flexible, allowing you to change how a piece of code works more easily. This means your code can evolve and mutate as you begin to understand more about the problem, instead of limiting your options and locking you into what you now know is the wrong (or at least sub-optimal) approach.

[2] Obviously, the answer's not "ten".

[3] When I'm learning a new language I know I'm not very good at it, because I have to keep stopping to look up language syntax or the meanings of various idioms. As I improve I know I'm improving, because I spend less time wrestling with the syntax and more time wrestling with the design and task-decomposition. Eventually I don't even notice the syntax any more - all I see is blonde, brunette, readhead... ;-)

[4] Regardless of your feelings about the languages themselves, it's a truism that for years many of the most skilled hackers have preferred to code in languages like Lisp or Perl (or these days, Python or Ruby), which look little or nothing like English. Conversely, it's a rare developer who would disagree that some of the worst code they've seen was written in VB, BASIC or PHP. Hmm.

[5] From Being Popular by Paul Graham, part 10 - Redesign:

To write good software you must simultaneously keep two opposing ideas in your head. You need the young hacker's naive faith in his abilities, and at the same time the veteran's scepticism... The trick is to realize that there's no real contradiction here... You have to be optimistic about the possibility of solving the problem, but sceptical about the value of whatever solution you've got so far. People who do good work often think that whatever they're working on is no good. Others see what they've done and are full of wonder, but the creator is full of worry. This pattern is no coincidence: it is the worry that made the work good.