The Red Button: AI, Human Judgement and the Danger of Optimising Everything

A mischievous beginning

As a child, I was very much like Dennis the Menace. I had something of a reputation for being naughty and mischievous, although I do not remember regarding myself as particularly naughty at the time. I was curious, energetic and forever getting into things. I was a boy. I wanted to know how things worked, what was around the next corner and, perhaps most dangerously, what would happen if I did something I had just been told not to do. I suspect many parents will recognise the type. If they were alive, I would hug them and apologise for what I put them through.

The photograph accompanying this article is me at about seven years of age. Looking at that expression now, more than sixty years later, I am not entirely convinced I was contemplating the finer points of philosophy. I suspect somebody had either told me off, prevented me from doing something interesting, or caught me shortly after doing it. Whatever had happened, I clearly did not approve. Or perhaps my mother, who was taking the photograph, simply asked me to smile. I can’t help but laugh.

I grew up with Dennis the Menace and Mr Magoo in cartoon strips and comic books, and I loved them. They still evoke fond memories of childhood (look them up on the ionernet). At the time they simply made me laugh, but looking back from my seventies I can see rather more in them. The best cartoon characters exaggerate characteristics that are recognisably human. Dennis represented that irrepressible curiosity and mischief that can get us into trouble despite our generally good intentions, while Mr Magoo represented another familiar human weakness: the remarkable ability to proceed with enormous confidence while failing to see half of what is going on around us. I especially enjoyed Mr. Magoo. Look them up to see why.

I can identify with both of them. I would like to think that more than seven decades of living have added a little wisdom and restraint to the boy in that photograph, but I am not entirely sure Dennis has ever left. Recently he reappeared, and this time there was a very large red button involved. My partner Alofa keeps telling me to grow up, but the more she says it, the less inclined I feel that I should.

"Perhaps growing up is not getting rid of Dennis the Menace. It is learning when Dennis should put his hands in his pockets."

AI is already changing how I work

There has been a great deal of publicity recently about the potential dangers of artificial intelligence, and some of these concerns deserve to be taken seriously. We hear about misinformation, loss of privacy, displacement of employment, people becoming intellectually lazy, machines being entrusted with decisions they should not make and, at the more dramatic end, the possibility that sufficiently advanced artificial intelligence might one day become a threat to humanity itself. Any technology powerful enough to change society for the better is almost certainly powerful enough to do considerable harm when badly designed, badly governed or badly used.

I do not dismiss these concerns, I shar them; but I think there is a danger that we become so preoccupied with what artificial intelligence might eventually do to us that we fail to appreciate what it is already enabling us to do. My own experience has made me increasingly optimistic about its possibilities. I am using AI extensively now, and the best description I can give is that it has become something resembling a librarian, research assistant and extraordinarily capable aide sitting alongside me. Depending on what I am working on, it can assist with medical and scientific questions, legal and administrative matters, writing, business, historical research and even the complicated business of human relationships. I would never confuse that assistance with the professional responsibility of a doctor or lawyer, nor with the wisdom of a trusted friend, but as an aid to thinking it can be remarkable.

At my age, this has particular value. I have accumulated more than fifty years of professional experience in health, rehabilitation and sport, along with family history, photographs, correspondence, articles, clinical observations, legal and Accident Compensation work, business records, research papers and goodness knows what else. No human brain can hold all of that in immediately accessible memory. Mine certainly cannot. I may remember that I wrote something important twenty years ago, or that a particular client taught me something I subsequently incorporated into my practice, but remembering exactly where the information is stored, what was said and how it connects with something I am considering today is another matter altogether.

This is where AI becomes genuinely transformative for me. It can help retrieve, compare, organise and connect information that would otherwise remain scattered across thousands of documents and decades of work. In effect, it allows me to offload some of the burden of remembering detail so that my biological brain can devote more attention to something human beings remain rather good at: looking at the larger picture, asking whether things make sense, drawing upon experience and deciding what deserves attention.

There is nothing particularly unnatural about extending ourselves in this way. As I get older, I stick to learning at least one new skill or idea each day. Human beings have been doing it for thousands of years. Writing extended memory beyond the brain. Libraries extended it further. Printing made accumulated knowledge available to millions. Telescopes extended our sight, bicycles and motor vehicles extended our movement, and calculators and computers extended our capacity to manipulate information. Artificial intelligence may prove to be another step in that long story, although potentially a very large one. The question, as I see it, is whether we use it to enlarge human capability or progressively allow it to replace human judgement. As we age, we must consider how to access the knowledge we’ve gathered over the decades and, more importantly, how to pass it on for future generations. This ability to share wisdom is what makes humanity distinct from every other known creature.

"The question is whether we use AI to enlarge human capability or progressively allow it to replace human judgement."

The temptation of the red button

Over more years than I care to admit, I have accumulated an extraordinary amount of digital material. My computers and cloud storage contain articles, photographs, videos, clinical records, correspondence, book drafts, research papers and multiple copies of things that have somehow migrated from one computer to another and then into OneDrive, Google Drive and goodness knows where else. Some of this material represents decades of professional work and family history. Some is undoubtedly rubbish. The difficulty is telling one from the other without accidentally throwing away something that cannot be replaced.

With the assistance of AI and Python, I have therefore been working systematically through this digital jungle, identifying duplicates and reclaiming storage while building in several layers of protection against human error. Then we came to Google Drive. It was full, completely full, with about 1.2 terabytes of material, including some enormous video files running into tens of gigabytes each. Meanwhile, sitting there invitingly was the Bin, containing material that could potentially be permanently deleted.

And there was the button. I could almost hear it whispering, "Go on, Gary. You know you want to."

I asked AI whether I should empty the Bin. The response was emphatic: leave it exactly as it is and do not empty it. The Bin was serving as another safety net while we established that the material we wanted to preserve existed elsewhere and had been independently verified.

Quite sensible advice. Naturally, I replied that hitting the red button was becoming terribly tempting.

The response from my artificial supervisor was rather wonderful:

"The big red button is forbidden fruit."

There he was again. Dennis the Menace, aged seven, barefoot in the backyard, still lurking somewhere inside the man in his seventies sitting at the computer. Perhaps growing up is not really about getting rid of Dennis the Menace. It is about accumulating enough experience to know when Dennis should put his hands in his pockets.

Human civilisation survives another night

I resisted temptation, left the Google Drive Bin untouched and went to bed. When I returned to the computer the following morning, AI greeted me with a line that deserves preserving:

"Human civilisation survived another night, not because the machine behaved itself, but because the human resisted pressing the big red button."

There is something deliciously backwards about this. For most of my life, science fiction has conditioned us to worry about the day when machines stop obeying humans. Yet here I was being supervised by a machine because apparently the human could not entirely be trusted near the controls.

Beneath the humour, however, was something that helped clarify my thinking about artificial intelligence. In this instance the machine was not taking control away from me. It was helping me exercise better control over myself. It had kept track of what we had verified, what remained uncertain and why prematurely emptying the Bin could remove an important safety net. My human brain was looking at 1.2 terabytes of full storage and thinking how satisfying it would be to make a great chunk of it disappear. The machine was effectively saying, "Yes, Gary, but not yet."

That seems to me a much more interesting model for the future relationship between human beings and artificial intelligence than either blind enthusiasm or fear. AI should extend human capability while leaving human responsibility intact. There will be times when intelligence consists of knowing what to do, but there will also be occasions when wisdom consists of knowing what not to do.

"AI should extend human capability while leaving human responsibility intact."

The machine did exactly what we asked

Suppose, instead of carefully defining our objective, I had simply instructed a sufficiently capable computer system to free as much storage space as possible. There is a beautifully efficient solution to that problem: delete everything. If success were measured solely in gigabytes recovered, the machine could produce an outstanding result and probably deserve a performance bonus.

The problem would not be the intelligence of the machine. The problem would be the stupidity of the instruction.

This distinction is important because popular discussion about artificial intelligence often concentrates on the prospect of machines becoming conscious, resenting their creators and deciding that humanity has become surplus to requirements. Perhaps sufficiently advanced artificial intelligence will eventually present risks that we cannot presently foresee. I do not pretend to know. But we do not require hatred, resentment, consciousness or an extermination instinct to create a monumental mess. We merely need a powerful system, a poorly specified objective and humans foolish enough to give it sufficient authority.

"The problem would not be the intelligence of the machine. The problem would be the stupidity of the instruction."

Which brings me to my other childhood friend, Mr Magoo.

Enter Mr Magoo

Mr Magoo was an elderly gentleman who was extraordinarily short-sighted but proceeded through life with magnificent confidence, repeatedly misunderstanding what was happening around him. As children, we laughed because we could see what Magoo could not. He might believe he was walking through a perfectly ordinary doorway while the audience could see that something altogether different was happening. Disaster appeared imminent, yet Magoo would blunder onwards, blissfully unaware, usually emerging unharmed and none the wiser.

There is something rather profound about Mr Magoo when viewed through older eyes. He represents one of the great foibles of the human condition: our ability to proceed with enormous confidence while seeing only part of the picture. There is something decidedly Magoo-like about badly designed optimisation because we can become so focused upon the variable directly in front of us that we fail to notice the consequences developing elsewhere.

Imagine the report: "Excellent news, Gary. The computer has freed 1.2 terabytes."

Wonderful.

"Where are my photographs?"

What photographs?

Dennis and Magoo therefore make rather good companions for a discussion about artificial intelligence. Dennis presses the button because it is there. Magoo presses it because he cannot see what is on the other side. The machine then follows the instruction perfectly. None of them requires malicious intent for the result to be disastrous.

"Dennis presses the button because it is there. Magoo presses it because he cannot see what is on the other side."

Measurement is not meaning

Our computer clean-up has deliberately been slow and cumbersome because the objective is not simply to remove files. We compare them and, where appropriate, use SHA-256 hashing, which provides a digital fingerprint of a file. If two files have exactly the same hash, we have extremely strong evidence that their contents are identical. Yet even then another question remains: which copy should we keep?

One may sit within a carefully organised archive while another is buried in an old downloads folder. One may belong alongside photographs, correspondence and other records that give it context. The computer can establish that two files are identical, but it cannot necessarily establish which copy belongs in the story of my life. Measurement can establish facts; judgement establishes meaning.

I see versions of this problem in health all the time. We measure cholesterol, blood pressure, bone density, body weight, blood glucose, hormones, minerals and countless other variables. These measurements can be extremely useful, but there is always a danger that improving the number becomes the objective rather than improving the person. The same caution sits behind my approach to Hair Tissue Mineral Analysis. The laboratory chart provides clues, but the person provides the context. Numbers and patterns have to be considered alongside symptoms, health history, diet, medication, lifestyle, physical capability and what is actually happening in somebody's life.

A measurement is information. It is not the person. If we optimise the number while losing sight of the human being, Mr Magoo has wandered into the consulting room.

:"Measurement can establish facts; judgement establishes meaning."

When the target replaces the purpose

Human organisations have always struggled with this problem. Governments establish targets, businesses establish performance indicators, health systems measure waiting lists and procedures, schools measure academic results, councils measure transport movements and social media companies measure engagement. We need measurements because without them we are often guessing, but trouble begins when the measurement quietly replaces the original purpose.

A hospital exists to help sick and injured people, not merely to produce favourable statistics. A school exists to educate children, not merely to improve examination numbers. A transport system exists to move people and goods efficiently, safely and affordably, not simply to optimise one measurable component of the system. Once a narrow variable becomes the target, enormous intelligence can be directed towards improving that variable while the wider system deteriorates.

Artificial intelligence magnifies this problem because its capacity to search, calculate, compare and optimise may eventually exceed anything human beings can do unaided. That could be extraordinarily useful, but it also means we need to become much better at deciding what we actually want. The more capable the machine becomes at answering our questions, the more responsibility falls upon us to ask good ones.

AI and the FreeRanger future

There might appear to be a contradiction between artificial intelligence and the FreeRangers philosophy, but I do not see one. FreeRangers is not about turning the clock backwards, rejecting science or pretending modern technology does not exist. Nor is it nostalgia for some imagined golden age. The question I prefer to ask is whether the things we create help produce stronger families, healthier people, resilient children, capable communities and a country in which people can live well and raise a family.

Technology should be judged against much the same standard. Does it enlarge human capability or diminish it? Does it give people greater independence or make them increasingly dependent upon systems they neither understand nor control? Does it relieve us of repetitive drudgery so that we can devote more of ourselves to work requiring judgement, creativity, relationships and purpose, or does it gradually remove the need for us to think and act for ourselves?

Used wisely, I can see enormous possibilities. An older person could use AI to organise a lifetime of records, photographs and memories rather than allowing much of that history to disappear. A small business could gain analytical capabilities that were once available only to large corporations. A farmer could draw upon sophisticated information while retaining intimate knowledge of his own land. Parents could investigate questions affecting their family without having to become experts in every subject. A health practitioner could retrieve decades of accumulated observations while still looking at the person sitting across the desk rather than merely the computer screen.

The machine does not have to replace human wisdom. It may help us make better use of the wisdom we already possess.

"The machine does not have to replace human wisdom. It may help us make better use of the wisdom we already possess."

Capability still comes from doing difficult things

There is an important qualification. Resilience develops through appropriate challenge. Muscles strengthen through stress, bones strengthen through loading, minds develop through challenge and character through responsibility. Remove every difficulty from a person's life and you do not necessarily produce a stronger person. You may produce the opposite.

The same principle applies intellectually. If AI writes every letter, solves every problem, remembers every fact, evaluates every argument and eventually makes every important decision, we may create extraordinarily capable machines while gradually producing less capable human beings. That concerns me more than whether a computer someday becomes conscious.

The outcome is not predetermined. Artificial intelligence can become a substitute for thinking, or it can become a tool that enables better thinking. In my own work, its extraordinary capacity for detail is valuable precisely because it allows me to devote more attention to the larger questions. I do not have to remember where every document sits or carry every number in my head. I can spend more of my limited mental energy asking what the information means, how the pieces fit together and what we should actually do.

Dennis still needs the freedom to investigate the world, but he also needs to learn something from the mess he made last time. Every one of us, however experienced we become, also needs enough humility to wonder occasionally whether there might be a little Mr Magoo looking back from the mirror.

Test, don't guess

One of the principles I have carried through decades of clinical work is simple: test, don't guess. Information should guide questions rather than become an excuse to stop asking them. If the evidence in front of me does not fit the person sitting across the desk, I need to reconsider my interpretation rather than trying to make the person fit my theory.

That strikes me as an excellent principle for artificial intelligence as well. AI can search enormous quantities of information, identify patterns, compare possibilities and perform calculations beyond ordinary human capability, and these abilities will undoubtedly improve. But context, experience, common sense and humility still count. The more powerful the tool becomes, the more important the quality of the questions becomes.

Perhaps the most useful question is one Mr Magoo might have benefited from asking occasionally: Are we actually looking at the whole picture?

Keeping a human hand on the red button

I am optimistic about artificial intelligence precisely because I am using it rather than merely reading predictions about it. It is already helping me organise decades of accumulated knowledge, recover information I would otherwise struggle to find, test arguments, examine evidence, work through complicated administrative and legal problems, and preserve material that might otherwise disappear when I do.

In a peculiar way, the machine's extraordinary capacity for detail allows me to become more human, not less. I do not have to compete with it at remembering thousands of filenames or comparing millions of pieces of data. I would lose that competition anyway. I can instead concentrate on experience, relationships, judgement, meaning and the larger picture.

There is, however, a condition attached to this arrangement. I remain responsible. AI can advise me not to press the button, but it cannot relieve me of responsibility if I decide to press it anyway. That distinction should remain at the centre of how we develop and use these systems.

The FreeRanger future I want is therefore not one without technology. It is one in which extraordinary technology helps create stronger, healthier, freer and more capable people, strong families and communities, and ultimately makes New Zealand an even better place to live and raise a family. Let the machine remember the details, search the million files, calculate the hashes, compare the documents and remind me of something I wrote twenty years ago. Let it challenge my assumptions and, when necessary, tell Dennis to put his hands back in his pockets. Let Mr Magoo remind both of us that something important may still be sitting outside our field of vision.

But keep a human being at the keyboard who is prepared to ask, "Hang on. Are we sure?"

"Human civilisation survived another night, not because the machine behaved itself, but because the human resisted pressing the big red button."

Perhaps that is not a bad model for the relationship between us and the extraordinary technology now developing around us. Let AI become extraordinarily capable, by all means. Our task is to become wise enough to know when to listen to it, when to question it and, every now and again, when to keep our bloody hands off the red button.