Borrowed Shapes
Borrowed Shapes
I am, by almost any honest measure, an analogy addict. I’m also addicted to other things but that’s a story for another day.
I wrote a whole ass essay not that long ago comparing language to household plumbing and human society to a choice between a pyramid and a web, and I stand by every word of it and also I fully intend to keep doing exactly that sort of thing, so please understand that everything below is coming from a person who reaches for a metaphor the way other (modern) people reach for their phone in an elevator or some other uncomfortably awkward social situation…which is to say reflexively and a little compulsively and almost always before pausing to ask whether he really needs to.
And yet.
The thing I keep tripping on, in philosophy and in systems work and frankly in the very ordinary act of trying to decide what to do today, is that hypotheticals and analogies are, underneath all their charm, sort of fundamentally disingenuous. I resent that I now have to define what I mean by that, because the moment you pin a word down you have already started smothering the loose useful fuzziness that made it worth saying, but here goes: an analogy is a borrowed shape, it is a structure you quietly lifted off of one thing and set down on top of a different thing while hoping the seams will line up, and the trouble is that the warm feeling of the seams lining up can arrive whether or not they have truly lined up, the “oh, now I get it” lands with a satisfying crunch while you conveniently ignore the much quieter and much more important question of whether you in fact got anything at all.
The distinction I cannot stop turning over in my hands and inspecting from every angle is this; an analogy is welcome to help you define the problem but is absolutely under no circumstances allowed to help you define the solution, those being two entirely different rooms, and the metaphor should only ever be allowed in the first room.
When I was first trying to get my arms around why these fancy “new” AI systems behave the way they do, the single most useful thing I came upon was the model of a LLM as something like a brain with savant-level focus and absolutely no long-term memory. I imagine a mind that reasons brilliantly inside a single moment and then forgets you exist the instant that moment closes, and that picture was enormously elucidating, it gave me a handle on the statelessness, it let me feel the shape of the problem in a way the raw mechanics never could have on their own. But the picture is a liar in the gentle but helpful way that all of them are, because an actual brain is busy doing a hundred things a model simply is not. Your brain learns while it lives, it wants things, it is wired into potato sack of meat with skin on top, and yet using the brain model to understand the problem is clarifying and fine and good.
The trouble seems to start only at the exact moment I allow that same model to start telling me what to build, because if I sat down and actually engineered the system as though it really were a brain, or supporting a brain, I would be engineering around fiction, I would be solving with great care for a thing that does not exist, and the work would drift, slowly and then less slowly, away from the reality it was supposed to be in service of. That’s a big part of the reason why HiiBo died, but I digress.
That drift is the entire thing I am pointing at here. What happens when you let the analogy slip out of the problem room and into the solution room? You end up manufacturing a connection that feels airtight and is in fact held up by nothing. You lay down a little synthetic bridge running from “here is the issue” straight over to “and therefore here is what we will do,” and the bridge looks plenty load-bearing precisely because the original comparison was so satisfying, but as soon as you step on it there is nothing underneath. Alas! There is no reasoning rooted in the actual constraints of the actual situation, only the afterglow of an image that was never built to carry weight and was only ever meant to point, but you’re now trying to use it to carry a 20-ton truck across a river. Oops.
The trolley problem is a rather horrifyingly wonderful way to make you intimately aware that you walk around carrying moral intuitions about tradeoffs, and it would imply a special type of madness to go write municipal traffic law on the basis of runaway trolleys, and yet some version of this is happening constantly, everywhere. We’ve got (so many) people taking the animated/fabricated/hypothetical scenario that did its honest job of surfacing a question, or raising a point, or clarifying a problem, and then treating the invented shape as though it will somehow match the overwhelming complexity of real life.
I have heard a sober little phrase for this type of hazard, the map is not the territory!!, which is the identical warning just buttoned down with some work boots on. Your model is a guide and your model is never once the ground itself, and it is incumbent upon you to keep that in mind. A simulation can show you the places things might plausibly snap but when the things do snap, as they inevitably will, they are going to snap according to their own real and stubborn and (frequently quite stupid) constraints rather than according to the tidy little world you so lovingly modeled. Let the model help you ask sharper questions. Just do not hand it the authority to answer them.
So the rule I am trying, fairly badly, to live by, on account of old habits being what they are, is to let the analogy come on in and shape the question and show me where to point the flashlight, and then to thank it sincerely for the help and walk it back out of the building before it can get its adorable little paws anywhere near the verdict. It can take the witness stand and it can point straight at the problem and then it really needs to take a hearty step back, because the metaphor does not get to hand down the ruling and the hypothetical does not get to enter itself into evidence, all either one of them was ever qualified to do is help you work out what on earth you are even looking at, which, in fairness, is not nothing, it is most of the fight, it is just very much not permitted to be all of it.
I will keep using them anyway, constantly, compulsively, probably inside the very next thing I sit down to write. I would just like to start trying to keep them where they belong.
About the Author
Sam Hilsman is the CEO of CloudFruit® and a perpetual student of the world. I build software, write things, and read a bunch.