How Can You Ever Be Sure?

How can you ever be sure. I have OCD. I ask this every day. The answer, clinically, is no, you can never be sure. And the amount of rampant, untreated mental illness out in the world is mind boggling. So it's not surprising to see this "urgent" question with an impossible answer being asked every day in every discipline.

Ultimate truth is beyond human comprehension. We leave it to god(s) or the cosmos. It's easier to build a hammer, than it is to know you have built the ultimate hammer.

My favorite villain is the Terrible Trivium from The Phantom Tollbooth. He asks the main character to pick away at a mountain with a needle until it is turned into fine dust. The moral is that there are so many things in life that demand your attention but produce nothing.

At engineering meetups and conferences there's XKCD coined "engineer sniping." The goal is to distract an engineer from whatever they were in the middle of by posing an interesting, but extremely deep question. It's cruel. I don't do it. But, I was at an event, listening to someone collecting their thoughts and then right in the middle of that another participant lobbed an errant quandary at them to which they dropped everything to ponder "Oh, actually, hmm... well if you think about..." Then the interloper pumped their fist, looked at me, and said "Sniped!" I hope I've evoked the state of mind that encircles "true" verification. There are too many scientific proofs and theorems to count around perception, verification, systems of measure, but I will list my favorites.

Goedels incompleteness theorem, any system able to describe itself will have a contradiction of some kind.

Entropy, the constant decay of any perfect system.

The double slit experiment and shrodingers uncertainty principle, that measuring something changes the outcome.

The blood vessel in the human eye that obscures a small dot in our vision, which we then hallucinate whatever should fill it in.

Ultraviolet light, named ultra because it's not on the visible spectrum, however there are animals in nature that can see it. The list goes on and on. But it is, in my opinion truly miraculous how infinitely we are hamstrung by our ability to understand.

However ships are launched on verity, campaigns are won on confidence, families sewed, fortunes made, and yet none of it was ever real. It was all confident hokum awash in the tides of reality. Scientifically, we still try. Formulae, papers, and proofs. Systems of equations, textbooks, lecture series, institutions, formal verification, significant figures, on and on.

An introduction more necessary than the following text. In computer science, we look at verity through many lenses. But there is no consensus on how to determine it. We have instead design principles. OOP, TDD, Agile, Scrum, diagrams, flowcharts, unit tests. These are meant to make us think about the many things that could happen with a piece of software so we can predictively iron out any kinks before release. And even then, a standard method is to just take the product, roll it out to a small set of users, and see what they do, as a way of admitting our own human limitation.

Cosmic rays flipping bits was not in someone's bug log to start I'm sure.

One way to find someone's password is to get access to their laptop microphone and then measure the percussion of their typing based on their laptop model and then assume the password based on that.

The amount of cases to consider with a piece of software is as well infinite. Many cases are trivial. An adding function sum (1,2) = 3 is trivial for any integer i such that sum(i,i) = i (i representing any three integers different or the same). But there is sum("meow", "woof"), sum (1.00000000000000001, 1), sum(pi, e), sum(,). And this is just what I can think of (adding letters, numbers bigger than the space they represent, infinitely long numbers, adding nothing).

The best example here is pi. We use it every day even though it's infinite. Collectively we decided on significant figures. Not for nothing. In astrophysics, calculating distance between stars with pi, if you use pi = 3.1 to calculate something in light years, you may be off by a lightyear. You need more significant figures to get a more accurate calculation, but the difference between 200 significant figures and 201, might be less than the distance of a human hair. The answer to pi's significant figures is "it depends," but it depends to a measurable amount and we can make that choice in a non-arbitrary way each time we use it.

If you're measuring circles in school, 3.14 is probably fine. For GPS satellite positioning, you might want 5-10 significant digits. For a precision woodworking project, maybe 4 rounded to the nearest millimeter. Pi in Python's math.pi library is 3.141592653589793, which is hopefully enough for most astrophysics. If it is not, there are ways to do math with longer floats.

The here is that Pi "for everyone" means you either make an executive decision and leave some folks out or you calculate pi forever and ever and never actually use it in a real function. What we've opted for instead in most industries is an interesting modular build where we cook in quite a lot of executive decision making (byte size, data types, edge cases) and allow people to build on top of it. I would say that allowance is a human architecture choice, but it is not. What actually happens, quite often, is that people try to set universal standards for things, deliberately thinking through every use case, and quite often someone else just didn't know about that solution and did it differently for their needs, birthing a competing standard, or the "comprehensive" solution wasn't comprehensive enough and a new standard was built to accomodate (firewire, DIN, midi, ethernet, XLR, on and on.)

The world itself proscribes no limits on creation. You can either make it or you can't. You either had the idea or you didn't. The argument in engineering rooms about the best universal standard is irrelevant to the moment of creation. It is just made. And now it exists.

There's more than one XKCD about it. We're up to USB-C (about 10 different options https://en.wikipedia.org/wiki/File:USB_A,_B,_C_(1.1-3.0)_connectors.svg and 3 generations of data protocols).

There's a growing conversation around this with regards to repeatable code because everyone wants to prompt their AI to infinitely generate SAAS server farms that generate cash. But you can't tell AI to build something that neither you nor the AI can describe. So you have to describe it, which is a facsimile of what you actually want. Ironically we have a saying for that in the software world which is "The best description of a system is working code," meaning that the functioning system itself is the best example. The functional requirements, the AI captured meeting notes, the whiteboard drawings, they're all a facsimile, a synecdoche of the working product. And the working product is in active decay and competition constantly.

So what are you to define? Where do you stop. What is your significant figure for description. How many test cases do you need?

This also we can define, and I'm surprised, as a discipline notorious for curious mental habits and obsessions, have not done yet. We already have the descriptive tools as well. OOP principles. Readable code. Solid design. OWASP standards. Sprints. Uptime. Customer conversion. Revenue. Cost.

In a way, asking software testers or AI to "test against human input" is like infinitely calculating pi. Instead you can say use OWASP standard principles for react form input, which should provide a comprehensive testing landscape, one where you could easily write a single test package for that other developers could import. Standardizing scope is one we haven't really done as an industry. We just think in millions of steps, billions, or less. Part of this is due to real world limitation, like Moore's Law prohibiting our ability to really be sure what the upper limit of computer power or cost will be next year. Another part of it is, I think, that we just get into habits as an industry. Everyone is doing X so it should be fine. And it probably is to a degree. But we could probably build things a lot faster if we thought a little more about how accurate we actually needed to be.