Chat GPT make me a sandwich! (SToAS)

"I'm sorry. I can't make a sandwich for you because I am a computer, but here are 5 highly rated sandwiches."

AI gets things wrong all the time. But, it's supposed to be as smart as 100 Einsteins.

Why?

Because of Sawyer's Theory Of Abstracting Specificity (SToAS).

Walk into 6th grade English. Your teacher says "We're making sandwiches today! But! You must write each step, and then I will follow your instructions!"

You write "Get the bread and jelly and make a PB&J." And your teacher says "I don't know where those are but if I did, I'd have made a jelly sandwich with no peanut butter and does the bread go on top of or underneath the jelly. Come to think of it, what is a sandwich?!"

Then the whole class goes "Ugghhhhhhh! Whyyyyy?"

That's SToAS.

We assume A LOT about making a sandwich. When you make one, you're assuming a lot of little details about what the ingredients are, where to find them, how to use them, how to clean up.

If you've ever made a sandwich, you're probably doing a lot of little details in different orders each time. For example today we are out of grape jelly in the fridge, but I did leave some strawberry on the counter, over there, right there, I am pointing at it.

All the knives are dirty but this is a spreadable sandwich so you can use a spoon, a fork, or your fingers if no one is looking.

These little details and variations get lost when you type into Chat GPT "Make me a sandwich."

Most LLM's just guess a reasonable response based on all of the other responses it's read.

Here's what I got from an LLM:

The "I Don't Want to Think About Lunch" Sandwich

Toast two slices of bread. Add:

• mayo + Dijon
• turkey or ham
• cheddar
• lettuce
• tomato
• pickles
• black pepper

Stack it, cut diagonally—diagonal is structurally important—and eat with chips.

If you tell me what's currently in your fridge, I'll make you a sandwich using only what you actually have.

So there's a fun part about that which is that it's asking me a question. We'll get back to how to improve AI conversations and go up the chain in abstraction. Right now, we're going to go down the chain, closer to the bare metal of the computer.

Read the last sentence of the LLM response. I'm currently at a cafe. I have to buy whatever they have. There is no fridge. That is lost through abstraction.

If you don't know what to put into a machine, then you don't really know what you're going to get out of one.

AI is complex in terms of input-output. Lets look at a simple parking meter. Put in coins, get parking time. Swipe your card, get a ticket. A parking meter is self-explanatory. But, what goes on inside of that thing... no one knows. We just park and pay and leave. And then maybe come back an hour too late to a ticket. Shame on us.

Metaphorically that's what we do as computer scientists. There's the idea we have. Then we express it in code. And the code may talk to a virtual machine or a compiler, and then it generates optimized machine code. And that machine code then tells the motherboard/memory/CPU/GPU/peripherals how to handle all of that stuff. And finally those devices operate based on the physical signals they get from each other. And finally that power comes from the power source on your computer which transforms the electrical signal from whatever comes out of your wall to whatever your computer needs (please tell me you have a surge protector). And finally that energy is converted from whatever the electrical grid needs to whatever your fusebox takes. And then that energy is converted from whatever the regional power line takes to whatever your local substation takes. And theres a power station somewhere in there. And the people that work in the power station are currently on lunch eating a sandwich.

Anyways... somethings coming up on the screen... success! Your code ran and passed all tests. You successfully forked the Nancy Pelosi ETF tracker. Pretty cool!

I'll actually give you a real theory you can throw around work. But not yet! You must stay for the comedy.

We are always dealing with levels of abstraction. And that abstraction is functionally infinite. The finite control of each domain is often labored over by industries full of people obsessed with the improvement of that domain. It's pretty difficult to then scoop that expertise out of your head and stick it in someone else's.

We often have to find some sort of way to pass off both a working system, and a simple way of interacting with that system to a non expert.

A USA power engineer obsesses over delivering 240v AC power to as many households as they can. Pretty difficult.

If done right you can... hold on... it's like, right under my, um, uh, here! Under my feet, sorry about that...

If done right you can plug anything that is FCC approved into the wall and turn it on. The device engineer has to worry about turning the power into a working computer. The OS engineer deals with making sure there's stuff we can put on the computer. And then I can write whatever code I want now.

See? I just made pong! Specifically I wrote: import pong; main Pong (run pong); Pretty easy.

Oh crap I cloned the wrong github repo! It was called pongDoom, and it's actually running Doom inside of an Atari emulator with pong loaded into the cartridge.

What I've laid down is a ton of empirical evidence (ahem, personal anecdotes, ahem) about the level of abstraction.

You often have little choice about what the current state of the system is. They are behemoths. Creating your own is almost impossible. Linux has taken many lifetimes and that's just one OS.

You often have to slot in somewhere and work within that system of abstraction. Hopefully it's well engineered enough that you can do a lot in that space.

What I'd like to do next is look at how we work within systems that are heavily abstracted.

At the very end I'll lay out, very simply, what SToAS is.

When I asked chatGPT to make me a sandwich it did in fact... tell me what ingredients could go on a fictional sandwich that I cannot make because I'm actually in a cafe.

And then it asked me a question! And that is a great thing to do. We will stray away from AI metaphorically and move to people (what AI is based off of.)

People do quite a lot!

They've grown up from babies. Babies, when they knock stuff off of their chair are actually exploring the world around them. Specifically what happens to a big bowl of applesauce when you push on it. Oh look at that. It goes on the floor. Splat! It's everywhere. Funny.

That's experience.

So... back to the cafe and the LLM and the sandwich it wants me to make. I look around and say "Hey we're actually in a cafe,"

I make some assumptions from experience. "They won't let you make a sandwich here."

And then it asks me a question. "Did you want to actually make a sandwich?"

You could say "Haha, you got me! I am just hungry. What sandwiches are good here?"

What I said was "I am observing your behavior for a blog article I'm writing about sandwiches."

On higher levels of abstraction, we can observe the system we're working in, look at prior knowledge we have about it, and then make whatever assumptions we can from that. There's a deeper scientific inquiry you can make to learn about the system and work with others. But that's the pretty basic toolset we have as humans and you'll see it every day.

Biologically, nature likes to save energy. Because entropic decay is ever present in every system, every system you encounter will be chaotic to some degree. Which means you'll have to improvise at some point. You can plan for a lot, but not everything. So you need to be good at improvising, which means good at learning and knowing what you need. So we get these tools like hunger, fear, excitement, that help us jump to important conclusions faster.

Theoretically, if you're in an ever present state of improvisation, it makes sense to combine learning, experience, emotion, assumption, and conversation to try to jump to the safest conclusion with the least amount of energy and then error correct as needed.

You mean talking to everyone at the office about traffic is just biological energy conservation in action? Yes. But, please don't talk to me about traffic.

That's the very top level view of working with abstraction. Learn the system. Learn it's specifics, and try to talk with it (metaphorically) in some way.

This might be why so many industries that do 3D modeling have completely different CAD software from company to company. Each version of this extremely complex software has an extrude function, but one CAD program is built for video rendering, another is built for real world millimeter accuracy, and another is free to use

We scaffold on top of abstraction to get to what we need faster. In computer science that's what writing your own library is.

And that leads right into THE THESIS!!!!

SToAS!

Saywer's theory of abstraction through repetition. Wait...

Sawyer's theory of mutation through repetition? No no hold on...

Sawyer's theory of abstracting specificity!

When you abstract something, you abstract the specific way that it is done. There are always flaws and limitations involved. If you want to interact with an abstracted system in a new way it must already be able to do that. If it can, you can work on the underlying system, if it is complex enough (like a person,) talk with it to zero in on what you want, or scaffold something on top of it based on what you've already learned.

But SToAS essentially means that any time you abstract away an idea, you lose control of exactly how it's executed and you accept tradeoffs.

On a meta level, if heavy abstraction is required, trust is required. Trust can come in all forms, verification, testing, review, word of mouth, prior experience with a thing.

I think as an industry we're looking for a cure-all for dealing with LLM generated code. Some are leaning into compilers. Others are sure that the constant nature of change necessitates a constant out of sample error in the LLM training data.

Generally progress is made by fixing one thing at a time. And you probably know at least one thing you want to fix.

So, go out there and talk to your robot!