✉ Envelope #52: How ChatGPT works & why can't it do engineering (yet)

Good morning! Andy from Back of the Envelope here.

You've probably heard a lot of AI buzzwords floating around over the past few months... things like "AI agent", "prompt engineering", "LLM (large language model)"...etc.

Maybe I'm in an AI echo chamber, but I also hear things like:

• Agents will disrupt the industry by automating everything!

• They can do in 5 minutes what used to take a whole team an entire week.

• They're going to replace our jobs!

(I’m paraphrasing, but you get the idea.)

If you’ve just started using chatbots and are beginning to see their "magic," some of these claims might feel a bit overwhelming, or even threatening.

Over the next few emails, I want to demystify AI chatbots and agents a bit. Once we understand how these systems actually work, it may be much easier for us to spot hype, ride the wave, and predict where this might all be heading.

We'll start with LLM today.

Let’s go!
(Estimated read time = 3 minutes & 30 seconds)

The Foundation: What Is an LLM?

LLM stands for "Large Language Model." It's the underlying tech behind chatbots like ChatGPT, Gemini, and Grok…etc.

How Does an LLM Work?

In a nutshell, it "predicts" the next word.

Kind of like your phone’s autocomplete, just on an entirely different scale -- like million times more advanced.

It doesn’t “think” like a human (except for reasoning models, which we’ll touch on in another email).

What it does is make educated, statistical guesses (called inferences) based on patterns it learned during training.

What Is "Training"?

Companies like Google and OpenAI gather massive amounts of text (billions and billions of words) from books, academic papers, websites, forums, and more. They use this to train the model to understand how language works.

The model doesn’t memorize like humans do, it learns patterns. Over time, it builds a statistical "feel" for grammar, facts, and even engineering concepts.

Think of it like looking at thousands of structural drawings. Eventually, you just know what a framing or foundation plan should look like. Similar idea.

So What’s an "Inference"?

The model doesn’t recall facts like we do. It recognizes what words and ideas tend to appear together.

So when you ask it a question, it doesn’t “understand” you. It predicts the "next most likely piece of text", based on everything it’s seen before.

For example:

• The sky is... "blue"

• Structural engineers design... "buildings"

• Tell me something funny. Structural engineers are... "the people who say, 'Yes, your design is beautiful... but will it fall down?'

But It Feels Smarter Than That…

Yes, it does.

… it’s not just spitting out a sentence... it’s generating full paragraphs, even entire essays.

But it’s still predicting one token at a time (a token might be a word or part of a word), using what it’s already written as context.

Kind of like how we write: we jot down a few sentences, re-read them, and then write the next part to make sure everything flows.

LLMs do the same thing, but thousands of times per second.

Here’s where it gets cool: because of how it's trained, it can combine patterns from different areas, even ones that never explicitly appeared together.

For example: Ask it to write a code section about a peanut butter sandwich in the style of the International Building Code.

It’s likely never seen that combo before, but it has seen peanut butter sandwiches, and it has seen building code formatting.

So it generates this:

Enter Hallucinations

This is where LLMs can go off the rails.

Sometimes they make things up. These mistakes are called hallucinations.

In our peanut butter example, it’s funny. But if you’re relying on it to draft a real engineering calc or report, it could be risky.

This is part of the LLM’s nature. Developers are constantly working to balance creativity vs. accuracy.

For instance, a tool like NotebookLM, which we’ve discussed before, is designed for accuracy. It’s great for summarizing specific sources, but not very creative. It might even say, “No relevant info found,” rather than guessing.

Compare that to ChatGPT, which tries to be helpful and creative, but might make stuff up.

To reduce hallucinations, some systems now have tools like reasoning chains, python calculators, and internet search. (We’ll dig into that in a future emails.)

So What Does This Mean for Structural Engineers?

By now it should make a bit more sense why LLMs sometimes get things wrong when you ask them to design a beam or calculate a moment.

They’re not actually crunching numbers line-by-line.

They’re guessing what a good answer might look like based on patterns.

Unless you’re using a reasoning model (more on that soon), you’re mostly getting statistical echoes, not real calculations.

And that's it for now. Hope this helped! (Let me know if it does).

Stay tuned for the next ones, where we’ll talk about reasoning models and AI agents.

Until next time,

Andy Lin
Back of the Envelope/Structural Engineer HQ

PS.

If you are working on a steel building and you haven’t heard of Durafuse, you might want to check it out.

It is a proprietary steel moment frame connection that offers high ductility and fast recovery after an earthquake.

And they’ve got your back with full design team support related to the connections: calcs, plan check comments, RFI responses, and shop drawing reviews.

Click here to learn more: https://go.sehq.co/durafuse

PPS.

If you think your colleagues, friends, and others would like a perfectly-made PB sandwich and might find this email helpful, tell them to sign up here. (thanks!)