LLMs = 3D Printing

A few years ago, a startup company started to "3D print houses". Really, they just splooged out concrete walls onto a slab with a gantry concrete thing. Lots of shitheads thought it was "disruptive". There were many breathless videos on youtube about how it would change everything, but those were videos made by shitheads for shitheads. Since youtube has a comments section, you could see what people in the home construction business thought about it: not much. The construction industry stats say only about 15% of the cost and time for home construction go into stick framing.

The 3D printed walls concept was a follow on to the massive hype wave of 3D printing in manufacturing and how it would "change everything" too. It's not clear why that particular technology got so much hype. Nobody cared similarly about pattern cutting machines, or lathes, or CNC controlled machine tools, or whatever: many of those things have been around since the 1950s. 3D printing turned out to be another tool in an already enormous toolbox. AFAIK, the 3D printed walls concept fizzled out entirely in home construction.

To me, LLMs look a whole lot like 3D printing with respect to the massive hype wave and speculative stock market spaz out. I finally got around to setting up a small home system to get some hands on understanding of these things. I'm currently using one of the smaller demo-ish models available from Meta's open source models because I don't want to spend $10,000 on e-junk GPUs to use one of the large models.

Initially, even the small LLM is mind-blowing. It's wild to type in a question like: who are all the Cleveland Browns quarterbacks since Tim Couch? And get an answer that's probably 99% correct. You can ask similarly obscure questions about many topics and get an answer. The whole Internet, or at least most of the wikipedia is compressed into a database of weights that's only around 4-5 GB. However, it becomes apparent quickly that the LLM has severe limits and is often "buggy". In fact, in some scenarios it will spout endless nonsense in response to some prompt.

Some engineers I know are riding the wave of agentic "coding", which is actually a really complicated version of human coding. It's another type of programming, where instead of directly writing code in "C" or System Verilog or whatever, an engineer writes up an elaborate prompt, which is then iteratively modeled and tested, ideally all by the agents.

The killer problem with all this stuff is it appears to 99% work as long as there is a bunch of prior art and public code it can model... Even though it's limited, it is amazing to see in practice. Also it only "works" for people who are already experts. It does not work at all for people who aren't--in fact for them it's like a slow acting poison or "rabbit starvation" scenario. The junior engineer people who adopt the agentic coding approach will never be experts.

I think this current approach will end up dominating several specific real-world use cases. For example, it's a significantly "better manual" than most manuals, or a better how-to book than most how to books. For example, say you have a specific farm tractor and want to change the oil and filters, it's a PITA to find that type of info in a PDF manual, for example, so it would be good to ask in natural language "what's the part number of the oil filter?" and get the correct answer. Companies will probably start to publish their info in some manner that's very friendly to LLMs.

The key factor with LLM as "manual" is there's little to no impact from an incorrect answer to a question, like if you order the wrong filter because the LLM produced nonsense, the typical impact would be it doesn't fit correctly. The same type of error could happen with the traditional methods of looking up a part number or even ordering the correct part number but getting the wrong thing delivered. All of that is already "imperfect".

I can see it replacing GUIs for complex systems, like large networks with lots of different hardware boxes, but a very specific system has to be constructed for that to really function the way people imagine it should: that is flawlessly. The model would have to be able to iteratively experiment and prove the network configuration with a high-fidelity simulator before applying the configuration to a real system. That scenario is a bit like the 3D printed walls scenario.

Anyway, the LLM technology really is cool and interesting, however the 4chan maxim "nothing ever happens" is a good one to keep in mind when something is hyped to the skies. The trope "this changes everything" is so common, yet so often wrong.