"The Turk", an illustration of Johann von Kempelen's chess playing automaton. (image in the public domain)
Sorting the utility from illusion of artificial intelligence
an interview with Jeff Prosise
by Paul Huber
October 2025
"The most important intelligence you will deal with in your lifetime is your own. Take care of it."
Businesses will profit or lose based on understanding what AI is and how to use it. And individuals will enhance their productivity or atrophy their
skills. Understanding what it is and isn’t, and knowing how to use it as a tool is important, because it’s not going away and will continue to evolve
and improve.
Explosive popularity:
In June of 2017, a research paper was published titled, “Attention Is All You Need.”
"We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less
time to train."
The Transformer architecture, and attention mechanism within it, gave AI models better software ability to “understand” the context and meaning of their
training material. And “more parallelizable” meant they could better utilize advanced multi-core processing hardware. Additionally, the scale of the
models got bigger giving them the benefit of the 'law of large numbers'. The combination of these things was a huge leap leading to the development of
the ChatGPT (text) and DALL-E (image) generative AI models. By version 3 of ChatGPT (released 2022) the generated output had gotten quite impressive, and
writing & images from these models were flooding social media, grabbing everyone’s attention.
Thought or math?
If you define thought as involving consciousness or awareness, then the simple answer is math. In the case of LLMs (large language models), they “scrape”
the Internet for large amounts of training material. This material is in the form of articles, published books and studies, social media discussions,
etc… All human intelligence, which it pattern-matches. For text material, it’s broken down to tokens (often smaller units than words) and then these are
given numerical values, called embeddings, which represent their meaning and relationship within the material. This is part of the first model phase,
called training. The second phase is called inference, in which it takes the user’s prompt and generates an output based on applying the prompt to what
it learned in training. A prompt can be a simple question, “What is a horse?” or you can ask it to write a short story about camping… The same numerical
conversion, “encoding”, is applied to the prompt and then statistical math is used to predict the best answer/output based on the patterns it learned in
training. It’s mathematically mimicking human intelligence. It’s not artificial general intelligence (AGI) or artificial superintelligence (ASI). Those
are currently just theoretical. But it’s important to note that AI is a moving target to define and future advancements can only be guessed at.
What to do with it?
A July 2025 study by MIT’s Project NANDA
found “95% of organizations are getting zero return” on AI investment. The results were based on “interviews, surveys, and analysis of 300 public
implementations” (quotations from the executive summary).
But looking past that headline, you can learn that the AI technology was probably not well understood, prepared for, and properly applied in those failed
cases. Those things are critical to generating a return on investment and improving customer satisfaction. It’s very important to first take time to
understand what AI is and isn’t, and figure out where its capabilities can realistically improve business operations. Resist the temptation to throw
money at it because of FOMO (fear of missing out).
Individuals should be careful not to over-rely on it. Trust in your own thinking, talents, and creativity, and work to improve them. Those skills will
deteriorate if unused because of having AI do it for you. So think of it as a tool to assist, not replace. One area where it’s becoming exceptional,
though, is research assistance. You can ask a question and it will find an answer in seconds that previously might take hours or days.
Jeff Prosise is the Chief Artificial Intelligence Officer at InRule Technology. He’s also
been a social media friend of mine. Interacting with him for a few years has given me trust in his thinking and judgment on the topic, so I asked him
to do the following interview:
Q1. What is your professional background and how did you become interested in AI?
I studied mechanical and aerospace engineering in college and practiced that for a few years. Then I discovered that my true passion was for writing code.
In the 1980s, I started writing articles and columns for publications such as PC Magazine and Microsoft Systems Journal. That led to writing books, doing
training and consulting work, speaking at conferences, and ultimately starting my own company (Wintellect). Since Wintellect was acquired in 2021, I’ve
served as Chief AI Officer at three companies. I’ve had a lot of fun working with clients to incorporate AI into their apps and business processes.
I first became interested in AI back in 2013 when I was doing some work for Microsoft Research that involved machine learning. I was fascinated by
everything I learned and that motivated me to learn more. I went from machine learning to deep learning and then vision models, language models, and
generative AI. I’m having more fun professionally right now than I’ve had in 30 years.
Q2. I’ve heard several business analysts say, “Every company will need to become an AI company.” How would you interpret that?
Companies that ignore rather than embrace AI place themselves at a competitive disadvantage. AI is a productivity enhancer. As a programmer, I routinely
rely on AI to help me write and modify complex SQL queries, CSS, and regular expressions. Rather than spend two hours trying to figure out how to modify
a huge CSS file to move a button a few pixels to the right, I let AI do it for me. This allows me to spend more time on what’s important and less time
wrestling with details that should be trivial but end up being time sinks.
Q3. What shortcomings in current AI technology concern you the most?
Most of what we refer to as AI today involves language models (LMs). Language models are amazing, but they aren’t intelligent (IMO). They are next-token
predictors trained on massive datasets. They don’t understand what they output. They simply parrot what they learned during training. Using techniques
such as Chain of Thought (CoT) prompting, they can mimic reasoning to an extent. But studies have shown that they can’t “reason” beyond the boundaries of
their training data.
That’s not a problem if you understand what language models are and how they work. They enable all sorts of things in software that weren’t possible a few
short years ago. But when people expect them to serve as search engines and be truthful and accurate (concepts that are hard to define mathematically),
they fall short.
Q4. For important tasks, the hallucination problem seems quite concerning. Can you briefly describe it, and your ideas of how it could be
minimized?
Language models hallucinate because they build responses one token at a time with no understanding of what they’re generating. They’re not smart.
They’re sophisticated statistical machines that mimic what they’ve seen in training. Because virtually all language models today use some form of
subword tokenization, in which some words are broken into multiple tokens, a model can even, on rare occasions, hallucinate words that don’t exist when
it outputs two consecutive tokens that are never paired in the real world.
There are prompting techniques that reduce the likelihood that a model will hallucinate. There are also patterns such as Retrieval-Augmented
Generation (RAG) that decrease the chance of hallucinations. But at the end of the day, you can never guarantee that hallucinations won’t occur. You
have to account for this and make sure there are humans in the loop at critical junctures. I would never, for example, send an e-mail generated by a
language model to a customer without reviewing it first.
Q5. Are you aware of any current capability that would allow artificial systems to generate “intelligence” beyond the intelligence of the
human data they train on?
I am not. I don’t believe language models are “intelligent” in the human sense. They are statistical next-token predictors, albeit incredibly good ones.
Sir Isaac Newton invented calculus. Could any of today’s LMs invent something so new and remarkable? Doubtful.
To be sure, language models can mimic intelligence. Give them a list of tools that allow them to access external resources and APIs and they are smart
enough to know when to call those tools. At risk of making them sound sentient (they’re not), LMs are aware of their own limitations and are perfectly
willing to use external tools when available. This is the foundation for most AI agents. We instill “intelligence” in them by providing them with useful
tools. Most agentic frameworks use the same mechanism (tool use, which is really just function calling) to pass messages between agents.
Q6. The computer science industry is working towards artificial general intelligence (AGI), the level at which systems have human-like
intelligence ability. Opinions on reaching that vary widely. What is yours?
I don’t think we’re close. Of course, it depends on how you define AGI, and opinions on that vary.
Here’s a recent experience I think about. I was asked to use AI to update a company’s case studies. It’s easy enough to upload a PDF to an LM and have it
extract and update the text. But if you ask it to update the text in the PDF, it can’t do that. Some LMs can generate a PDF, but to my knowledge, none
will replace the text in an existing PDF while preserving the formatting and such. Yet most humans would have no trouble following instructions to update
the text in a PDF, even if they had to download a tool for editing PDFs. Sure, AI could be trained to do that, but there would still be a million other
things it couldn’t do without specific training.
Here's another thought. We can train a neural network to differentiate between photos and dogs and cats. To first order, it needs thousands, or perhaps
tens of thousands, of dog and cat images to differentiate as well as a person. Yet a child doesn’t have to see thousands of cats and dogs to learn to
differentiate between the two. We have found clever ways to allow machines to do things that were impossible a few years ago, but are we mimicking the
human brain with those machines? I don’t think so.
Q7. Where do you see AI having great impact, and where do you think it will disappoint?
It will have a huge impact in medicine, finance, accounting, law, engineering (including software engineering), and just about any “professional” career
you can name. AI can already spot breast cancer up to 5 years earlier than doctors.
Where will it disappoint? Anywhere we expect it to be perfect. It’s not and never will be, just as humans aren’t perfect. It learns everything it knows
from us, after all. And when AI paired with search gives an incorrect answer, it’s because it relied on inaccurate information generated by humans (or
perhaps because it hallucinated). Accuracy is a difficult thing to measure. In some cases, it’s impossible to measure. We can tell whether an answer to
the question “What year was George Washington born” is accurate. But if you ask AI whether Bill Clinton or Obama was the better president, how do you know
if the answer is accurate?
AI will also disappoint when it’s used for nefarious purposes, which it inevitably will be. I tell folks that our job is to make sure the positives
outweigh the negatives and that AI, on balance, makes this world a better place to live.
Q8. What is a good way for businesses, new to AI, to get started in a way that will help their brand and have the best chances for a positive
return on investment?
Find a handful of employees who are smart and passionate about AI and empower them to experiment with it and devise ways to use it to benefit the company.
I spoke to the CTO of a bank this week who did just that. His employees came up with brilliant ideas that he alone would never have thought of.
In my experience, success with AI in a corporate environment comes from the ground up, not the top down. I see many companies setting up AI committees
whose job is to plan how the enterprise will use AI. That’s fine, but in many cases, the people who form those committees don’t know much about AI
themselves, and they may not understand all aspects of the business. Trust your people. Erect safeguards to be sure, but don’t assume that the AI
equivalent of a centrally planned economy will produce the results you want.
Q9. Generative AI can produce some clever writing and images, but it can be clichéd, a sort of ‘regression to the mean’ of the content it has
trained on. What is a good way for individuals and creators, new to AI, to get started in a way that allows their talents to stand out above that?
Roll up your sleeves. Build something. Don’t be afraid to make mistakes; that’s how you learn.
Realize that generative AI is all about prompting. The more detailed and specific your prompts, the more likely you are to get what you want. Talk to a
language model as if you were talking to child. Be specific. Tell it what you want it to generate. Faced with ambiguity, a model will typically make
assumptions and resolve those ambiguities itself.
Incrementalism is useful, too. Let’s say you’re using an LM to build an app. Rather than give it a 100-page prompt detailing everything you want in
the app, iterate. Build it up like a layer cake. You’re more likely to be successful that way.
Q10. Things don’t always progress at a linear rate. There’s been a big leap in the last 8 years but that could be difficult to sustain
without continuing major breakthroughs. How do you see artificial intelligence evolving from here?
That big leap came from the invention of the transformer architecture. Without that, we wouldn’t have LMs. I believe progress will be incremental
until we find a new architecture for neural networks (or perhaps an alternative to neural networks altogether) that are as revolutionary as
transformers.
I firmly believe that transformer-based LMs, while remarkable, will NOT get us to AGI. Of course, whether we achieve AGI will depend on how we
define AGI. And right now, there is no general consensus. □
Major AI models (alphabetical order):
Anthropic Claude
Google Gemini
Meta AI
Microsoft Copilot
OpenAI
xAI Grok
