HilliTech's Newsletter: Misplaced trust in AI

Siri AI is a useful tool, but knowing what the tool is good for is most important

Artificial intelligence is a dumb name for a useful technology that can be a productivity-enhancing tool when used appropriately. "Trust" shouldn't even be a part of the discussion, but here we are.

Here are some stories I'd like to share this week:

Trust is a funny thing

You can trust a friend, a teacher, a colleague, or even a hammer. So, sure, someone could place trust in something like an AI tool or chatbot.

However, I believe that the trust needs to be placed appropriately. I don't trust my hammer to do my taxes, nor do I trust my ketchup to give me relationship advice.

The only "trust" we should be discussing when it comes to AI is its reliability to be a tool, not a person. AI isn't our friend, partner, doctor, or even educator.

There is no situation where you should be anthropomorphizing your AI chatbot, no matter how good it is at playing pretend.

Look, we can point fingers in a million different directions for this one example of a person using ChatGPT for medical advice and nearly dying. OpenAI should have safeguards, the person involved should know better, or so many other things should have happened before the person nearly died of a blood clot.

I think humanity needs to learn two very specific lessons, and soon: Tools aren't replacements for people and AI isn't intelligent.

I place the blame for the first lesson squarely on OpenAI and other companies for creating the language around AI that applies human conditions to the tool. AI doesn't think, reason, or even technically "chat." AI is a readout, not a conversation. You don't "talk" to the chatbot, you provide input.

Nearly every word we use to describe interacting with AI is based on human interaction, and that's fundamentally wrong. I don't think there's any going back, at least not easily, but we as a society need to understand this concept.

Apple might be one of the only companies bothering to ensure AI is a tool, not a friend. It does so through several UI decisions:

Siri can't and won't roleplay or pretend to be anything other than Siri
Information with a verifiable source has a link to that source data
World knowledge is often provided verbatim from a source, not summarized, which can lead to hallucinations
Anything involving health, harm, threats, violence, or similar will lead you straight to contacting emergency services
Siri doesn't attempt to inject information to flatter you, like your religious preference, nor does it become sycophantic (at least not easily)

AI, for me, is a tool that can provide me with information. Siri AI takes things further by being attached to various controls, like music and the home, so I can use Apple's AI to interact with the world. These are the foundations of Siri when it debuted in 2011 and continue to be the same today, just vastly expanded in capability.

I don't go to Siri for medical advice, though I won't be opposed to an AI in Apple Health powered by Apple Foundation Models. Why Apple and not OpenAI in this regard? Because of their approach to AI and how it interacts with your data.

Apple's Health AI will inevitably play things safe and avoid giving direct advice beyond seeing a medical professional. Leaks suggest that Apple has built a directory of doctor-backed medical information and videos of real doctors describing some conditions and concepts. Your health data and interactions with the Health AI will lead you to these pre-determined results, not AI summaries of information.

That is how AI should work. At its base level, it is a computer program that is great for cataloging, sorting, and finding trend lines. That data can be used as a reference point to find related details in videos and papers created by human doctors, or whatever it is you're doing.

Apple's Support Assistant works similarly. If you ask it about Apple products, it surfaces a manual written by a human. It doesn't suddenly start telling you how cool you are for asking the question and faking having thoughts. It just fetches the data like a good program should.

AI doesn't "know" anything, just like Microsoft Word doesn't know grammar. We don't pretend that a dictionary is smart, it's a book containing information that we can reference.

When it comes to "trust" and AI, I am very careful in how I interact with Siri AI, because so far, that's the only one I'm using. When I ask a question, I often skim over what is shared in the interface then open the source links. The problem being, if you don't know what is correct or not, how will you know which part is potentially hallucinated without checking first?

That's the strength of Siri AI from what I've observed so far. It directly quotes sources, avoids summarization for the most part, and rarely encounters situations where it might have the possibility to hallucinate. The problem is you can still encounter a hallucination, so keep that in mind. Trust only what you already know and verify the rest.

For those of you that want to use AI as a chatbot or companion, even roleplay, it's important to maintain a respectable understanding of the difference between reality and fiction. No matter how human-like an AI is, it will never be human and can't "understand" anything. It's an information resource and tool for coding, analysis, and similar.

Humans understand that a toy is just a toy and won't suddenly spring to life to give us marital advice. If you're engaging with AI chatbots for entertainment, then let it remain entertainment. We don't suddenly expect Marathon to become sentient and provide medical advice, so we shouldn't have that occur with AI chatbots.

The blame lies in a lot of places. Our education system in the United States is built around test scores, not understanding. OpenAI's bottom line depends on people mistaking AI for their friend, doctor, teacher, investor, or partner. The government benefits from having a society dependent on anthropomorphic computer programs.

AI companies aren't going to change things. Regulators aren't going to do anything. Our education system is impossible to fix anytime soon. Sadly, as with many things today, it is on us, the individuals, to make a change.

If you're the nerd in your family or community, it is on you to help the people around you understand the nature of AI. It's a tool, a toy, not a human.

Trust your hammer to hammer in nails. Trust your vending machine to provide soda when you pay for it. Trust your AI to provide a link to human-recorded knowledge when you ask for it.

A gray cat on the left, a dark gray cat with a splotch of white over its left eye on the right, in front of a window — Mina and Evie share a brain cell believe balls of receipt paper are sentient