Jianshanxing Technology

English

中文

English

中文
Sponsored by [KnowFun]: Transform your materials into personalized learning with AI.

You Think AI Can Do Everything? 90% of People Overestimate AI — and Underestimate the Risks ​

We · October 2026

This is the fifth article in the "Core Abilities for the AI Era" series. The first four covered the amplifier effect, taste and discernment, problem definition ability, and systems thinking. This article discusses the most easily overlooked foundational ability: AI literacy — knowing what AI can do, what it can't do, and how and when to use it.

Let's start with two true stories.

The first story is about a lawyer.

In 2023, an American lawyer submitted a brief to court citing several precedents to support his argument. The judge looked at it, didn't recognize these cases, and asked him to verify. The verification came back — all the precedents were made up by ChatGPT. They didn't exist at all.

The lawyer was fined and faced disciplinary action.

You might think this is an old joke, that the guy was just stupid — who would just use something AI wrote directly in court?

But did you know that by 2026, this kind of thing hasn't gotten less common — it's gotten more common.

Deloitte's 2026 AI Adoption Survey has a data point:

"Forty-seven percent of enterprise AI users made at least one major business decision based on hallucinated content in 2024."

— Deloitte, 2026 AI Adoption Survey

47% of enterprise AI users made at least one major business decision based on AI-fabricated content in 2024.

47%. That's roughly one out of every two people.

How much damage do AI hallucinations cause globally?

"AI hallucinations cost global enterprises an estimated $67.4 billion in 2024 alone."

— NeuralWired, 2026

In 2024 alone, AI hallucinations cost global enterprises an estimated $67.4 billion.

$67.4 billion — that's the annual GDP of a mid-sized country.

And this problem still hasn't been truly solved. Even the best models have shockingly high hallucination rates in certain fields:

"Even the best AI models still hallucinate at least 0.7% of the time on basic summarization tasks — and rates skyrocket to 18.7% on legal questions and 15.6% on medical queries."

— AI Hallucination Statistics Report, 2026

Even the best AI models hallucinate at least 0.7% of the time on basic summarization tasks — and rates skyrocket to 18.7% on legal questions and 15.6% on medical queries.

In the legal field, roughly 1 out of every 5 answers is made up. Would you dare use that directly?

The second story is about Black Friday.

November 28, 2025 — Black Friday. One of the most important days of the year for global e-commerce.

But on that very day, the ChatGPT API experienced widespread outages — many e-commerce customer service bots, product recommendation systems, and AI shopping guide features that relied on ChatGPT API all went down.

Consumers clicked on websites wanting to ask about products, but the smart客服 didn't respond. They wanted AI to recommend something, but the page just kept spinning.

How big were the losses? There's no exact number, but you can imagine — what does a few hours of customer service and recommendation system downtime on Black Friday mean for an e-commerce business?

And that wasn't even the worst one.

On November 18, 2025, Cloudflare went down globally — ChatGPT wouldn't open, Claude wouldn't open, even DownDetector, the website people use to check outages, wouldn't open. According to statistics, about 20% of internet services worldwide were affected that day.

"A single infrastructure failure took down ChatGPT, X, and 20% of the internet. The incident exposed a fundamental vulnerability: when everyone relies on the same few cloud providers, everyone goes down together."

— Axis Intelligence, Cloudflare Outage Report, 2025

A single infrastructure failure took down ChatGPT, X, and 20% of the internet. The incident exposed a fundamental vulnerability: when everyone relies on the same few cloud providers, everyone goes down together.

On December 26, during the holiday shopping season, the ChatGPT API went down again for over 4 hours.

These stories all point to the same problem: most people use AI but don't truly understand AI.

They don't know that AI can talk nonsense with complete confidence.

They don't know that AI services can go down too — and when they do, it's global.

They don't know what it means to send data to online AI services.

They don't know that in some scenarios, AI simply shouldn't be used at all.

What Is AI Literacy? ​

We often talk about "digital literacy" and "media literacy." In the AI era, there's a new term: AI Literacy.

What is AI literacy? Simply put: knowing what AI can do, what it can't do, and how to use it in different situations.

It sounds simple, but most people don't actually have it.

Many people's understanding of AI falls into two extremes:

  • Deifying AI: Thinking AI can do anything and everything, will eventually replace humans
  • Dismissing AI: Thinking AI is just a toy, always talking nonsense, completely unreliable

Both are wrong. AI is neither an omnipotent god nor a useless toy — it's a tool with specific capability boundaries and specific risks. True AI literacy is figuring out where those boundaries are, where the risks are, and how to use AI to maximize strengths and minimize weaknesses.

UNESCO defines AI literacy as:

"AI literacy is the set of competencies that enable individuals to engage meaningfully with AI technologies — including understanding how AI works, knowing its capabilities and limitations, using it critically and responsibly, and making informed decisions about when and how to use it."

— UNESCO, AI Competency Frameworks

AI literacy is the set of competencies that enable individuals to engage meaningfully with AI technologies — including understanding how AI works, knowing its capabilities and limitations, using it critically and responsibly, and making informed decisions about when and how to use it.

Notice the key words: using it critically, making informed decisions.

That's the core of AI literacy — not using it blindly, and not avoiding it blindly, but clearly knowing what you're doing.

Judging How to Use AI: Five Dimensions ​

Specifically, to judge how to use AI in a given scenario and which kind of AI to use, you mainly look at five dimensions.

Dimension One: Accuracy — Is What AI Says Correct? ​

This is the most basic question: can you trust the content AI generates?

The answer is: it depends.

  • Casual chat, brainstorming, writing first drafts — AI is probably fine
  • Factual questions, data, citations — you need to verify yourself
  • Professional fields like law, medicine, finance — hallucination rates can reach 15-20%, absolutely can't be used directly

Gartner estimated:

"The average cost of a single high-stakes hallucination incident is over $1.2 million when legal exposure, remediation, and reputational impact are included."

— Gartner, 2025

The average cost of a single high-stakes hallucination incident exceeds $1.2 million — including legal exposure, remediation costs, and reputational damage.

So for all important, high-stakes content, human review is essential. This isn't a question of whether AI is good enough — it's basic risk management.

Dimension Two: Privacy and Compliance — Is Your Data Safe? ​

This is a question many people overlook: where does the data you send to AI go?

When you use online AI services, every sentence you type, every document you upload, gets sent to the provider's servers. That data might be used to train models, or stored, or analyzed.

What if the data contains sensitive information?

  • Customer personal information → might violate GDPR, personal information protection laws
  • Company trade secrets → might leak to competitors
  • Internal documents and code → might create intellectual property risks

EU GDPR penalties for personal data can reach up to 4% of global turnover. For large companies, that could be hundreds of millions or even billions of euros.

"Seven obligations define GDPR-compliant AI: data minimization, purpose limitation, transparency, right to explanation, DPIA, DPO consultation, and limited data retention — with penalties of up to 4% of worldwide turnover."

— Jaikin EU, AI and GDPR, 2025

Seven obligations define GDPR-compliant AI: data minimization, purpose limitation, transparency, right to explanation, DPIA (Data Protection Impact Assessment), DPO (Data Protection Officer) consultation, and limited data retention — with penalties of up to 4% of worldwide turnover.

For scenarios involving sensitive data, local deployment is much safer than online APIs. Because data never leaves your computer or server — you're in complete control.

Dimension Three: Reliability — Will AI Go Down? ​

Many people take it for granted that AI services are "always online." But the reality is:

  • November 18, 2025: Cloudflare global outage, ChatGPT/Claude and many other services unavailable
  • November 28, 2025: ChatGPT API widespread outage on Black Friday
  • December 11, 2025: ChatGPT mass 429 errors, rate-limited even when limits weren't reached
  • December 26, 2025: ChatGPT API down for 4+ hours during holiday season

And those are just the major outages. Smaller-scale rate limiting, slowdowns, and partial feature unavailability happen almost every month.

For individual users, a few hours of downtime might not matter. But for businesses —

  • Customer service systems down → user complaints, lost orders
  • Recommendation systems down → sales decline
  • Content generation down → production schedules affected

If you depend on AI for critical business, reliability is a big problem.

Locally deployed models can have problems too, but at least control is in your hands — you know when it's up and when it's down, and your business doesn't grind to a halt just because a data center thousands of miles away has an issue.

Dimension Four: Cost — Can You Afford It? ​

Many people think online APIs are cheap, and it's true — a few cents, a few dimes per call, doesn't seem expensive.

But what if your call volume goes up?

  • A company of 100 people, each using the API 100 times a day → that's hundreds of thousands of calls a month
  • A RAG system that calls the model multiple times per query → costs grow linearly
  • Peak traffic doubles → your bill doubles too

And online API prices aren't up to you — providers can raise them whenever they want, and there's not much you can do.

Local deployment is different:

  • Hardware is a one-time investment, once bought it's yours
  • Models are free open-source models, no extra cost no matter how many times you use them
  • The more people using, the more frequently used, the more cost-effective local deployment becomes

Cost isn't about being as low as possible — it's about being as controllable as possible. Online APIs are "pay-as-you-go," which seems flexible but costs are unpredictable. Local deployment is "upfront investment," which seems expensive but long-term costs are clear and controllable.

Dimension Five: Control — Can You Modify It? ​

The final dimension: how much control do you have over AI?

With online AI services, you can only change the prompt — the model is someone else's, data is stored on someone else's servers, features are decided by someone else. You can use it, but you can't change it.

With local deployment?

  • You can choose the model, if you don't like this one try another
  • You can fine-tune the model to make it understand your business better
  • You can connect your own data, your own knowledge base
  • You can build your own workflows, your own systems
  • You can fully control output quality and style

The essence of control is whether you can define AI according to your own needs.

When requirements are simple, online APIs are enough. When requirements get complex, when customization demands are high, you need more control — and that's when local deployment becomes the natural choice.

Self-Assessment: What Level Is Your AI Literacy At? ​

Ask yourself a few questions:

  1. Do you know how the AI services you use regularly handle your data? (Will it be used for training? Will it be stored? For how long?)
  2. After you generate content with AI, do you verify factual information? Or do you just use it directly?
  3. Do you know which fields AI is most likely to make mistakes in? (Law? Medicine? Code? Math?)
  4. If an AI service goes down, do you have a backup plan? Or does your business just stop?
  5. Can you judge whether a scenario is better suited for online APIs or local deployment? What's your basis?

Four Levels ​

LevelCharacteristicsRisk
NoviceAI is a black box, just use it, don't care about principles or risksHigh — might fall into pitfalls without even knowing how
CautiousKnow AI makes mistakes, check important things yourselfMedium — can avoid most obvious risks
InformedKnow AI's capability boundaries, choose appropriate usage based on scenarioLow — can use AI well and control risks
ArchitectCan design AI solutions based on needs, know when to use online vs. localVery low — actively manage risk, maximize value

Most people are between "Novice" and "Cautious." Reaching "Informed" already puts you ahead of 90% of people.

How to Improve AI Literacy ​

AI literacy isn't some deep学问. It's more of an awareness — an attitude of "not being blind, having judgment" when using AI.

Three specific practice methods:

1. Develop the Habit of "Verification" ​

No matter how much you trust an AI, whenever factual content is involved — numbers, citations, cases, legal provisions, medical advice — develop the habit of verifying.

You don't need to check everything, but for important, high-risk, decision-driving content — always check.

How to verify?

  • Key numbers → find the original source
  • Citations and references → search to see if the original exists
  • Legal/medical → get professional confirmation
  • Product/pricing information → check the official website

AI can help you find direction, but the final judgment has to be yours. This isn't distrust of AI — it's taking responsibility for yourself.

2. Figure Out Where Your Data Goes ​

Next time before you use an AI service, spend two minutes looking at its privacy policy — focus on these questions:

  • Will data I input be used to train models?
  • Will data be stored? For how long?
  • Will data be shared with third parties?
  • Can I delete my data?

If you're okay with all the answers, then use it freely. If you can't accept them — for example, involving company secrets, customer privacy, sensitive data — then find an alternative:

  • Use a locally deployed model
  • Use an enterprise service that promises data won't be used for training
  • Anonymize/desensitize data before using it

The data is yours, and the control should be yours too.

3. Try Local Deployment to Build a Feel for "How AI Runs" ​

One of the best ways to improve AI literacy is to run a model locally yourself, hands-on.

You don't need to do anything complicated — even just installing Ollama, running a 7B model, chatting a few times, and your understanding of AI will be completely different.

You'll learn:

  • How big models are, how much VRAM they take
  • What generation speed depends on
  • Where the differences in效果 between different models lie
  • What quantization means, and why it's needed
  • How the experience of local running differs from online calling

These intuitive feelings can't be gained just by reading articles. Run it once yourself, and your understanding of AI shifts from "abstract concept" to "concrete thing."

And you'll discover — local deployment isn't that hard, it runs on a regular computer. You can install it in one click on We, and start experiencing it right away.

Final Thought: Knowing the Boundaries Is How You Use Tools Well ​

Returning to our core proposition: AI is an amplifier.

But there's a前提 for amplifiers — you have to know where their boundaries are.

People who don't know the boundaries either treat AI like a god, believing everything, and eventually get burned by hallucinations — or they write off AI as useless, abandoning it entirely after one mistake.

People who know the boundaries can accurately judge:

  • This task, AI can do well → use it confidently, boost efficiency
  • This task, AI might make mistakes → human review, control risk
  • This task, the data is too sensitive → use local deployment, protect privacy
  • This task, business can't stop → build backup plans, ensure reliability
  • This task, needs customization → build your own system, take control

The essence of AI literacy isn't how many AI tools you can use — it's having a clear understanding of AI: knowing what it can do, what it can't do, how it should be used, and how it shouldn't.

With this understanding, AI is your tool. Without it, you might be AI's guinea pig.

So don't be in a hurry to learn more AI tricks. First take some time to figure out where AI's boundaries are.

Only when you know the boundaries can you truly use the tool well.

References: