Why Universities Need to Build Local AI Experiment Environments: From Online APIs to Autonomy and Control ​

We · September 2026

This article is the second piece in the "Generative AI Education Trends" series. The first piece, "AI Education Is Being Rewritten in the Wave of Generative AI," discussed the overall changes in curricula, tools, and deployment. This piece focuses on one core question: why universities must build AI experiment environments locally, and how to do it.

In the first piece of the series, We briefly mentioned that online AI services have clear limitations in teaching scenarios, and that local deployment is a "must-have" for generative AI education. But this conclusion is too important to gloss over in a single sentence—it directly determines the direction of university AI teaching infrastructure.

This article answers two questions in detail: Why don't online services work? And how should universities build?

The Four Fundamental Flaws of Online AI Services ​

Many people ask a natural question: Can't students just use online AI services like ChatGPT, Midjourney, and Claude directly? Why should schools spend money building local environments?

We believe this question deserves a serious answer—because if online services could truly meet teaching needs, then the construction costs and deployment difficulties discussed below would be meaningless. But the reality is that online services have four fundamental flaws in teaching scenarios, and each one points to the same conclusion: universities must build local AI experiment environments.

First, Uncontrollable Costs ​

Online services charge per API call or by subscription. Each call looks cheap, but in a teaching scenario, costs accumulate quickly.

Let's do the math: A class of 40 students, each calling the LLM API 50 times a day (for projects, prompt tuning, running tests), over a 15-week semester with 5 days per week, totals 40 × 50 × 15 × 5 = 150,000 calls. At mainstream API pricing (around $0.01–0.03 per call), one class for one semester costs $1,500–4,500. If a school has 5 classes running AI courses simultaneously, that's nearly $10,000 per semester.

And this money isn't a one-time expense—every semester, the same cost recurs, and when students graduate, nothing remains. More critically, once students get used to free tiers or school-paid APIs, any budget cut, price hike, or free-tier reduction immediately disrupts the course. We believe a teaching environment cannot be built on a third-party service that could raise prices at any time—this isn't a cost issue, it's a teaching continuity issue.

Second, Data Privacy and Compliance ​

Much of the data used in teaching and research cannot be sent to third-party servers.

• Students doing capstone projects may work with anonymized data provided by companies

• Finance students building RAG projects use real research reports and financial data

• Medical students doing AI imaging analysis work with patient CT/MRI scans

• Design students doing AI creation may work with companies' unreleased design drafts

Under China's Data Security Law, Personal Information Protection Law (PIPL), and the EU's GDPR, this data cannot simply be sent to OpenAI or Google servers. If a course depends on online services, it's equivalent to telling students "just send the data to a U.S. company"—which in many scenarios is not only unworkable but potentially illegal.

Shanghai University of Finance and Economics' Dishui Lake Advanced Institute of Finance MF program offers courses in machine learning, large language models, and Python for financial applications in its AI finance track. Financial data is inherently sensitive, making local deployment almost the only option. Academia has dedicated research on financial RAG, designing modular system architectures that support private knowledge bases and compliance constraints. The banking industry has already begun using RAG systems to build intelligent customer service, combining LLM reasoning capabilities with banks' internal private knowledge bases.

Third, Students Only Learn to "Use," Not to "Build" ​

We believe this is the most fundamental problem.

Online services are black boxes—students call an API, send text, and get a response back. But what is the core goal of a generative AI course? It's to help students understand how models work, how to fine-tune them, how to build RAG systems, how to construct Agents, and how to quantize models.

None of these skills can be learned by calling APIs. Calling an API 100 times won't teach you what Transformer's attention mechanism is; using ChatGPT 100 times won't teach you how to do LoRA fine-tuning on a 7B model.

Traditional machine learning teaching is different—students train a model from scratch and can see how data flows in, how parameters update, how loss decreases, and how evaluation metrics change. If generative AI wants students to reach the same depth of understanding, they must run models locally, tweak parameters, and inspect intermediate results. Teaching with API calls is like teaching students only how to order takeout, not how to cook.

Nanjing Institute of Technology's "Introduction to Large Language Models" course uses Ollama for local practice, and Guangzhou Software College uses LangChain for AI-native application development. These courses couldn't possibly meet their teaching goals with APIs alone.

Fourth, Services Can Shut Down or Change at Any Time ​

The AI industry changes extremely fast. A free service today may charge tomorrow; a model available today may be pulled tomorrow; an open API today may be rate-limited tomorrow.

This isn't hypothetical—multiple similar incidents have already occurred: online AI services adjusting pricing strategies, tightening free tiers, and changing model interfaces. If a course fully depends on a particular online service, a single feature change can invalidate the entire teaching plan. Students' learning materials, code, and projects all have to be revised accordingly.

Teaching requires stability and controllability—something online services inherently cannot provide.

Summary ​

Our conclusion is clear: Local deployment isn't "icing on the cake"—it's the "infrastructure" of generative AI teaching. Just as chemistry departments need labs and biology departments need microscopes, AI courses need a local environment that can run models.

So How Should Universities Build? ​

Having established that "we must build," the next question is "how to build." In this section, We want to discuss this from three layers—hardware, software, and management—drawing on real case studies from around the world.

Hardware: More Expensive Isn't Better ​

When people hear "build an AI lab," their first reaction is often "buy GPUs." But there's a common pitfall in hardware investment: emphasizing procurement while neglecting utilization. Buying a pile of high-end GPUs without supporting software environments and management systems means they can't actually be used—which is a waste.

We believe hardware planning should be layered according to teaching scenarios:

Key principle: Not every course needs top-tier hardware. A lab with 20 RTX 3060 machines running quantized 7B LLMs and FLUX 4B image models can already cover most undergraduate AI teaching needs. For non-AI majors like design, media, and architecture, this configuration is more than sufficient.

Software: This Is the Real Threshold ​

Once hardware is purchased, the software environment becomes the bigger challenge. In the first piece, We covered the "six layers of hurdles" (hardware, model selection, weight downloads, dependency configuration, quantization optimization, and management/maintenance). Here, rather than rehashing the pain points, We'll focus on how to solve them.

Approach 1: Choose appropriate quantized models to match existing hardware.

This is a point We want to emphasize. Many schools want to run original models (FP16 precision) right from the start, but a 14B original model requires 28GB of VRAM—beyond what most schools' machines can handle. Switching to an INT4 quantized version needs only about 7GB, which a single RTX 3060 can run. Black Forest Labs' FLUX.2 [klein] 4B version requires only 8.4GB of VRAM, with quality sufficient for teaching, and uses the Apache 2.0 open-source license.

The goal of teaching is to help students understand principles, master workflows, and produce results—not to chase inference quality identical to the paper. Deploying appropriate quantized models based on actual hardware and course needs is the more pragmatic and sustainable approach.

Approach 2: Use containerization or image solutions to unify environments.

In the traditional ML era, installing Python packages on each machine was enough. But generative AI environments are far more complex—ComfyUI requires specific Python, CUDA, and PyTorch versions, plus various custom node dependencies. If you install manually one machine at a time, 40 machines could take a week, and each machine's configuration might end up slightly different.

A more reasonable approach: configure the environment on one machine, package it as a Docker image or system image, then deploy it in bulk to other machines. This keeps all machines' environments consistent and makes troubleshooting easier—instead of "what's wrong with this machine," it becomes "how does it differ from the standard image."

Approach 3: Centralized model distribution.

Generative AI model weights often range from several GB to tens of GB. If 40 machines each download from Hugging Face independently, it's not only slow but also prone to download failures due to network issues.

A more reasonable approach: download all needed models on one server to serve as an internal model repository. Other machines pull from the internal network, which is fast, stable, and controllable. When a new model comes out, you only need to update one copy on the server, and all machines sync at once.

Management: Long-Term Operation Is the Real Test ​

Getting the environment set up isn't the end—it's just the beginning. We've observed that many schools' AI labs go through this cycle: built → usable → unmaintained → environment breaks → nobody can fix it → rebuilt from scratch.

Long-term operation of a lab environment requires solving at least these problems:

Version management — Models and tools update quickly. ComfyUI 0.3 installed today may be superseded by 0.4 next month. The school's standard environment needs regular updates, but updates can also introduce compatibility issues. A "test first, then roll out" update workflow is needed.

Failure recovery — Students using machines may accidentally delete dependencies, change configurations, or install conflicting packages—all of these happen. There need to be fast recovery mechanisms—such as image restoration or snapshot rollback—so machines can return to a usable state within minutes.

Multi-discipline adaptation — Different majors have different needs for AI environments. Design departments want ComfyUI + Stable Diffusion; finance departments want Ollama + LangChain + RAG; media departments want video generation tools. A lab can't just configure one environment—it needs to flexibly switch based on the course.

Case Studies from Around the World ​

Top Universities: Building AI Computing Platforms ​

Shanghai Jiao Tong University established its AI college in April 2024 and simultaneously built the "Zhiyuan No. 1" AI computing platform—one of the large Chinese-made AI computing platforms at a Chinese university. The university offers 357 AI-related courses, all of which rely on this platform for teaching and experiments.

Beijing Institute of Technology jointly released the world's first AI Practice LAB (AIPL) showcase site with Huawei in March 2026. This is a case of industry-education integration—the school provides the teaching needs, Huawei provides the technology and equipment, and together they build the practice environment.

Zhejiang University established its AI college in 2025, building on two existing national-level platforms: the National AI Industry-Education Integration Innovation Platform and the National AI Textbook Base.

These top universities build large-scale AI computing platforms—they have funding, technical teams, and corporate partnerships. But We want to point out that this model is not replicable for ordinary universities. Not every school can build an AI computing platform or partner with Huawei.

Ordinary Universities and Vocational Colleges: Need More Pragmatic Solutions ​

China added 91 new AI major programs in 2024, of which 85 were at non-"Double First-Class" universities. The reality for these schools is:

• Limited budgets—buying A100 clusters is out of the question

• Small technical teams—may not have dedicated AI environment operations staff

• Uneven student foundations—many non-computer science students have no programming background

• Many cross-disciplinary needs—not just computer science, but design, media, finance, and medicine all need it

We believe these schools don't need "AI computing platforms" but rather AI teaching environments that are "sufficient, manageable, and updatable":

• Hardware: consumer-grade GPUs (RTX 3060/4060/4070) can already run quantized models

• Software: a solution that enables fast deployment, unified management, and one-click updates

• Models: use quantized versions rather than originals, matching existing hardware

• Curriculum: flexibly configure tool combinations based on major requirements

Global Perspective ​

It's not just Chinese universities facing this problem. Globally, the "last mile" of AI education—from "having hardware" to "being able to use it"—is a universal pain point.

UNESCO's AI competency framework for teachers and European research reports on AI adoption in vocational education both discuss the challenges educational institutions face in AI deployment: insufficient hardware, inadequate teacher training, and curricula that can't keep up. Some research directly points out that the resource-intensive nature of generative AI models creates deployment barriers for resource-constrained institutions.

Researchers at Oklahoma Christian University noted in a paper that LLM technology evolves so quickly that course materials need frequent updates, and instructors need to restructure courses around stable foundational concepts. This problem is the same for all schools—no matter what country or what level of institution, "how to build the environment, how to maintain it, and how to keep up with material updates" is unavoidable.

Policy Is Pushing, but Implementation Needs Methods ​

On the policy front, the push has already been substantial.

Five government bodies including the Ministry of Education jointly issued the "AI + Education" Action Plan, promoting the upgrading and transformation of traditional vocational majors, scientifically designing an "AI +" professional system, curriculum system, and teaching system, and implementing a cluster training plan for highly skilled talent in the AI field.

Data from the "Vocational Education Industry Development Report 2025" shows that nearly half of vocational colleges have begun responding to regional AI-related policies, and nearly 40% have formed clear AI application development plans.

We believe that policy has driven the willingness to "build," but the methodology for "how to build" is still being explored. Many schools have bought GPUs and built labs, but can't set up the software environment or maintain it, leading to low hardware utilization—this is the most common dilemma today.

Our View ​

After reviewing these cases and data, We want to summarize three core viewpoints:

First, a local AI experiment environment isn't an "optional configuration"—it's the "infrastructure" of generative AI teaching. The costs, privacy issues, black-box nature, and instability of online services dictate that universities must go the local deployment route. This isn't an individual choice by a particular school but a common trend in global AI education.

Second, building an environment doesn't mean building an AI computing platform. Top universities have large budgets, large teams, and large partnerships, and can build large-scale computing platforms. But ordinary universities and vocational colleges need lighter, more pragmatic solutions—consumer-grade GPUs + quantized models + unified management tools. A lab equipped with RTX 3060 machines running quantized 7B LLMs and FLUX 4B image models can already cover most teaching needs.

Third, the core of environment building isn't hardware—it's software and management. Buying GPUs doesn't mean you can use them. How to select models, configure environments, manage versions, and recover from failures—these are the real challenges of long-term operation. The market already offers solutions that support rapid deployment of original, quantized, and accelerated versions of common models. The value of such tools lies in: schools don't need to research how to quantize each model or configure its dependencies themselves—they just select the appropriate version based on hardware conditions and can quickly set up a usable teaching environment.

The policy tailwinds for AI education have arrived, and schools have the willingness to build. The next step is solving "how to build quickly, manage well, and make it usable"—so that environment setup is no longer a prerequisite hurdle for practical courses, but instead lets students spend their time where it truly matters: training models, validating ideas, and iterating on solutions.