At the Milwaukee School of Engineering, a single AmpereOne® Arm64 server is doing something that would cost a small fortune in cloud computing: giving every pair of students in two concurrent graduate classes their own dedicated virtual machine to run full MLOps pipelines, all semester long, at a fixed cost.

How is a modern engineering school able to train a generation of students in AI and Machine Learning, in a world where AI hardware is increasing in price? Given the choice between using cloud services with operating expenses tied to consumption, or acquiring hardware to host on premise, many universities prefer to host their own hardware.

“There are operating expenses and there are capital expenses. A university’s budget is all CapEx,” remarked Dr. R. J. Nowling, machine learning program director at Milwaukee School of Engineering (MSOE). “So, the idea of buying cloud compute to be able to run these classes doesn’t really fit the university budgeting model.”

With the cost of state-of-the-art hardware to run AI workloads increasing, this presents a challenge to schools. How can you provide enough Compute resources to students to allow them to learn about how machine learning works in industry, without breaking the bank? How can you do so in a small enough space to serve the needs of an industry-leading master’s program, without finding space, power, and cooling infrastructure for racks of servers?

A four-year university’s primary source of revenue is its periodic tuition. So, it can’t readily procure virtual cloud infrastructure on a pay-as-you-go scheme — especially not the wide tranches of data, memory, compute, and bandwidth needed to stage MLOps environments for students.

“While some cloud providers do provide free credits for students,” continued Dr. Nowling, “that’s for the individual student, not for the instructor running a class. I wouldn’t be able to oversee all their deployments in a controlled environment; they’d have completely separate accounts.”

A modern university depends, to a large extent, upon generosity. Nowling developed and teaches the ML Production Systems course at MSOE’s Diercks’ Computational Science Hall. MSOE’s GPU cluster — dubbed “Rosie,” named for the women who were the first computing machine operators during World War II — is now based around twin, donated Nvidia DGH H100 clusters.

However, as is the case in most universities, clusters with large compute infrastructure like this are primarily used for research purposes, not for conducting classes. For undergraduate and graduate classes with many students but lower resource demand it is practical to consider other vehicles for teaching infrastructure.

Not all AI and machine learning workloads resemble generative AI models in their computational structure or resource requirements. Recommendation engines, ranking models, and computer vision applications are all examples of practical ML systems that operate under different constraints, often with smaller datasets and lower resource demands. While data in these systems is still represented in vectorized forms (for example with one-hot encoding or embedded representations), their underlying algorithms may rely on data structures that allow CPU-based inference capable of providing better efficiency and price/performance.

As MLOps emerges from the realm of research institutions and development laboratories to become an everyday skill taught, the schools that have taken up the task of teaching these students find themselves faced with a unique predicament: Acquiring the platforms best suited for practice deployments of ML models is nowhere near as easy for a small college or university as for a commercial enterprise.

“All these programs focus on developing super-sophisticated models, including ours,” remarked Dr. Nowling. “Where our program is different is our secondary focus, which is actually deploying those models and operating them. These models are useless if they’re just sitting on your laptop or on your cluster and not actually being integrated into software and products that serve users, customers.”

It’s not really teaching MLOps, Nowling believes, unless it’s demonstrating the “Ops” portion of the concept: the “outer loop,” as MLOps engineers refer to it. This is the part where models are deployed, managed, and observed for performance. Deployment requires a platform capable of hosting that deployment, which means that schools teaching the full MLOps lifecycle should support the full infrastructure that the entire lifecycle would require.

What Dr. Nowling needed was a way to present students with an MLOps environment that was representative of what they would see in production environments, that they could own and manage throughout the course. To accomplish this, he provided each pair of students with an 8-core virtual machine, a virtual environment in which students could deploy and manage applications using Docker, run a PostgreSQL database, and in which they could run their Machine Learning algorithms using standard Python frameworks. Nothing about this environment is specific to any processor architecture — x86 or Arm64. However, to offer this MLOps environment to every student in his class with the limited space in his lab, Dr. Nowling needed a lot of CPU cores. As a result, the environments that he built for his students were staged on a single Ampere Arm64 server.

“Without the Ampere server, there would be no way for us to provide that experience to our students,” said Dr. Nowling. “Ampere allows us to run a local virtual machine solution on premise, that our students can access remotely.”

An Ampere server makes it possible for courses such as Nowling’s, and the others taught, to stage an ample number of virtual environments for building simulated MLOps pipelines. “We use this server to give students virtual machines they can use to deploy the software they write over the course of the semester,” said Nowling. “By being able to deploy it on these [VMs], they’re able to keep it running consistently, and get that experience of operating a production system.

“The main benefit for us,” he continued, “is that the Ampere platform is so well supported by existing open-source software that we are able to seamlessly use the Ampere servers in place of Intel- or AMD-based servers.  We can use and manage our Ampere servers just like any of the other servers we have. The Ampere platform is much less costly while offering superior density.  Having fewer nodes to serve the same number of VMs means we save quite a bit of money on RAM and NVMe drives for storage. . . Ampere allows us to run a virtual machine solution on premise that our students can access, running whatever flavor of Linux we need, while having great control over costs because the costs are fixed.”

According to Dr. Nowling, the industry impact of the master’s program has been significant. Milwaukee is home to a large number of traditional industrial companies - “they are not the East Coast and West Coast companies that are pushing the bleeding edge - they are adopting all of this and want to integrate machine learning into all of their products and services”. These companies have a real appreciation for the practical approach and skills that are taught as part of the course - “our undergrads have a greater than 95% graduation outcome in terms of being hired” - many of these employers then pay for those recent graduates to complete the Master’s program because they can see the value in these skills. In addition, engineering leaders from these companies are completing the master’s course as continuing education for themselves, to enable them to lead innovative efforts at their company.

By using an Ampere server to deliver long-lived, real-world MLOps deployments, the master’s program is also exposing students to Arm64 - giving students the ability to bring that knowledge back into industry and realize cost, power, and performance benefits in their workplaces by using Arm instances for production Machine Learning workloads.

“We’re making sure our students understand what it means to actually deploy this stuff,” Nowling stated. “So, we’re reducing the knowledge gap so that these companies can be more successful in their AI journeys.”