Ampere Computing Logo
Contact Sales
Ampere Computing Logo
Hero Image

LLM Inference with Ampere-based OCI A1

> Oracle Partner Page > AI Solutions   > AI Developer Center  

LLM Inference on OCI

Meet Your Performance Needs While Minimizing TCO

Ampere Cloud Native Processors with Ampere Optimized AI Frameworks are uniquely positioned to offer GPU-Free AI Inference at performance levels that meet client needs of all AI functions be it generative AI, NLP, recommender engines, or computer vision.

Choosing CPUs for Efficient Generative AI Deployments

Choosing CPUs for Efficient Generative AI Deployments

Democratizing Generative AI with CPU-based Inference

Democratizing Generative AI with CPU-based Inference

Introducing Meta Llama 3 on OCI Ampere A1

Introducing Meta Llama 3 on OCI Ampere A1

Serge Chat

This demo shows that the Ampere-developed chatbot called Serge running Llama 2 7B on Ampere-based OCI A1 matches the user experience provided by ChatGPT 3.5 based on the 3.5 GPT model. Serge, a simple chatbot made solely for showcase purposes, rivals the performance and the quality of output provided by ChatGPT 3.5 while running GPU-Free on efficient and scalable Ampere-based OCI A1 cloud instances.


Deploy On OCI

Access OCI Marketplace Listing

Try OCI Free

Ampere Optimized llama.ccp

Docker Hub

This Docker image can be run on bare metal Ampere® CPUs and Ampere® based VMs available in the cloud. 

> Docker Hub

GitHub

Release notes and binary executables are available on our GitHub

GitHub

Resources

Developer Resource

RAG Examples with Vector Embeddings

Ampere

Developer Resource

Python bindings for llama.cpp

Ampere

Connect with your peers and
get the latest tips, trends, and tools

Created At : June 14th 2024, 6:12:58 pm
Last Updated At : July 9th 2024, 7:04:32 pm
Ampere Logo

Ampere Computing LLC

4655 Great America Parkway Suite 601

Santa Clara, CA 95054

image
image
image
image
image
 |  |  |  |  |  | 
© 2024 Ampere Computing LLC. All rights reserved. Ampere, Altra and the A and Ampere logos are registered trademarks or trademarks of Ampere Computing.
This site is running on Ampere Altra Processors.