Ampere AI
Ampere Optimized Frameworks
Ampere Model Library (AML)
AI Solutions
Solutions for AI on Ampere Altra
Ampere AI delivers world class AI Inference solutions! Ampere Optimized AI delivers a significant inference performance benefit to any existing model that runs on our supported frameworks out of the box. Ampere AI currently supports the following frameworks available for free download here or at some of our supporting partners:
- TensorFlow
- PyTorch
- ONNX
Ampere hardware supports native FP16 data format providing nearly 2X speedup over FP32 with almost no accuracy loss for most AI models.
Ampere provides easy-to-use Docker containers that include Computer Vision and Natural Language Processing model examples and benchmarks that enable developers to get started quickly. Download one of our Docker containers today to experience the benefits of our best-in-class performance. Read more about our solutions below.
Key Benefits
Ampere AI optimized frameworks + Ampere Altra Max deliver disruptive value for MLPerf and more:
- Predictable Performance: Up to 5X Higher Throughput than AWS Graviton using FP16!
- Predictable Performance: More than 2X Higher Throughput than x86 competition using FP16
System configurations, components, software versions, and testing environments that differ from those used in Ampere’s tests may result in different measurements than those obtained by Ampere. The system configurations and components used in our testing are detailed here
Downloads
Ampere Optimized AI Software
Ampere Altra and Ampere Altra Max, with high performance Ampere optimized frameworks, offers the best-in-class Artificial Intelligence inference performance for frameworks including Tensorflow, PyTorch and ONNX Runtime. Ampere Model Library (AML) offers pretrained models to help accelerate AI development.
How It Works
Ampere Optimized Framework Components
With an inference engine, Ampere optimized frameworks offer significant benefits. Click here to view the demo.
Ampere helps customers achieve superior performance for AI workloads by integrating optimized inference layers into common AI frameworks.
This seamless integration to any AI framework accelerates inference without any accuracy loss, conversions, or model retraining. The architecture is diagrammed in the figure above. The main components are as follows:
- Framework Integration Layer: Provides full compatibility with popular developer frameworks. Software works with the trained networks “as is”. No conversions or approximations are required.
- Model Optimization Layer: Implements techniques such as structural network enhancements, changes to the processing order for efficiency, and data flow optimizations, without accuracy degradation.
- Hardware Acceleration Layer: Includes a “just-in-time”, optimization compiler that utilizes a small number of Microkernels optimized for Ampere processors. This approach allows the inference engine to deliver high-performance and support multiple frameworks.
All model types are supported
- Object detection
- Image classification
- Recommendation engines
- Natural language processing
Key benefits
- Up to 5x in performance acceleration
- Multi Framework support
- Seamless “out-of-the-box” deployment
- Native support of FP16 data format
- No loss in accuracy
AML is a collection of optimized AI models pretrained on standard datasets. The library contains scripts running the most common AI tasks. The models are available for Ampere customers to quickly and seamlessly build into their applications.
AML Benefits Include:
-
Benchmarking AI architecture with different frameworks
-
Accuracy testing of AI models on application-specific data
-
Comparison of AI architectures
-
Conducting tests on AI architectures
FAQs
Ampere AI FAQ
Resources
Tutorials
Documentations
Publications
TESTING AND REGRESSION
Solutions and Regression Testing
Frameworks
Regression of the currently available Ampere Optimized AI images
