The Ampere Porting Advisor is a fork of the Porting Advisor for Graviton, an open source project from AWS, which, in turn, is a fork of the Arm High Performance Computing group's Porting advisor. Originally, it was coded as a Python module that analyzed known incompatibilities for C and Fortran code. This tutorial walks you through building and using the tool and how to act on issues identified by the tool.

The Ampere Porting Advisor is a command line tool that analyzes source code for known code patterns and dependency libraries. It then generates a report with any incompatibilities with Ampere's processors. This tool provides suggestions of minimal required and/or recommended versions to run on Ampere processors for both language runtime and dependency libraries. It can be run on non-Arm64 based machines (like Intel and AMD) and Ampere processors are not required. This tool does not work on binaries, only source code. It does not make any code modifications, it doesn't make API level recommendations, nor does it send data back to Ampere.

Please Note: Even though we do our best to find known incompatibilities, we still recommend to perform the appropriate tests to your application on a system based on Ampere processors before going to production.

This tool scans all files in a source tree, regardless of whether they are included by the build system or not. As such, it may erroneously report issues in files that appear in the source tree but are excluded by the build system. Currently, the tool supports the following languages/dependencies:

Python 3+

• Python version
• PIP version
• Dependency versions in requirements.txt file

Java 8+

• Java version
• Dependency versions in pom.xml file
• JAR scanning for native method calls (requires JAVA to be installed)

Go 1.11+

• Go version
• Dependency versions on go.mod file

C, C++, Fortran

• Inline assembly with no corresponding aarch64 inline assembly.
• Assembly source files with no corresponding aarch64 assembly source files.
• Missing aarch64 architecture detection in autoconf config.guess scripts.
• Linking against libraries that are not available on the aarch64 architecture.
• Use of architecture specific intrinsic.
• Preprocessor errors that trigger when compiling on aarch64.
• Use of old Visual C++ runtime (Windows specific).
• The following types of issues are detected, but not reported by default:
  • Compiler specific code guarded by compiler specific pre-defined macros.
• The following types of cross-compile specific issues are detected, but not reported by default.
  • Architecture detection that depends on the host rather than the target.
  • Use of build artifacts in the build process.

For more information on how to modify issues reported, use the tool's built-in help: ./porting-advisor-linux-x86_64 -–help

If you run into any issues, see our CONTRIBUTING file in the project’s GitHub repository.

By using this option, you don't need to worry about Python or Java versions, or any other dependency that the tool needs. This is the quickest way to get started.

Pr-requisites

• Docker or containerd + nerdctl + buildkit

Run Container Image

After building the image, we can run the tool as a container. We use -v to mount a volume from our host machine to the container.

We can run it directly to console:

Copy
docker run --rm -v my/repo/path:/repo porting-advisor /repo 

Or generate a report:

Copy
docker run --rm -v my/repo/path:/repo -v my/output:/output porting-advisor /repo --output /output/report.html 

Windows example:

Copy
docker run --rm -v /c/Users/myuser/repo:/repo -v /c/Users/myuser/output:/output porting-advisor /repo --output /output/report.html 

Pr-requisites

• Python 3.10 or above (with PIP3 and venv module installed).
• (Optionally) Open JDK 17 (or above) and Maven 3.5 (or above) if you want to scan JAR files for native methods.
• Unzip and jq is required to run test cases.

Enable Python Environment

Linux/Mac:

Copy
python3 -m venv .venv 
source .venv/bin/activate 

Powershell:

Copy
python -m venv .venv 
.\.venv\Scripts\Activate.ps1 

Install requirements

• pip3 install -r requirements.txt

Run tool (console output)

• python3 src/porting-advisor.py ~/my/path/to/my/repo

Run tool (HTML report)

• python3 src/porting-advisor.py ~/my/path/to/my/repo --output report.html

Generating the Binary

Pre-requisites

• Python 3.10 or above (with PIP3 and venv module installed).
• (Optionally) Open JDK 17 (or above) and Maven 3.5 (or above) if you want the binary to be able to scan JAR files for native methods.

The build.sh script will generate a self-contained binary (for Linux/MacOS). It will be output to a folder called dist.

By default, it will generate a binary named like porting-advisor-linux-x86_64. You can customize generated filename by setting environment variable FILE_NAME.

Copy
./build.sh 

For Windows, the Build.ps1 will generate a folder with an EXE and all the files it requires to run.

Copy
.\Build.ps1 

Running the Binary

Pre-requisites

Once you have the binary generated, it will only require Java 11 Runtime (or above) if you want to scan JAR files for native methods. Otherwise, the file is self-contained and doesn't need Python to run.

Default behaviour, console output:

Copy
$ ./porting-advisor-linux-x86_64 ~/my/path/to/my/repo

Generating HTML report:

Copy
$ ./porting-advisor-linux-x86_64 ~/my/path/to/my/repo --output report.html

Generating a report of just dependencies (this creates an Excel file with just the dependencies we found on the repo, no suggestions provided):

Copy
$ ./porting-advisor-linux-x86_64 ~/my/path/to/my/repo --output dependencies.xlsx --output-format dependencies 

Here is an example of the output report generated with a sample project:

Copy
./dist/porting-advisor-linux-x86_64 ./sample-projects/ 
| Elapsed Time: 0:00:03 
 
Porting Advisor for Ampere Processor v1.0.0 
Report date: 2023-05-10 11:31:52 
 
13 files scanned. 
detected go code. min version 1.16 is required. version 1.18 or above is recommended. we detected that you have version 1.19. see https://github.com/AmpereComputing/ampere-porting-advisor/blob/main/doc/golang.md for more details. 
detected python code. if you need pip, version 19.3 or above is recommended. we detected that you have version 22.3.1 
detected python code. min version 3.7.5 is required. we detected that you have version 3.10.9. see https://github.com/AmpereComputing/ampere-porting-advisor/blob/main/doc/python.md for more details. 
./sample-projects/java-samples/pom.xml: dependency library: leveldbjni-all is not supported on Ampere processor. 
./sample-projects/java-samples/pom.xml: using dependency library snappy-java version 1.1.3. upgrade to at least version 1.1.4 
./sample-projects/java-samples/pom.xml: using dependency library zstd-jni version 1.1.0. upgrade to at least version 1.2.0 
./sample-projects/python-samples/incompatible/requirements.txt:3: using dependency library OpenBLAS version 0.3.16. upgrade to at least version 0.3.17 
detected go code. min version 1.16 is required. version 1.18 or above is recommended. we detected that you have version 1.19. see https://github.com/AmpereComputing/ampere-porting-advisor/blob/main/doc/golang.md for more details. 
./sample-projects/java-samples/pom.xml: using dependency library hadoop-lzo. this library requires a manual build  more info at: https://github.com/AmpereComputing/ampere-porting-advisor/blob/main/doc/java.md#building-jar-libraries-manually 
./sample-projects/python-samples/incompatible/requirements.txt:5: dependency library NumPy is present. min version 1.19.0 is required. 
detected java code. min version 8 is required. version 17 or above is recommended. see https://github.com/AmpereComputing/ampere-porting-advisor/blob/main/doc/java.md for more details. 
 
Use --output FILENAME.html to generate an HTML report. 

• In the report, we see several language runtimes (Python, pip, golang, Java) and their versions detected. All these messages communicate the minimum version and recommended version for these languages. Some of these lines detect that prerequisite versions have been found and are purely informative.

• We also see some messages from the dependencies detected in the Project Object Model (POM) or a Java project. These are dependencies that will be downloaded and used as part of a Maven build process, and we see three types of actionable messages:

Dependency requires more recent version

./sample-projects/java-samples/pom.xml: using dependency library snappy-java version 1.1.3. upgrade to at least version 1.1.4

• Messages of this type indicate that we should use a more recent version of the dependency, which will require rebuilding and validation of the project before continuing .

Dependency requires a manual build

./sample-projects/java-samples/pom.xml: using dependency library hadoop-lzo. this library requires a manual build more info at: https://github.com/AmpereComputing/ampere-porting-advisor/blob/main/doc/java.md#building-jar-libraries-manually

• In this case, a dependency does support the architecture, but for some reason (perhaps to test hardware features available and build an optimized version of the project for the target platform) the project must be manually rebuilt rather than relying on a pre-existing binary artifact

Dependency is not available on this architecture

./sample-projects/java-samples/pom.xml: dependency library: leveldbjni-all is not supported on Ampere processor.

• In this case, the project is specified as a dependency but is not available for the Ampere platform. In this case, an engineer may have to examine what is involved in making the code from the dependency compile correctly on the target platform. This process can be simple but may also take considerable time and effort. Alternatively, you can adapt your project to use an alternative package providing similar functionality which does support the Ampere architecture and modify your project’s code appropriately to use this alternative.

MEGAHIT is an NGS assembler tool available as a binary for x86_64. A customer wanted to run MEGAHIT on Arm64 as part of an architecture transition. But the compilation failed on Arm64 in the first file:

The developer wanted to know what needed to be changed to make MEGAHIT compile correctly on Arm64.

In this case, Ampere Porting Advisor (APA) can play a key role. After scanning the source repository of the MEGAHIT project with APA, we get a list of issues that need to be checked before rebuilding MEGAHIT on Arm64:

Let’s investigate each error type in the list and correct them for Arm64 if necessary.

Architecture-specific build options

These errors will be triggered once APA detected build options not valid on Arm64.

The original CMakeList.txt is using x86_64 compile flags by default without checking CPU Architectures. To fix this, we can test a CMAKE_SYSTEM_PROCESSOR condition to make sure the flags reported by APA will be only applied to x86_64 architectures.

Architecture specific instructions

The architecture specific instructions error will be triggered once APA detected non-Arm64 C-style functions being used in the code. Intrinsic instructions are compiled by the compiler directly into platform-specific assembly code, and typically each platform will have their own set of intrinsics and assembly code instructions optimized for that platform.

In this case, we can make the use of pre-processor conditionals to only compile the _pdep_u32/64 and __cpuid/ex instructions when #if defined(x86_64) is true for the HasPopcnt() and HasBmi2() functions. For vec_vsx_ld, it is already wrapped in a pre-processor conditional, and will only be compiled on Power PC architecture, so we can leave it as is.

Architecture specific inline assembly

The architecture specific instructions error will be triggered once APA detected assembly code being used in the code. We need to check whether the snippet of assembly code is for Arm64 or not.

The MEGAHIT project only uses the bswap assembly code in phmap_bits.h when it is being compiled on the x86_64 architecture. When being compiled on other architectures, it compiles a fall-back implementation from glibc. So no changes are required in phmap_bits.h. In cpu_dispatch.h,two inline functions HasPopcnt() and HasBmi2() unconditionally include the x86_64 assembly instruction cpuid to test for CPU features on x86_64. We can add a precompiler conditional flag #if defined(x86_64) to make sure this code is not called on Arm64, and we will always return false.

Architecture specific SIMD intrinsic

The architecture specific instructions error will be triggered once APA detected x86_64 SIMD instructions like AVX256 or AVX512 being used in the code. These SIMD instructions are wrapped by precompiler conditional flags and will usually not cause any functionality issue on Arm64. If there were no SIMD implementation of the algorithm for Arm64, there could be a performance gap compared to x86_64. In this case, there is a NEON SIMD implementation for Arm64 in xxh3.h and this implementation will be cherry picked by the compiler based on the CPU architecture. No further actions need to be taken.

Preprocessor error on AArch64

The preprocessor error will be triggered by APA to indicate that the Arm64 architecture may not be included in a pre-compile stage. In this case, we can see that the pre-compile conditional is for x86_64 only and does not concern the Arm64 architecture.

Rebuild and test

Once all these adjustments have been made, we could rebuild the project:

The project compiled successfully. We then checked whether it passed the project’s test suite:

After we have manually checked and fixed all the potential pitfalls reported by APA, MEGAHIT is now able to build and run on Ampere processors.