Virtual machines were provisioned in a private network space as depicted above. Hadoop 3.3.1 (aarch64 binaries) and Spark 3.1.2 were installed on the test bed.

• A single VM was spun across each of the architectures with the configuration as outlined in the table.
• All the virtual machines had identical configurations of CPU cores/threads, memory and storage.
• The storage bandwidth was limited to 1000 MB/s across all the VM’s. The maximum bandwidth for an x86 VM with 8 OCPU’s is 8 Gb/s in OCI. An A1 instance with 16 OCPU’s receives a max bandwidth of 16 Gb/s. The A1 Instance was throttled to 8 Gb/s in our benchmark, to keep it at par with x86 VM’s. -Very few changes like disabling transparent huge pages and, reducing VM swappiness were altered on the guest Operating system.
• Few configuration parameters in Spark were tuned to maximize the utilization of CPU, memory and storage.
• Oracle JDK8 EPP was used on the test bed. This patch 34375301 can be downloaded from Oracle support site. JDK 17 improvements are added to Oracle JDK 8 EPP (enterprise performance pack).

VM and Spark on Yarn Configuration

Spark and Yarn Configuration

Spark TeraSort

The TeraSort workload sorts 100-byte records generated by the TeraGen program contained in the Hadoop distribution. The Intel HiBench benchmark tool was used on each of the VM’s to generate a 250GB dataset. Spark TeraSort benchmark was run on these VM’s and the TeraSort output in MB/s was captured.

Join Operations

The following scala query was executed multiple times in Spark shell to capture the time taken to complete the join query.

Copy
val df = sc.makeRDD(1 to 10000000, 7).toDF 

val df2 = sc.makeRDD(1 to 10000000, 7).toDF 

df.select( $"value" as "a").join(df2.select($"value" as "b"), $"a" === $"b").count 

Word Count

Word Count programs are representative of a large subset of real-world MapReduce jobs, one transforming data from one representation to another and another extracting a small amount of interesting data from a large data set. Uploaded 10GB of text file to HDFS and then attempted to count the words in the file.

TPC-DS

TPC-DS is a decision support benchmark that models several aspects of a decision support system. The tpcds-kit was cloned from databricks github site. Spark 3.2 was run in yarn mode with a scale factor of 250 and in parquet format. The time taken to execute all the 99 sql statements were captured.

The relative performance data captured on OCI test bed with Spark on Yarn is shown below.

The relative price per performance data captured with Spark TeraSort is shown below. Similar graphs for other benchmarks and tests like Wordcount and Join operations can be plotted by taking the prices from compute and storage pricing.

1. After tuning the parameters, CPU was fully utilized and was hovering around 85-90% making this a fair comparison under high load conditions. 2. A1 VM’s performed well compared to its x86 peers. The performance graphs were plotted by taking s3flex as the baseline reference point. 3. Ampere A1 instances price performance was observed to be 50% better than Intel and 30% better over AMD shapes.

Note: Price-performance was calculated from OCI Compute pricing list, for 16 core VM’s and 96G Memory (Oct 2022). Storage Costs were calculated from OCI Storage pricing sheet for 2x500GB iSCSI luns at 50 VPU ( 480 MB/s).

Oracle OCI A1 instances with Ampere Altra processors provide high performance for big data solutions like Spark. The performance advantage on the Ampere instances combined with the price advantage provides up to 50% higher value when using OCI Ampere A1 instances for Spark TeraSort and TPCDS workloads.

For More Information

Oracle Ampere A1 compute instances

Ampere solutions on Big Data

All data and information contained herein is for informational purposes only and Ampere reserves the right to change it without notice. This document may contain technical inaccuracies, omissions and typographical errors, and Ampere is under no obligation to update or correct this information. Ampere makes no representations or warranties of any kind, including but not limited to express or implied guarantees of noninfringement, merchantability, or fitness for a particular purpose, and assumes no liability of any kind. All information is provided “AS IS.” This document is not an offer or a binding commitment by Ampere. Use of the products contemplated herein requires the subsequent negotiation and execution of a definitive agreement or is subject to Ampere’s Terms and Conditions for the Sale of Goods.

System configurations, components, software versions, and testing environments that differ from those used in Ampere’s tests may result in different measurements than those obtained by Ampere.

Price performance was calculated from OCI Compute pricing list, for A1 Flex VMs in March of 2022. Refer to individual tests for core counts. Memory and Storage is same across all the VM’s, and hence not considered

©2022 Ampere Computing. All Rights Reserved. Ampere, Ampere Computing, Altra and the ‘A’ logo are all registered trademarks or trademarks of Ampere Computing. Arm is a registered trademark of Arm Limited (or its subsidiaries). All other product names used in this publication are for identification purposes only and may be trademarks of their respective companies.

Ampere Computing^® / 4655 Great America Parkway, Suite 601 / Santa Clara, CA 95054 / amperecomputing.com