Test Notes for Big Data Workloads
Scalability Tests were performed on a 9 node cluster of bare metal nodes.
- CPU - Single socket per server with Ampere Altra N1 processor with 80 cores @3.0GHz
- Memory - Samsung DDR4 512GB DRAM 3200 MHz
- Storage - 4 Micron 7300 NVME Drives each with 3.5 TB storage space
- Network Cards - Mellanox CX-6, 2 x 100Gb/s with a bonded configuration
Terasort Hadoop and Spark tests were done on Ampere Altra and x86 VM's with the following configurations.
- Intel - Skylake 2000 GHz
- AMD - Rome 2249 MHz
- AMD - Milan 2449 MHz
- Ampere - Altra processors 3000 MHz
- vCPU's - 16 ARM Cores or 16 x86 threads.
- Memory - 64 GB
- Storage - 1TB storage with 1000 MB/sec read/write performance
Note: TPC-DS tests were done on Intel Ice Lake processors(Graph 2).
Few yarn and spark configuration details used on VM's listed below:
| dfs.block.size | 512M |
| yarn.scheduler.maximum-allocation-mb | 40g |
| yarn.scheduler.maximum-allocation-vcores | 15 |
| yarn.nodemanager.resource.cpu-vcores | 16 |
| yarn.nodemanager.resource.memory-mb | 64g |
| mapreduce.map.memory.mb | 1024 |
| mapreduce.reduce.memory.mb | 3072 |
| mapred.reduce.parallel.copies | 16 |
| mapreduce.reduce.shuffle.parallelcopies | 14 |
| mapreduce.map.java.opts | Xmx2048m |
| spark.master | yarn |
| spark.executor.cores | 5 |
| spark.executor.memory | 18g |
| spark.executor.instances | 3 |
Created At : May 15th 2022, 7:14:43 pm
Last Updated At : April 25th 2023, 5:04:17 pm
Ampere Computing
4655 Great America Parkway
Suite 601 Santa Clara, CA 95054
© 2023 Ampere Computing LLC. All rights reserved. Ampere, Altra and the A and Ampere logos are registered trademarks or trademarks of Ampere Computing.