Distributed memory is an architecture where each processor or node owns private memory, and nodes communicate by sending messages over a network. It contrasts with shared-memory systems where processors access a common RAM space.

Common models and communication patterns:

• Message Passing Interface (MPI): explicit message-passing in high performance computing.
• Partitioned Global Address Space (PGAS): provides a a global address abstraction over distributed storage.
• Client-server and microservices patterns: Application-level distributed memory across services.
• Sharding and distributed caches: partitioned data with replication for throughput and locality.

Distributed memory enables systems to scale beyond a single node’s memory and compute limits, supporting. They are essential for large-scale simulations, analytics, and cloud-native services that require horizontal scale and resilience.

Key advantages:

• Linear scalability by adding nodes to increase capacity.
• Failure isolation and elasticity for resilient deployments.
• Ability to handle datasets larger than a single machine’s memory.
• Cost effective by using many commodity servers instead of one very large node.

• Distributed Memory: Wikipedia
• Message Passing Interface Tutorial