Why not just use a CPU

Modern science and engineering uses computing for a vast array of tasks. Some are basic such as text or image processing, while others are more specific to solving difficult math and physics equations using numerical methods¹. For much of computing the central processing unit (CPU) was the most important component for handling arithmetic, logic, controlling, and input/output (I/O) operations that are used to solve computational problems. The reason is tha the CPU is a very general purpose device and is very flexible in use. However, this comes with a downside: operations that might be simple to execute could actually be quite time consuming on a CPU.

So what is there to compare to, in other words, what other type of computing device could be used? A processing device that is tailored for specific type of operations. There are application specific integrated circuits(ASICs) which are gate arrays of transistors that can be programmed to behave in a specific manner that is enhanced for a specific type of application or task. Another is the Graphical Processing Unit (GPU) which is what I want to talk about today.

Ancient 🗞️

I'm writing this post because I wanted to capture the difference between GPUs and CPUs, but this post is probably 10 years too late, in other words, many have written post like this before.

The GPU vs. CPU

The fundamental difference between GPUs and CPUs is in their architecture and design philosophy [1-2]. The CPU is considered "general purpose" and is optimized for sequential processing. They typically have a few physical cores (typically 4-16 in consumer models) that are highly sophisticated, capable of handling a wide variety of operations efficiently.

The GPU, on the other hand, was originally designed for rendering computer graphics, which involves performing many similar calculations simultaneously [1-2]. This led to an architecture with hundreds or thousands of simpler cores, each capable of executing the same instruction on different data points simultaneously. This approach, known as Single Instruction, Multiple Data (SIMD), makes GPUs extraordinarily efficient at parallel processing [1].

Why can GPUs be better for Scientific Computing?

We can go through various points to understand why a GPU is better for many scientific² computing applications:

1. Parallel Processing: Many scientific algorithms, such as matrix operations, Monte Carlo simulations, and numerical integrations, can be parallelized. GPUs excel at these tasks, potentially offering orders of magnitude speedup compared to CPUs.

2. Memory Bandwidth: GPUs typically have much higher memory bandwidth than CPUs. This is crucial for data-intensive computations where the bottleneck is often moving data rather than performing calculations.

3. Floating-Point Performance: Modern GPUs are optimized for floating-point arithmetic, which is essential in many scientific computations. They can perform these operations much faster than CPUs, especially when dealing with single-precision numbers.

4. Energy Efficiency: For the same computational power, GPUs often consume less energy than CPUs, which can be a significant factor in large-scale scientific computing environments.

5. Cost-Effectiveness: In terms of raw computational power per dollar, GPUs often outperform CPUs, making them an attractive option for budget-constrained research projects.

When is a GPU not better?

Because GPUs are specialized computing devices, they have some downsides:

1. Sequential Tasks: Algorithms that can't be easily parallelized often perform better on CPUs due to their higher clock speeds and more sophisticated cores.

2. Complex Control Flow: Tasks involving many conditional statements or complex branching logic are generally better suited to CPUs.

3. Small Data: The overhead of transferring data to and from the GPU can outweigh the benefits for small data.

4. I/O-Bound Tasks: For operations that are limited by input/output speeds rather than computational power, the GPU's advantages may be negated.

Some challenges with GPU programming

For at least 10 years I've tried to get reasonable familiar with GPU programming, mostly with CUDA, which is Nvidia's GPU programming language. There are other GPU programming languages such as OpenCL and HIP that are similar to CUDA. The issue is they are less adopted and seem more difficult. But in general I have found GPU programming to be a tour-de-force, that is it is always more difficult then I anticipate. Here are some reasons:

1. Different Programming Model: GPU programming requires thinking in terms of parallel execution, which can be counterintuitive or unnatural.

2. Memory Management: GPU memory is limited and efficient GPU programming requires careful management memory types and data transfer between CPU and GPU.

3. Debugging Tools: I've never been good with debugging tools, but GPU code is much harder to debug.

4. Optimization: Achieving peak performance usually requires understanding specific GPU architectures.

Example PyCUDA code

To illustrate GPU programming in a context relevant to materials science, let's consider a simple example of calculating the Lennard-Jones potential, which is commonly used to model interatomic interactions in molecular dynamics simulations. Here's a CUDA implementation using Python and PyCUDA [3]:

import numpy as np
import pycuda.autoinit
import pycuda.driver as drv
from pycuda.compiler import SourceModule


# GPU implementation
cuda_module = SourceModule("""
global void lennard_jones_gpu(float potential, 
                                  float positions, 
                                  int num_atoms, 
                                  float epsilon, 
                                  float sigma)
{
    const int i = threadIdx.x + blockDim.x * blockIdx.x;
    if (i < num_atoms) {
        float pote = 0.0f;
        float3 pos_i = make_float3(positions[3i], 
                                   positions[3i+1], 
                                   positions[3i+2]);


        float sigma6 = powf(sigma, 6);
        float sigma12 = sigma6 * sigma6;
        for (int j = 0; j < num_atoms; j++) {
            if (i != j) {
                float3 pos_j = make_float3(positions[3j], 
                                           positions[3j+1], 
                                           positions[3j+2]);
                float3 r = make_float3(pos_i.x - pos_j.x, 
                                       pos_i.y - pos_j.y, 
                                       pos_i.z - pos_j.z);
                float r2 = r.xr.x + r.yr.y + r.zr.z;
                float r6 = r2r2r2;
                float r12 = r6r6;


                pote += 4.0f * epsilon * ((sigma12/r12) - (sigma6/r6));
            }
        }
        potential[i] = pote;
    }
}
""")


lennard_jones_gpu = cuda_module.get_function("lennard_jones_gpu")


# Parameters
num_atoms = 1000
epsilon = 1.0

sigma = 1.0 



# Arrays
positions = np.random.rand(num_atoms * 3).astype(np.float32)
potential = np.zeros(num_atoms).astype(np.float32)


# Call the kernel
lennard_jones_gpu(
    drv.Out(potential), drv.In(positions),
    np.int32(num_atoms), np.float32(epsilon), np.float32(sigma),
    block=(256, 1, 1), grid=((num_atoms + 255) // 256, 1)
)

The keyword args block and grid specify to the kernel how to perform the parallelization (i.e., how to decompose the arrays into parallel computes). This example demonstrates how we can use GPU computing to calculate the Lennard-Jones potential for a system of atoms, which is a fundamental operation in many molecular dynamics simulations. The GPU's parallel processing capabilities allow us to compute the potential for many atoms simultaneously, which can lead to significant speedups compared to CPU-based calculations, especially for larger systems.

Benchmarking CPU vs GPU: Lennard-Jones Potential Calculation

To truly appreciate the performance gains in using GPUs for computational materials science, I put together a jupyter notebook which compares the execution time of Lennard-Jones potential calculations on both CPU and GPU. I think it helps demonstrate how much speed-up can be had with GPU hardware for the right problem.

Number of Atoms	Single CPU Time (s)	GPU Time (s)	Speedup
100	0.1490	0.0008	~1830x
500	2.7130	0.0009	~3100x
1000	10.8806	0.0009	~11000x
2000	43.7833	0.0127	~3400x
5000	274.5926	0.0135	~20000x

One pecularity is that the 2000 atom system shows a drop in speed-up, not sure what causes this. It could be the block and grid settings, but then for 5000 atoms we see the speed-up factor jumps again.

Attention

The speed-up will be GPU hardware dependent and I'm not comparing it to multi-core CPU performance. The hardware used to generate the speed-up table was a RTX A4000 GPU so other cards may be better or worse, just depends. It's also important to note that this is a simplified example. Practical atomistic simulations often include additional routines and more complex force fields.

Summary

I think this benchmark demonstrates why GPUs can be so powerful for scientific computing tasks like MD simulations. As the number of atoms in the system increases, the GPU's ability to perform many calculations in parallel leads to increasingly dramatic performance improvements over an single CPU. Yes MPI and OpenMP and can be used to provide great speed-ups.

As I continue to explore the use of GPUs in materials science and AI, I'm pretty excited by the possibilities these performance improvements open up. With GPUs, we can simulate larger systems for longer timescales, potentially leading to new insights in materials research and drug discovery.

Footnotes

---

1. In numerical methods, the problem at hand is represented in a discretized fashion that is solved using basic arithmetic operations. However, it is possible to represent the problem using symbolic formalism and then the symbolic form is manipulated based on rules and conditions to arrive at a solution. ↩

2. I'm using scientific computing applications as a stand in for both physical science and ML/AI computing. ↩

References

[1] D.B. Kirk, W.W. Hwu, Programming Massively Parallel Processors, Third Edition: A Hands-On Approach, CreateSpace Independent Publishing Platform, 2017. URL.

[2] J. Nickolls, W.J. Dally, The GPU Computing Era, IEEE Micro 30 (2010) 56–69. DOI.

[3] A. Klöckner, N. Pinto, Y. Lee, B. Catanzaro, P. Ivanov, A. Fasih, PyCUDA and PyOpenCL: A scripting-based approach to GPU run-time code generation, Parallel Computing 38 (2012) 157–174. DOI.

---

Reuse and Attribution