An Introduction to Parallel Programming

Title pageTable of ContentsCopyrightDedicationPrefaceChapter 1: Why parallel computing1.1. Why we need ever-increasing performance1.2. Why we're building parallel systems1.3. Why we need to write parallel programs1.4. How do we write parallel programs?1.5. What we'll be doing1.6. Concurrent, parallel, distributed1.7. The rest of the book1.8. A word of warning1.9. Typographical conventions1.10. Summary1.11. ExercisesBibliographyChapter 2: Parallel hardware and parallel software2.1. Some background2.2. Modifications to the von Neumann model2.3. Parallel hardware2.4. Parallel software2.5. Input and output2.6. Performance2.7. Parallel program design2.8. Writing and running parallel programs2.9. Assumptions2.10. Summary2.11. ExercisesBibliographyChapter 3: Distributed memory programming with MPI3.1. Getting started3.2. The trapezoidal rule in MPI3.3. Dealing with I/O3.4. Collective communication3.5. MPI-derived datatypes3.6. Performance evaluation of MPI programs3.7. A parallel sorting algorithm3.8. Summary3.9. Exercises3.10. Programming assignmentsBibliographyChapter 4: Shared-memory programming with Pthreads4.1. Processes, threads, and Pthreads4.2. Hello, world4.3. Matrix-vector multiplication4.4. Critical sections4.5. Busy-waiting4.6. Mutexes4.7. Producer–consumer synchronization and semaphores4.8. Barriers and condition variables4.9. Read-write locks4.10. Caches, cache-coherence, and false sharing4.11. Thread-safety4.12. Summary4.13. Exercises4.14. Programming assignmentsBibliographyChapter 5: Shared-memory programming with OpenMP5.1. Getting started5.2. The trapezoidal rule5.3. Scope of variables5.4. The reduction clause5.5. The parallel for directive5.6. More about loops in OpenMP: sorting5.7. Scheduling loops5.8. Producers and consumers5.9. Caches, cache coherence, and false sharing5.10. Tasking5.11. Thread-safety5.12. Summary5.13. Exercises5.14. Programming assignmentsBibliographyChapter 6: GPU programming with CUDA6.1. GPUs and GPGPU6.2. GPU architectures6.3. Heterogeneous computing6.4. CUDA hello6.5. A closer look6.6. Threads, blocks, and grids6.7. Nvidia compute capabilities and device architectures6.8. Vector addition6.9. Returning results from CUDA kernels6.10. CUDA trapezoidal rule I6.11. CUDA trapezoidal rule II: improving performance6.12. Implementation of trapezoidal rule with warpSize thread blocks6.13. CUDA trapezoidal rule III: blocks with more than one warp6.14. Bitonic sort6.15. Summary6.16. Exercises6.17. Programming assignmentsBibliographyChapter 7: Parallel program development7.1. Two n-body solvers7.2. Sample sort7.3. A word of caution7.4. Which API?7.5. Summary7.6. Exercises7.7. Programming assignmentsBibliographyChapter 8: Where to go from hereBibliographyBibliographyBibliographyIndex