CS 213

CS 213 — Multiprocessor Architecture and Programming

Time: Tuesday & Thursday, 5:00 – 6:20 pm

Instructor: Elaheh Sadredini elaheh@cs.ucr.edu
Office hours: By appointment

Course Overview

The inevitable and rapidly growing adoption of multi-core parallel architectures within a processor chip pushes explicit parallelism to the forefront of computing for all applications and scales, and makes the challenge of parallel programming and system understanding more crucial. This course builds a strong understanding of the fundamentals of the architecture of parallel computers and the tradeoffs made in their design. We study multi-core architectures, parallel memory systems, cache coherence in shared-memory architectures, memory consistency, vector architectures, dataflow machines, interconnection networks, and programming models deployed on parallel machines (message passing, data-parallel, and shared memory).

Course Prerequisite(s)

CS 203 or consent of instructor

Textbooks and Reading Material

The following books could be useful as supplements to lectures. However, they are not required.

◆ Parallel Computer Architecture: A Hardware/Software Approach — Culler, Singh, and Gupta.

◆ Principles and Practices of Interconnection Networks — Dally and Towles.

◆ Computer Architecture: A Quantitative Approach — Hennessy and Patterson.

Grading and Policies

Grading Breakdown

Homework assignments (two): 20%

Programming assignments (two): 40%

Paper presentation (one): 10%

Quizzes (six): 30%

 - Quizzes will be taken throughout the course.
 - We will have quizzes on eLearn (Canvas).
 - The lowest score will be dropped.

Bonus points for in-class activities: 10%

Class Policies

Forum: We will be using Slack as our class forum; all general inquiries must be made there. For group-specific or private questions, email me.

Late submission: 10% penalty per day up to 5 days for homeworks and programming assignments (no credit after 5 days). No late submissions for quizzes. You may skip one quiz with no penalty; if you submit all, the lowest score is dropped.

Cheating policy: Discussing problem meaning and approaches is encouraged, but all submitted work must be your own. Exams are strictly individual. Violations result in a failing grade.

Errors in grading: You have one week from return to request a regrade via email with a written description.

Dues: All assignments are due at 11:59pm of the deadline date.

Grades: Your score will be available on Canvas.

Grading Scale

A+: 96+
A: 93–95.9
A−: 90–92.9
B+: 87–89.9
B: 83–86.9
B−: 80–82.9
C+: 77–79.9
C: 73–76.9
C−: 70–72.9
D+: 67–69.9
D: 63–66.9
D−: 60–62.9
F: < 60

Academic Integrity

Here at UCR we are committed to upholding the values of Integrity, Accountability, Excellence, and Respect. As a student in this class, complete assignments as described and report suspected misconduct. Policy details: http://conduct.ucr.edu.

Student Resources

◆ If you need special accommodation, contact me or see: UCR campus resources.

Course Schedule

The following schedule is tentative and is subject to change.

#	Date	Session	Topic	Due/Assignment

Reading Materials

Topic	Readings
Parallel Processing Basics	Reading(s): The Task of the Referee, by Smith. Amdahl’s Law in the Multicore Era, by Hill and Marty. The Landscape of Parallel Computing Research, by Asanovic et al. The Future of Microprocessors, by Borkar and Chien. Hardware Trends in Distributed Computing, by Harris. Software and the Concurrency Revolution, by Sutter and Larus. Cost-Effective Parallel Computing, by Wood and Hill. Book Chapter(s): Parallel Computer Architecture: A Hardware/Software Approach, Chapter 1.
Shared Memory Basics	Reading(s): How to make a multiprocessor that correctly executes multiprocess programs, by Lamport. Shared Memory Consistency Models: A Tutorial, by Adve & Gharachorloo. Foundations of the C++ concurrency memory model, by Boehm & Adve. The Java memory model, by Manson et al. Book Chapter(s): H&P, Chapter 5 (5th ed.). Parallel Computer Architecture: A Hardware/Software Approach, Chapter 5. A Primer on Memory Consistency and Cache Coherence, Ch. 1–3.
Memory Organization	Reading(s): Subarray-level parallelism (SALP) in DRAM, by Yoongu et al. LISA: Fast inter-subarray data movement in DRAM, by Chang et al. Book Chapter(s): Memory Systems: Cache, DRAM, Disk, Ch. 10
Programming Models and Architectures	Book Chapter(s): Multiprocessors and Multicomputers, pp. 551–560. Parallel Computer Architecture: A Hardware/Software Approach, Chapter 1.
Cache Coherence	Reading(s): Survey of Cache Coherence Schemes, by Stenstrom. Compatible Cache Consistency Protocols, by Sweazey & Smith. Starfire: Extending the SMP Envelope, by Charlesworth. Directory-Based Coherence for DASH, by Lenoski et al. SGI Challenge Multiprocessor, by Galles & Williams. Verifying Snooping Protocols, by Sorin et al. Verifying Memory Coherence in Cray X1, by Abts et al. Reducing Directory Coherence Traffic, by Gupta et al. Cache Invalidation Patterns, by Gupta & Weber. False Sharing and Locality, by Torrellas et al. Book Chapter(s): H&P, §5.2–5.4 (5th ed.). Parallel Computer Architecture, Ch. 6 & 8. Primer on Consistency & Coherence, Ch. 6–9.
Shared Memory Synchronization	Reading(s): Efficient Synchronization Primitives, by Goodman et al. Scalable Synchronization Algorithms, by Mellor-Crummey & Scott. Let Them Eat QOLB, by Kägi et al. Synchronization/Communication in T3E, by Scott. Book Chapter(s): Shared-Memory Synchronization, Ch. 1–5 & 7.
Memory Consistency Models	Reading(s): Consistency Models: A Tutorial, by Adve & Gharachorloo. Event Ordering in Scalable SMPs, by Gharachorloo et al. Enhancing Model Performance, by Gharachorloo et al. Evaluating Models with ILP, by Pai et al. Threads Cannot Be Implemented As a Library, by Boehm. Book Chapter(s): Primer on Consistency & Coherence, Ch. 3–5.
Interconnection Networks	Reading(s): Alpha 21364 Network, by Mukherjee et al. Tile Processor Interconnect, by Wentzlaff et al. Flattened Butterfly, by Kim et al. Cray T3E Torus, by Scott & Thorson. High-Radix Router, by Kim et al. SeaStar Interconnect, by Brightwell et al. Gemini System Interconnect, by Alverson et al. Book Chapter(s): Datacenter Networks (synthesis lecture). On-Chip Networks, Ch. 2, 5 & 6. Principles and Practices of Interconnection Networks (highly recommended).
Vector, SIMD, and SIMT Processing	Reading(s): The Cray-1 Computer System, Cray Research, Inc. Tarantula: A Vector Extension to the Alpha Architecture, by Espasa et al. NVIDIA Kepler GK110 Architecture, NVIDIA. NVIDIA GeForce GTX 980 (Maxwell), NVIDIA. Book Chapter(s): Computer Architecture: A Quantitative Approach (5th Ed), Ch. 4 & App. G.

Papers Presentation

TBA ...

Resources

◆ WWW Computer Architecture Home Page a comprehensive guide to research, tools, and general information on computer architecture.