Allreduce

• Traverse tree twice … 2log_2(P) cycles

Reduce

stage 0: process 1 sends to 0

process 3 sends to 2

process 5 sends to 4

process 7 sends to 6

stage 1: process 2 sends to 0

process 6 sends to 4

stage 2: process 4 sends to 0

Broadcast

stage 0: process 0 sends to 4

stage 1: process 0 sends to 2

process 4 sends to 6

stage 2: process 0 sends to 1

process 2 sends to 3

process 4 sends to 5

process 6 sends to 7

Previous slide

Next slide

Back to first slide

View graphic version