Comments on: Big Oh in the parallel world http://igoro.com/archive/big-oh-in-the-parallel-world/ On programming, technology, and random things of interest Thu, 29 Jul 2010 00:28:29 +0000 hourly 1 http://wordpress.org/?v=3.0 By: Nayuki http://igoro.com/archive/big-oh-in-the-parallel-world/comment-page-1/#comment-902 Nayuki Mon, 03 May 2010 01:54:40 +0000 http://igoro.com/archive/big-oh-in-the-parallel-world/#comment-902 "One interesting thought exercise is to think about this: if the fastest possible sequential algorithm is O(F(N)), is it possible that there is a parallel algorithm which is asymptotically faster than O(F(N) / P) on a machine with P processors?" I think this isn't possible. A single processor can simulate P processors with an O(P) increase in running time. If parallelization can produce an asymptotic speedup better than O(P), then the single processor can take advantage of that to reduce the running time of the sequential algorithm. This would contradict the best running time for the sequential algorithm. This can also contradict trivial lower bounds (such as O(N^2) memory accesses for matrix multiplication) or information-theoretic lower bounds (e.g. O(N log N) for comparison sorting). (This is in the realm of theoretical computer science, disregarding implementation issues like memory hierarchy or superscalar execution or out-of-order execution.) “One interesting thought exercise is to think about this: if the fastest possible sequential algorithm is O(F(N)), is it possible that there is a parallel algorithm which is asymptotically faster than O(F(N) / P) on a machine with P processors?”

I think this isn’t possible. A single processor can simulate P processors with an O(P) increase in running time. If parallelization can produce an asymptotic speedup better than O(P), then the single processor can take advantage of that to reduce the running time of the sequential algorithm. This would contradict the best running time for the sequential algorithm. This can also contradict trivial lower bounds (such as O(N^2) memory accesses for matrix multiplication) or information-theoretic lower bounds (e.g. O(N log N) for comparison sorting).

(This is in the realm of theoretical computer science, disregarding implementation issues like memory hierarchy or superscalar execution or out-of-order execution.)

]]> By: Recent Faves Tagged With "multiplication" : MyNetFaves http://igoro.com/archive/big-oh-in-the-parallel-world/comment-page-1/#comment-458 Recent Faves Tagged With "multiplication" : MyNetFaves Mon, 23 Feb 2009 14:50:17 +0000 http://igoro.com/archive/big-oh-in-the-parallel-world/#comment-458 [...] — Algebraic Properties of Equality and Properties ... First saved by cxcheng | 8 days ago Big Oh in the parallel world First saved by kulaju8 | 11 days ago Matrices for programmers First saved by janus | 21 days [...] [...] — Algebraic Properties of Equality and Properties … First saved by cxcheng | 8 days ago Big Oh in the parallel world First saved by kulaju8 | 11 days ago Matrices for programmers First saved by janus | 21 days [...]

]]>