I am writing a work and had a big argument with a friend about the way I said one algorithm is asymptotically faster running on parallel than the one running serial, using the big oh notation. BTW your post helped a lot in my argumentation. But do you know any paper or scientific work that explored this concept?

Tks

Counting the ones takes O(N) time, so the sequential complexity would be O(N * m * m).

Then, the parallel complexity would be:

O(N * m * m / p), where p must be O(m).

Basically, the algorithm now has more work to do by a factor of N, but it does not have any more opportunity for parallelism.

Of course, you could try to further parallelize the work of XORing and counting the 1s. For that algorithm, you could claim a higher bound on p. In practice, I doubt the extra parallelism would be worthwhile on a typical modern computer unless N is very large.

If we had a “binary” data set with m objects with each object of size N, how would we write the parallel complexity for calculating the distance matrix using method 3? Here the edit distance turns out to be Hamming distance computed by XORing two objects + counting 1s.

O((m^2 * N) / p) and p is O(m^2) ???

I think this isn’t possible. A single processor can simulate P processors with an O(P) increase in running time. If parallelization can produce an asymptotic speedup better than O(P), then the single processor can take advantage of that to reduce the running time of the sequential algorithm. This would contradict the best running time for the sequential algorithm. This can also contradict trivial lower bounds (such as O(N^2) memory accesses for matrix multiplication) or information-theoretic lower bounds (e.g. O(N log N) for comparison sorting).

(This is in the realm of theoretical computer science, disregarding implementation issues like memory hierarchy or superscalar execution or out-of-order execution.)