Multi-Core compression with pbzip2
Because I now have 16 logical processors, according to OSXs Activity Monitor (8 cores with Hyperthreading), I've been looking for any and all software that can make use of them. I was dissapointed to find out that almost no compression software supports more than one thread.
Until I found pbzip2!
Until I found pbzip2!
I used bzip2 and pbzip2 to compress a 133MB installer file for a random webapp I had in my Downloads directory:
bzip2: 40.479 seconds
pbzip2: 5.516 seconds
Not quite 16 times faster, but quite an improvement!
I also tried it with it running in 8 threads:
7.334 seconds. This must be because it probably uses some virtual processors that are running on the same core.
Decompressing also shows quite some improvement, although with one little caveat:
bzip2: 16.832 seconds
pbzip2: 2.643 seconds, but only if the .bz2 file was compressed with pbzip2!
If the file was compressed with bzip2, then decompressing took just as long as when I'd use the regular bzip2 program.
The only drawback of using pbzip2 on all 16 cores is that my CPU usage goes to "1590%", according to Activity Monitor... That was actually the first time I got my Mac to slow down, reminded me of working on my laptop again...
bzip2: 40.479 seconds
pbzip2: 5.516 seconds
Not quite 16 times faster, but quite an improvement!
I also tried it with it running in 8 threads:
7.334 seconds. This must be because it probably uses some virtual processors that are running on the same core.
Decompressing also shows quite some improvement, although with one little caveat:
bzip2: 16.832 seconds
pbzip2: 2.643 seconds, but only if the .bz2 file was compressed with pbzip2!
If the file was compressed with bzip2, then decompressing took just as long as when I'd use the regular bzip2 program.
The only drawback of using pbzip2 on all 16 cores is that my CPU usage goes to "1590%", according to Activity Monitor... That was actually the first time I got my Mac to slow down, reminded me of working on my laptop again...
Posted in Reviews | no comments |