Platonides wrote: > Yuvi Panda wrote: > > Hi, I'm Yuvi, a student looking forward to working with MediaWiki via > > this year's GSoC.
<snip/> > An idea I have been pondering is to pass the offset to the previous > revision to the compressor, so it would need much less work in the > compressing window to perform its work. You would need something like > 7z/xz so that the window can be big enough to contain at least the > latest revision (its compression factor is quite impressive, too: 1TB > down to 2.31GB). Note that I haven't checked on how factible it can be > such modification to the compressor. Consider using pigz for the compression step. + Much (7x?) faster than gzip + Straighforward install + Stable + One or more threads per CPU (settable) - Only compresses to .gz or .zz formats - Linux only Alternately, could Gnu make's parallel feature be used? For example, "make -j --load-average=30" will keep adding jobs in parallel until the load average reaches 30. + It's make - It's make -- Charles Polisher _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
