On Feb 15, 2014, at 12:19 AM, Colin Percival <[email protected]> wrote:
> On 02/14/14 15:00, Scott Wheeler wrote: >> My point was rather that Tarsnap’s slowness isn’t intrinsic to it being a >> internet-based backup. Even for an off-site network backup, it’s extremely >> slow. > > Yep. And I'm working on it. In the mean time, the "parallel extracts" > workaround is useful in most cases. I ended up cleaning up and gemifying our little tool for running parallel extracts: https://github.com/directededge/redsnapper http://rubygems.org/gems/redsnapper Doing this will use 25 parallel clients (from testing 25 seemed to be about the place where the benefits maxed out) each extracting up to 50 files to extract an archive: sudo redsnapper ARCHIVE We see about a 5x speedup doing things that way. It’s slower than the normal tool for small archives since it has to fetch the file list first. -Scott -- Scott Wheeler | Co-founder | Directed Edge | www.directededge.com | @directededge
