I'm backing up some large unchanging files (web server logs). Aside from the current log, they mostly are unchanging on a daily basis. As per recommendations I've not compressed these files which gives Tarsnap the best chance to deduplicate and compress.
But, the problem is that Tarsnap is reading these files every day in their entirety. I guess it has to so it can identify changed blocks, but this is making the backup take a long time and creates a fair amount of I/O. And aside from the monthly log rollover, these files haven't changed from one day to the next. Are there any useful tips for how to handle this situation? I've pondered looking at the nodump flag on the rotated log files, and making sure that Tarsnap usually skips these files, except after they've rotated. Or I could just compress them - they'd compress really well and speed up the backups significantly, but I'd lose the benefits of Tarsnap's deduplication as a result. Tim.
