Hi Roy, > zfs has built in de-duplication.
ZFS sounds awesome in theory but have you actually tried it? If so, how is it working for you? In particular, how is the performance? -Curtis On Wed, Oct 30, 2013 at 10:43 AM, Lyons, Roy <[email protected]> wrote: > :) but like I said, you wouldnt worry about the space if it was all on > zfs. zfs has built in de-duplication. you could have 2000 local maven > repos and probably not fill your disk since most of it has to do with > duplicate jars and such. > > On 10/30/13 10:37 AM, "Curtis Rueden" <[email protected]> wrote: > > >Hi all, > > > >There is plenty of room for improvement regarding reuse of Maven's local > >repository cache. Releases in particular are supposed to be immutable so > >once they are downloaded they could go into a read-only tier as suggested > >by Stephen. Inventing such a scheme to reuse large portions of the repo > >cache would be of great benefit to the Maven community. > > > >E.g.: the recommended CIS strategy is for every job to use its own local > >repo cache, which becomes very large. My Jenkins has dozens of Maven build > >jobs and I cannot afford the bloat; my Jenkins backups are huge enough > >already. So what I do instead is limit my Maven Jenkins node to a single > >executor, which is a real waste on a 16 core machine. Much better would be > >if the jobs could share the bulk of the repo cache. > > > >So it's definitely an itch, but not quite itchy enough for anyone to > >scratch yet... > > > >Regards, > >Curtis > > On Oct 30, 2013 8:35 AM, "Mark H. Wood" <[email protected]> wrote: > > > >> On Wed, Oct 30, 2013 at 10:18:49AM +0100, Matthieu Moy wrote: > >> > Barrie Treloar <[email protected]> writes: > >> > > >> > > On 29 October 2013 23:56, Lyons, Roy <[email protected]> > wrote: > >> > >> Unfortunately, you will always have something in > >>$HOME/.m2/repository > >> > >> because that's how maven works. > >> > >> > >> > >> Can I suggest perhaps that you use zfs for deduplication in /home? > >> > >> Otherwise, you can add something like > >> > > > >> > > Or give them more disk space - isn't this stuff meant to be cheap > >> now-a-days? > >> > > >> > Local disk space is cheap. NFS-shared, RAID & backed-up disk space, > >>less > >> > so. I can live with a few Gb of waste, but I was just wondering > >>whether > >> > we could do any better. > >> > >> Disks are cheap. But not free. Running the procurement gantlet is > >> not free. Downtime to install new storage is not free. Lord knows > >> that additional backup tapes are not free, not even cheap. Longer > >> backup windows are not free. Throwing storage at the problem is often > >> a reasonable choice, but it's also reasonable to always ask if there > >> isn't a better way. > >> > >> Sorry, I've been aching to write that for a long time.... > >> > >> -- > >> Mark H. Wood, Lead System Programmer [email protected] > >> Machines should not be friendly. Machines should be obedient. > >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
