Re: System-wide, read-only repository
On 13 November 2013 17:53, Matthieu Moy Matthieu.Moy@grenoble- It wouldn't be that great. We have a fast internet connection, and our project doesn't have _that_ many dependencies. Hehehe. Isn't common adage that the first use of Maven downloads the Internet (and the kitchen sink)? It's the students first contact with Maven (and with tons of other tools), so we really need to make it easy for them (and for the teacher's team too, indeed, we have a bunch of things to maintain, and reducing the list by one is nice). Keeping the default configuration as much as possible is a real gain here (nothing to configure, works at school, works on their laptop too). I'm not having a real problem to solve here, just trying to avoid waste. Fair enough, I can understand that. Actually there is good reason to install a Maven Repository Manager on laptops. It means that you can work disconnected (as long as you have already downloaded the artifacts previously) as the offline flag (-o) doesn't always work that great. I guess the more inquisitive students will find enough stuff on the internet that they can do this themselves when they face this problem. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
It's the students first contact with Maven (and with tons of other tools), so we really need to make it easy for them (and for the teacher's team too, indeed, we have a bunch of things to maintain, and reducing the list by one is nice). Keeping the default configuration as much as possible is a real gain here (nothing to configure, works at school, works on their laptop too). You could also offer your own packaging of Maven with a customized settings.xml etc to make things even easier on your students. This is typical in sufficiently large corporate environments. Wayne - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
I'd go for what Wayne is suggesting. Installing Maven Repository Manager on laptops? hmmm... It will cause artifacts to be duplicated in .m2/repository and in Nexus/Artifactory folder structure. There's no way (IMHO) of forcing users not to change settings.xml - but if provided template works for most cases, not many will see the need to change anything. Having control over repos used or artifacts allowed can be accomplished on a project level. My suggestion is to set-up organisation-wide repository, proxy to central, codehous, jboss, ..., then: 1) Route everything through it with mirror configured, together with a) routes defined (in case of Nexus) and b) take extra care of mirrorOf (see http://maven.apache.org/guides/mini/guide-mirror-settings.html - Adv. Mirror Spec) 2) When setting repositories override updatePolicy daily (default) with never for releases repositories. (I consider re-release of an artifact bad idea anyway). repository idorganisation-repo/id urlhttps://repo.organisation.com/the/right/path/url releases enabledtrue/enabled updatePolicynever/updatePolicy /releases snapshots enabledfalse/enabled /snapshots /repository Ziga On Wed, Nov 13, 2013 at 4:23 PM, Wayne Fay wayne...@gmail.com wrote: It's the students first contact with Maven (and with tons of other tools), so we really need to make it easy for them (and for the teacher's team too, indeed, we have a bunch of things to maintain, and reducing the list by one is nice). Keeping the default configuration as much as possible is a real gain here (nothing to configure, works at school, works on their laptop too). You could also offer your own packaging of Maven with a customized settings.xml etc to make things even easier on your students. This is typical in sufficiently large corporate environments. Wayne - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
Barrie Treloar baerr...@gmail.com writes: When you decide on a solution, a write up of what you did would be helpful for the archives and anyone in the future that has the same needs as you. After a bit of testing, the speed difference is almost unnoticeable. I've documented a way to put the local repo in /var/tmp/ (it's french, but if anyone cares, it's here: http://ensiwiki.ensimag.fr/index.php/Questions_fr%C3%A9quentes_avec_Maven_pour_le_projet_GL ). Most students won't read the docs, they will eat our disk space and load the NFS server, but that's not a very big issue anyway. -- Matthieu Moy http://www-verimag.imag.fr/~moy/ - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
On 13 November 2013 04:46, Matthieu Moy matthieu@grenoble-inp.fr wrote: Barrie Treloar baerr...@gmail.com writes: When you decide on a solution, a write up of what you did would be helpful for the archives and anyone in the future that has the same needs as you. After a bit of testing, the speed difference is almost unnoticeable. I've documented a way to put the local repo in /var/tmp/ (it's french, but if anyone cares, it's here: http://ensiwiki.ensimag.fr/index.php/Questions_fr%C3%A9quentes_avec_Maven_pour_le_projet_GL ). Most students won't read the docs, they will eat our disk space and load the NFS server, but that's not a very big issue anyway. Thanks for that. Your wiki doesn't include a Maven Repository Manager setup. Are you not going to set one up? Reduction in your internet bandwidth would be the biggest gain for the organisation. Download speedup will be the biggest gain for the students, instead of taking 10s of minutes it should be downloaded quite quickly. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
Barrie Treloar baerr...@gmail.com writes: Your wiki doesn't include a Maven Repository Manager setup. Are you not going to set one up? That's not planed, no. Reduction in your internet bandwidth would be the biggest gain for the organisation. It wouldn't be that great. We have a fast internet connection, and our project doesn't have _that_ many dependencies. It's the students first contact with Maven (and with tons of other tools), so we really need to make it easy for them (and for the teacher's team too, indeed, we have a bunch of things to maintain, and reducing the list by one is nice). Keeping the default configuration as much as possible is a real gain here (nothing to configure, works at school, works on their laptop too). I'm not having a real problem to solve here, just trying to avoid waste. -- Matthieu Moy http://www-verimag.imag.fr/~moy/ - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
When you decide on a solution, a write up of what you did would be helpful for the archives and anyone in the future that has the same needs as you. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
Lyons, Roy roy.ly...@cmegroup.com writes: :) but like I said, you wouldnt worry about the space if it was all on zfs. zfs has built in de-duplication. you could have 2000 local maven repos and probably not fill your disk since most of it has to do with duplicate jars and such. Deduplication in filesystem is nice, but does not necessarily do all the job. If your backup system does an rsync to another machine, then you blow up the other machine's disk. If you have disk quotas, then you have to decide which user will be accounted for the deduped space (most likely both users). -- Matthieu Moy http://www-verimag.imag.fr/~moy/ - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
Barrie Treloar baerr...@gmail.com writes: I'd put things the student's local Maven cache in /tmp/maven_cache/userid. That's an option. I'd put it in /var/tmp/... instead, as it is less likely to be a small partition or RAM-disk. If they dont have direct access to the internet, Then you need to make sure there is sufficient documentation on how they should configure Maven, if they are not connecting to their Maven Repository Manager it will be too slow. A simple FAQ Why is Maven slow? should fix that. And its a natural penalty for those who cant follow instructions. Yes. And having it opt-in means that students will at worse ignore it. If you have advanced students then they might want to attempt to patch maven so the local cache is thread safe. They'd need to also think about how to handle failed/stalled downloads correctly so it doesn't block everyone else using the cache. But you still have the problem that people working on the same artifacts will be clobbering each others build output. So Stephen's suggestion of a tiered repository layout would be necessary. And as Curtis points out it is something that would be useful, especially for Jenkins, At the very least, writing up some thoughts on the problem/solutions would be helpful. I propose another project to some students, where they contribute to a free software of their choice. I guess I'll add Maven to the list this year, then ;-). -- Matthieu Moy http://www-verimag.imag.fr/~moy/ - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
Barrie Treloar baerr...@gmail.com writes: On 29 October 2013 23:56, Lyons, Roy roy.ly...@cmegroup.com wrote: Unfortunately, you will always have something in $HOME/.m2/repository because that's how maven works. Can I suggest perhaps that you use zfs for deduplication in /home? Otherwise, you can add something like Or give them more disk space - isn't this stuff meant to be cheap now-a-days? Local disk space is cheap. NFS-shared, RAID backed-up disk space, less so. I can live with a few Gb of waste, but I was just wondering whether we could do any better. If they dont have direct access to the internet, They do. Thanks, -- Matthieu Moy http://www-verimag.imag.fr/~moy/ - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
If you can control the version of Maven they run, you could create a aether plugin that would let you use a tiered set of repositories... at least if I understand aether correctly... then by putting that in the $MAVEN_HOME/lib and doing some magic you might be able to at least share the read-only part... but you would be neck deep in aether to write such a thing... and I would advise checking with Benjamin to confirm that such a thing is possible before starting down that road On 30 October 2013 09:18, Matthieu Moy matthieu@grenoble-inp.fr wrote: Barrie Treloar baerr...@gmail.com writes: On 29 October 2013 23:56, Lyons, Roy roy.ly...@cmegroup.com wrote: Unfortunately, you will always have something in $HOME/.m2/repository because that's how maven works. Can I suggest perhaps that you use zfs for deduplication in /home? Otherwise, you can add something like Or give them more disk space - isn't this stuff meant to be cheap now-a-days? Local disk space is cheap. NFS-shared, RAID backed-up disk space, less so. I can live with a few Gb of waste, but I was just wondering whether we could do any better. If they dont have direct access to the internet, They do. Thanks, -- Matthieu Moy http://www-verimag.imag.fr/~moy/ - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
On Wed, Oct 30, 2013 at 10:18:49AM +0100, Matthieu Moy wrote: Barrie Treloar baerr...@gmail.com writes: On 29 October 2013 23:56, Lyons, Roy roy.ly...@cmegroup.com wrote: Unfortunately, you will always have something in $HOME/.m2/repository because that's how maven works. Can I suggest perhaps that you use zfs for deduplication in /home? Otherwise, you can add something like Or give them more disk space - isn't this stuff meant to be cheap now-a-days? Local disk space is cheap. NFS-shared, RAID backed-up disk space, less so. I can live with a few Gb of waste, but I was just wondering whether we could do any better. Disks are cheap. But not free. Running the procurement gantlet is not free. Downtime to install new storage is not free. Lord knows that additional backup tapes are not free, not even cheap. Longer backup windows are not free. Throwing storage at the problem is often a reasonable choice, but it's also reasonable to always ask if there isn't a better way. Sorry, I've been aching to write that for a long time -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient. signature.asc Description: Digital signature
Re: System-wide, read-only repository
Hi all, There is plenty of room for improvement regarding reuse of Maven's local repository cache. Releases in particular are supposed to be immutable so once they are downloaded they could go into a read-only tier as suggested by Stephen. Inventing such a scheme to reuse large portions of the repo cache would be of great benefit to the Maven community. E.g.: the recommended CIS strategy is for every job to use its own local repo cache, which becomes very large. My Jenkins has dozens of Maven build jobs and I cannot afford the bloat; my Jenkins backups are huge enough already. So what I do instead is limit my Maven Jenkins node to a single executor, which is a real waste on a 16 core machine. Much better would be if the jobs could share the bulk of the repo cache. So it's definitely an itch, but not quite itchy enough for anyone to scratch yet... Regards, Curtis On Oct 30, 2013 8:35 AM, Mark H. Wood mw...@iupui.edu wrote: On Wed, Oct 30, 2013 at 10:18:49AM +0100, Matthieu Moy wrote: Barrie Treloar baerr...@gmail.com writes: On 29 October 2013 23:56, Lyons, Roy roy.ly...@cmegroup.com wrote: Unfortunately, you will always have something in $HOME/.m2/repository because that's how maven works. Can I suggest perhaps that you use zfs for deduplication in /home? Otherwise, you can add something like Or give them more disk space - isn't this stuff meant to be cheap now-a-days? Local disk space is cheap. NFS-shared, RAID backed-up disk space, less so. I can live with a few Gb of waste, but I was just wondering whether we could do any better. Disks are cheap. But not free. Running the procurement gantlet is not free. Downtime to install new storage is not free. Lord knows that additional backup tapes are not free, not even cheap. Longer backup windows are not free. Throwing storage at the problem is often a reasonable choice, but it's also reasonable to always ask if there isn't a better way. Sorry, I've been aching to write that for a long time -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient.
Re: System-wide, read-only repository
:) but like I said, you wouldnt worry about the space if it was all on zfs. zfs has built in de-duplication. you could have 2000 local maven repos and probably not fill your disk since most of it has to do with duplicate jars and such. On 10/30/13 10:37 AM, Curtis Rueden ctrue...@wisc.edu wrote: Hi all, There is plenty of room for improvement regarding reuse of Maven's local repository cache. Releases in particular are supposed to be immutable so once they are downloaded they could go into a read-only tier as suggested by Stephen. Inventing such a scheme to reuse large portions of the repo cache would be of great benefit to the Maven community. E.g.: the recommended CIS strategy is for every job to use its own local repo cache, which becomes very large. My Jenkins has dozens of Maven build jobs and I cannot afford the bloat; my Jenkins backups are huge enough already. So what I do instead is limit my Maven Jenkins node to a single executor, which is a real waste on a 16 core machine. Much better would be if the jobs could share the bulk of the repo cache. So it's definitely an itch, but not quite itchy enough for anyone to scratch yet... Regards, Curtis On Oct 30, 2013 8:35 AM, Mark H. Wood mw...@iupui.edu wrote: On Wed, Oct 30, 2013 at 10:18:49AM +0100, Matthieu Moy wrote: Barrie Treloar baerr...@gmail.com writes: On 29 October 2013 23:56, Lyons, Roy roy.ly...@cmegroup.com wrote: Unfortunately, you will always have something in $HOME/.m2/repository because that's how maven works. Can I suggest perhaps that you use zfs for deduplication in /home? Otherwise, you can add something like Or give them more disk space - isn't this stuff meant to be cheap now-a-days? Local disk space is cheap. NFS-shared, RAID backed-up disk space, less so. I can live with a few Gb of waste, but I was just wondering whether we could do any better. Disks are cheap. But not free. Running the procurement gantlet is not free. Downtime to install new storage is not free. Lord knows that additional backup tapes are not free, not even cheap. Longer backup windows are not free. Throwing storage at the problem is often a reasonable choice, but it's also reasonable to always ask if there isn't a better way. Sorry, I've been aching to write that for a long time -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
Hi Roy, zfs has built in de-duplication. ZFS sounds awesome in theory but have you actually tried it? If so, how is it working for you? In particular, how is the performance? -Curtis On Wed, Oct 30, 2013 at 10:43 AM, Lyons, Roy roy.ly...@cmegroup.com wrote: :) but like I said, you wouldnt worry about the space if it was all on zfs. zfs has built in de-duplication. you could have 2000 local maven repos and probably not fill your disk since most of it has to do with duplicate jars and such. On 10/30/13 10:37 AM, Curtis Rueden ctrue...@wisc.edu wrote: Hi all, There is plenty of room for improvement regarding reuse of Maven's local repository cache. Releases in particular are supposed to be immutable so once they are downloaded they could go into a read-only tier as suggested by Stephen. Inventing such a scheme to reuse large portions of the repo cache would be of great benefit to the Maven community. E.g.: the recommended CIS strategy is for every job to use its own local repo cache, which becomes very large. My Jenkins has dozens of Maven build jobs and I cannot afford the bloat; my Jenkins backups are huge enough already. So what I do instead is limit my Maven Jenkins node to a single executor, which is a real waste on a 16 core machine. Much better would be if the jobs could share the bulk of the repo cache. So it's definitely an itch, but not quite itchy enough for anyone to scratch yet... Regards, Curtis On Oct 30, 2013 8:35 AM, Mark H. Wood mw...@iupui.edu wrote: On Wed, Oct 30, 2013 at 10:18:49AM +0100, Matthieu Moy wrote: Barrie Treloar baerr...@gmail.com writes: On 29 October 2013 23:56, Lyons, Roy roy.ly...@cmegroup.com wrote: Unfortunately, you will always have something in $HOME/.m2/repository because that's how maven works. Can I suggest perhaps that you use zfs for deduplication in /home? Otherwise, you can add something like Or give them more disk space - isn't this stuff meant to be cheap now-a-days? Local disk space is cheap. NFS-shared, RAID backed-up disk space, less so. I can live with a few Gb of waste, but I was just wondering whether we could do any better. Disks are cheap. But not free. Running the procurement gantlet is not free. Downtime to install new storage is not free. Lord knows that additional backup tapes are not free, not even cheap. Longer backup windows are not free. Throwing storage at the problem is often a reasonable choice, but it's also reasonable to always ask if there isn't a better way. Sorry, I've been aching to write that for a long time -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
I am trying to get my unix admins to use it for us. here are some benchmarks I have seen. On 10/30/13 10:51 AM, Curtis Rueden ctrue...@wisc.edu wrote: Hi Roy, zfs has built in de-duplication. ZFS sounds awesome in theory but have you actually tried it? If so, how is it working for you? In particular, how is the performance? -Curtis On Wed, Oct 30, 2013 at 10:43 AM, Lyons, Roy roy.ly...@cmegroup.com wrote: :) but like I said, you wouldnt worry about the space if it was all on zfs. zfs has built in de-duplication. you could have 2000 local maven repos and probably not fill your disk since most of it has to do with duplicate jars and such. On 10/30/13 10:37 AM, Curtis Rueden ctrue...@wisc.edu wrote: Hi all, There is plenty of room for improvement regarding reuse of Maven's local repository cache. Releases in particular are supposed to be immutable so once they are downloaded they could go into a read-only tier as suggested by Stephen. Inventing such a scheme to reuse large portions of the repo cache would be of great benefit to the Maven community. E.g.: the recommended CIS strategy is for every job to use its own local repo cache, which becomes very large. My Jenkins has dozens of Maven build jobs and I cannot afford the bloat; my Jenkins backups are huge enough already. So what I do instead is limit my Maven Jenkins node to a single executor, which is a real waste on a 16 core machine. Much better would be if the jobs could share the bulk of the repo cache. So it's definitely an itch, but not quite itchy enough for anyone to scratch yet... Regards, Curtis On Oct 30, 2013 8:35 AM, Mark H. Wood mw...@iupui.edu wrote: On Wed, Oct 30, 2013 at 10:18:49AM +0100, Matthieu Moy wrote: Barrie Treloar baerr...@gmail.com writes: On 29 October 2013 23:56, Lyons, Roy roy.ly...@cmegroup.com wrote: Unfortunately, you will always have something in $HOME/.m2/repository because that's how maven works. Can I suggest perhaps that you use zfs for deduplication in /home? Otherwise, you can add something like Or give them more disk space - isn't this stuff meant to be cheap now-a-days? Local disk space is cheap. NFS-shared, RAID backed-up disk space, less so. I can live with a few Gb of waste, but I was just wondering whether we could do any better. Disks are cheap. But not free. Running the procurement gantlet is not free. Downtime to install new storage is not free. Lord knows that additional backup tapes are not free, not even cheap. Longer backup windows are not free. Throwing storage at the problem is often a reasonable choice, but it's also reasonable to always ask if there isn't a better way. Sorry, I've been aching to write that for a long time -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
gah, resending with the link this time. http://openbenchmarking.org/result/1304257-FO-MERGE786390 On 10/30/13 11:21 AM, Lyons, Roy roy.ly...@cmegroup.com wrote: I am trying to get my unix admins to use it for us. here are some benchmarks I have seen. On 10/30/13 10:51 AM, Curtis Rueden ctrue...@wisc.edu wrote: Hi Roy, zfs has built in de-duplication. ZFS sounds awesome in theory but have you actually tried it? If so, how is it working for you? In particular, how is the performance? -Curtis On Wed, Oct 30, 2013 at 10:43 AM, Lyons, Roy roy.ly...@cmegroup.com wrote: :) but like I said, you wouldnt worry about the space if it was all on zfs. zfs has built in de-duplication. you could have 2000 local maven repos and probably not fill your disk since most of it has to do with duplicate jars and such. On 10/30/13 10:37 AM, Curtis Rueden ctrue...@wisc.edu wrote: Hi all, There is plenty of room for improvement regarding reuse of Maven's local repository cache. Releases in particular are supposed to be immutable so once they are downloaded they could go into a read-only tier as suggested by Stephen. Inventing such a scheme to reuse large portions of the repo cache would be of great benefit to the Maven community. E.g.: the recommended CIS strategy is for every job to use its own local repo cache, which becomes very large. My Jenkins has dozens of Maven build jobs and I cannot afford the bloat; my Jenkins backups are huge enough already. So what I do instead is limit my Maven Jenkins node to a single executor, which is a real waste on a 16 core machine. Much better would be if the jobs could share the bulk of the repo cache. So it's definitely an itch, but not quite itchy enough for anyone to scratch yet... Regards, Curtis On Oct 30, 2013 8:35 AM, Mark H. Wood mw...@iupui.edu wrote: On Wed, Oct 30, 2013 at 10:18:49AM +0100, Matthieu Moy wrote: Barrie Treloar baerr...@gmail.com writes: On 29 October 2013 23:56, Lyons, Roy roy.ly...@cmegroup.com wrote: Unfortunately, you will always have something in $HOME/.m2/repository because that's how maven works. Can I suggest perhaps that you use zfs for deduplication in /home? Otherwise, you can add something like Or give them more disk space - isn't this stuff meant to be cheap now-a-days? Local disk space is cheap. NFS-shared, RAID backed-up disk space, less so. I can live with a few Gb of waste, but I was just wondering whether we could do any better. Disks are cheap. But not free. Running the procurement gantlet is not free. Downtime to install new storage is not free. Lord knows that additional backup tapes are not free, not even cheap. Longer backup windows are not free. Throwing storage at the problem is often a reasonable choice, but it's also reasonable to always ask if there isn't a better way. Sorry, I've been aching to write that for a long time -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
On 30 October 2013 19:48, Matthieu Moy matthieu@grenoble-inp.fr wrote: Local disk space is cheap. NFS-shared, RAID backed-up disk space, less so. I can live with a few Gb of waste, but I was just wondering whether we could do any better. As others have pointed out, its not possible to do better at the moment. The students ~/.m2 does not need to be backed-up/NFS/RAID as everything is generated. Local disk on each machine is fine. As long as you have a Maven Repository Manager to cache the central artifacts, otherwise it will be too slow to re-seed the local Maven cache (i.e. ~/.m2/repository). I'd put things the student's local Maven cache in /tmp/maven_cache/userid. It doesn't need to survive reboots. Depending on your level of trust, you will need to write scripts to scan these directories for unauthorized files being squirreled here. If you are doing full release cycles then you will also need somewhere for students to deploy their released artifacts. The local Maven cache is not where you deploy these, so its fine for the local cache to get removed as the released artifacts will be fetched from the deployed location. A side benefit of local disk is that you wont get NFS latency and your builds will continue to work when NFS is flaky. If they dont have direct access to the internet, Then you need to make sure there is sufficient documentation on how they should configure Maven, if they are not connecting to their Maven Repository Manager it will be too slow. A simple FAQ Why is Maven slow? should fix that. And its a natural penalty for those who cant follow instructions. If you have advanced students then they might want to attempt to patch maven so the local cache is thread safe. They'd need to also think about how to handle failed/stalled downloads correctly so it doesn't block everyone else using the cache. But you still have the problem that people working on the same artifacts will be clobbering each others build output. So Stephen's suggestion of a tiered repository layout would be necessary. And as Curtis points out it is something that would be useful, especially for Jenkins, At the very least, writing up some thoughts on the problem/solutions would be helpful. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
System-wide, read-only repository
Hi, I have a setup with many users (200 students), each of them having a limited $HOME. I'm looking for a way to provide them the most common plugins and dependencies and save disk space. Is there a way to have a system-wide (e.g. /usr/share/maven/... or so) repository, where the sysadmin could download stuff that the students wouldn't have to download and store in their $HOME ? I found many solutions using a proxy, that would allow me to download from the intranet, but these solutions still require a local repository in $HOME/.m2/, so this saves bandwidth but not disk space. I also saw some people advising to set localRepository in their pom.xml, but this does not seem to be a good idea (not thread-safe), and doesn't really solve my issue, as I would like the system-wide repository to be read-only (managed by root only), and to come in addition to $HOME/.m2/, not as a replacement. Thanks in advance, -- Matthieu Moy http://www-verimag.imag.fr/~moy/ - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
Unfortunately, you will always have something in $HOME/.m2/repository
because that's how maven works.
Can I suggest perhaps that you use zfs for deduplication in /home?
Otherwise, you can add something like
-Dmaven.repo.local=/tmp/${USER}_repository (with a mkdir -p in their
profile or something to make the directory ahead of time). I believe this
can be added to MAVEN_OPTS and it would work...
Either way, with 200 students, I would recommend the zfs since there will
be LOTS of duplicated jars.
Making a single local cache is a very bad idea. there isnt any locking,
so they would clobber each other and get errors. One student would be
accessing the jar as part of compilation and the other will be
mid-overwriting the jar. its messy. Even messier if they change their
umask, and noone can read or write to the location.
So, thats my 2 bits on the topic.
On 10/29/13 4:38 AM, Matthieu Moy matthieu@grenoble-inp.fr wrote:
Hi,
I have a setup with many users (200 students), each of them having a
limited $HOME. I'm looking for a way to provide them the most common
plugins and dependencies and save disk space.
Is there a way to have a system-wide (e.g. /usr/share/maven/... or so)
repository, where the sysadmin could download stuff that the students
wouldn't have to download and store in their $HOME ?
I found many solutions using a proxy, that would allow me to download
from the intranet, but these solutions still require a local repository
in $HOME/.m2/, so this saves bandwidth but not disk space. I also saw
some people advising to set localRepository in their pom.xml, but this
does not seem to be a good idea (not thread-safe), and doesn't really
solve my issue, as I would like the system-wide repository to be
read-only (managed by root only), and to come in addition to $HOME/.m2/,
not as a replacement.
Thanks in advance,
--
Matthieu Moy
http://www-verimag.imag.fr/~moy/
-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org
Re: System-wide, read-only repository
On 29 October 2013 23:56, Lyons, Roy roy.ly...@cmegroup.com wrote: Unfortunately, you will always have something in $HOME/.m2/repository because that's how maven works. Can I suggest perhaps that you use zfs for deduplication in /home? Otherwise, you can add something like Or give them more disk space - isn't this stuff meant to be cheap now-a-days? If they dont have direct access to the internet, then you can setup a Maven Repository Manager, to vet/restrict what artifacts they use. My ~/.m2/repository is 1.2GB but I would expect that to be an upper limit considering the cruft I have accumulated in there. Probably time to delete it and reseed from the local Maven Repository Manager, So back of the envelope calculations say 200 * 1.2GB = 240GB. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
