Then we could make a profile that turns off the code gen and turns on the build helper to add the generated source dir instead.
On Fri, Feb 5, 2010 at 4:49 PM, Robin Anil <robin.a...@gmail.com> wrote: > Its just meant to be a dev only hack :) > > > On Sat, Feb 6, 2010 at 3:09 AM, Benson Margulies <bimargul...@gmail.com>wrote: > >> Yes, the codegen could drop a timestamp file. It's a fair amount of >> work, and if we're killing this code for HPCC I'm dubious. >> >> If I could make the split work I could do this next. >> >> >> On Fri, Feb 5, 2010 at 12:19 PM, Drew Farris <drew.far...@gmail.com> >> wrote: >> > So, I'm running: mvn -o install -DskipTests=true at project root (in >> mahout) >> > >> > Comment out or remove the maven-assembly-plugin definition in >> > core/pom.xml -- it reduced my core build time from 26s to 6s -- I can >> > submit a patch for this. >> > >> > Mahout math is still 17s here due to code generation. I'm wondering if >> > there's a way to modify the generation plugin to that it doesn't >> > re-generate if there are no changes to the templates. You can remove >> > the plugin definition from math/pom.xml and it doesn't seem to break >> > anything unless you're doing a clean. Brings down math compilation to >> > 3s without it. Total compile time is 22s. >> > >> > re: the job, I'll have to look into that further later. >> > >> > On Fri, Feb 5, 2010 at 12:06 PM, Robin Anil <robin.a...@gmail.com> >> wrote: >> >> Yes for editing i use eclipse in the same fashion. If i want to try out >> a >> >> job and see how it performs on hadoop I need job compiled fast. >> >> >> >> On another note. I think there will be a lot of dead code in the >> job(with >> >> all the jar files bundles) Is there an optimiser for that i.e to remove >> >> classes which mahout never use indirectly >> >> >> >> I see loading jar takes 10-20 seconds when initializing mapper or >> reducer. >> >> It doesnt affect long running jobs but 20 sec overhead for processing >> 64MB >> >> chunk sucks >> >> >> >> On Fri, Feb 5, 2010 at 10:19 PM, Ted Dunning <ted.dunn...@gmail.com> >> wrote: >> >> >> >>> I usually do an initial compilation using mvn package. Then, during >> >>> development I use IntelliJ's incremental compilation which generally >> only >> >>> takes a few seconds. Since that compilation doesn't handle things like >> >>> copying resources, I get caught out and surprised now and again, but >> this >> >>> works almost all the time. >> >>> >> >>> On Fri, Feb 5, 2010 at 12:27 AM, Robin Anil <robin.a...@gmail.com> >> wrote: >> >>> >> >>> > When developing mahout core/util/examples we dont need to generate >> math >> >>> > often and dont need to tar gzip bzip2 the jar files. We are mostly >> >>> > concerned >> >>> > with the job file/ jar file. >> >>> > Cant there be another target like develop which does this. (waiting >> 2-3 >> >>> > mins >> >>> > for a 2 line change is frustrating) >> >>> > >> >>> > Robin >> >>> > >> >>> >> >>> >> >>> >> >>> -- >> >>> Ted Dunning, CTO >> >>> DeepDyve >> >>> >> >> >> > >> >
