Yes, the codegen could drop a timestamp file. It's a fair amount of work, and if we're killing this code for HPCC I'm dubious.
If I could make the split work I could do this next. On Fri, Feb 5, 2010 at 12:19 PM, Drew Farris <drew.far...@gmail.com> wrote: > So, I'm running: mvn -o install -DskipTests=true at project root (in mahout) > > Comment out or remove the maven-assembly-plugin definition in > core/pom.xml -- it reduced my core build time from 26s to 6s -- I can > submit a patch for this. > > Mahout math is still 17s here due to code generation. I'm wondering if > there's a way to modify the generation plugin to that it doesn't > re-generate if there are no changes to the templates. You can remove > the plugin definition from math/pom.xml and it doesn't seem to break > anything unless you're doing a clean. Brings down math compilation to > 3s without it. Total compile time is 22s. > > re: the job, I'll have to look into that further later. > > On Fri, Feb 5, 2010 at 12:06 PM, Robin Anil <robin.a...@gmail.com> wrote: >> Yes for editing i use eclipse in the same fashion. If i want to try out a >> job and see how it performs on hadoop I need job compiled fast. >> >> On another note. I think there will be a lot of dead code in the job(with >> all the jar files bundles) Is there an optimiser for that i.e to remove >> classes which mahout never use indirectly >> >> I see loading jar takes 10-20 seconds when initializing mapper or reducer. >> It doesnt affect long running jobs but 20 sec overhead for processing 64MB >> chunk sucks >> >> On Fri, Feb 5, 2010 at 10:19 PM, Ted Dunning <ted.dunn...@gmail.com> wrote: >> >>> I usually do an initial compilation using mvn package. Then, during >>> development I use IntelliJ's incremental compilation which generally only >>> takes a few seconds. Since that compilation doesn't handle things like >>> copying resources, I get caught out and surprised now and again, but this >>> works almost all the time. >>> >>> On Fri, Feb 5, 2010 at 12:27 AM, Robin Anil <robin.a...@gmail.com> wrote: >>> >>> > When developing mahout core/util/examples we dont need to generate math >>> > often and dont need to tar gzip bzip2 the jar files. We are mostly >>> > concerned >>> > with the job file/ jar file. >>> > Cant there be another target like develop which does this. (waiting 2-3 >>> > mins >>> > for a 2 line change is frustrating) >>> > >>> > Robin >>> > >>> >>> >>> >>> -- >>> Ted Dunning, CTO >>> DeepDyve >>> >> >
