Not removing Guava or any other dependencies from the jar. I don’t have time right now to fix all those Preconditions that might allow Guava to be removed and the other classes are needed by various Spark client code.
Again, I propose we factor this into client and worker jars. Removing Preconditions may allow us to do away with the Worker jar altogether since guava is not used in Scala now. On May 1, 2015, at 2:18 PM, Pat Ferrel <[email protected]> wrote: removing guava shows up a bunch of uses of google Preconditions in math. Guess I’ll have to remove those. I’ll leave mr and the rest alone since only math code gets run on a spark worker. On May 1, 2015, at 10:01 AM, Andrew Palumbo <[email protected]> wrote: ResultAnalyzer is Also used in SparkNaiveBayes.test (...). Sent from my Verizon Wireless 4G LTE smartphone <div>-------- Original message --------</div><div>From: Andrew Palumbo <[email protected]> </div><div>Date:05/01/2015 12:57 PM (GMT-05:00) </div><div>To: [email protected] </div><div>Subject: RE: dependency-reduced jar </div><div> </div> I added T-digest and math3. the CLI Naive Bayes driver needs them. Specifically the ResultAnalyzer in TestNBDriver. Sent from my Verizon Wireless 4G LTE smartphone <div>-------- Original message --------</div><div>From: Suneel Marthi <[email protected]> </div><div>Date:05/01/2015 12:14 PM (GMT-05:00) </div><div>To: mahout <[email protected]> </div><div>Subject: Re: dependency-reduced jar </div><div> </div>T-digest is being used in Mahout-MR, I believe its also packaged as part of Spark -> AddThis jar. On Fri, May 1, 2015 at 12:11 PM, Pat Ferrel <[email protected]> wrote: > There is an assembly xml in > mahout/spark/src/main/assembly/dependency-reduced.xml. It contains > dependencies that are external to mahout but required for either the client > or backend executor distributed code. > > Guava has recently been removed but scopt is still used by the client. For > some reason the following artifacts were added to the assembly and I’m not > sure why. This is only used with Spark. > >
