On 05/02/2015 10:48 AM, Pat Ferrel wrote:
Not removing Guava or any other dependencies from the jar. I don’t have time
right now to fix all those Preconditions that might allow Guava to be removed
and the other classes are needed by various Spark client code.
+1 to dealing with the Guava precondition and assembly stuff in an other
issue.
Again, I propose we factor this into client and worker jars. Removing
Preconditions may allow us to do away with the Worker jar altogether since
guava is not used in Scala now.
On May 1, 2015, at 2:18 PM, Pat Ferrel <[email protected]> wrote:
removing guava shows up a bunch of uses of google Preconditions in math. Guess
I’ll have to remove those. I’ll leave mr and the rest alone since only math
code gets run on a spark worker.
On May 1, 2015, at 10:01 AM, Andrew Palumbo <[email protected]> wrote:
ResultAnalyzer is Also used in SparkNaiveBayes.test (...).
Sent from my Verizon Wireless 4G LTE smartphone
<div>-------- Original message --------</div><div>From: Andrew Palumbo <[email protected]>
</div><div>Date:05/01/2015 12:57 PM (GMT-05:00) </div><div>To: [email protected] </div><div>Subject: RE:
dependency-reduced jar </div><div>
</div>
I added T-digest and math3. the CLI Naive Bayes driver needs them.
Specifically the ResultAnalyzer in TestNBDriver.
Sent from my Verizon Wireless 4G LTE smartphone
<div>-------- Original message --------</div><div>From: Suneel Marthi <[email protected]>
</div><div>Date:05/01/2015 12:14 PM (GMT-05:00) </div><div>To: mahout <[email protected]> </div><div>Subject:
Re: dependency-reduced jar </div><div>
</div>T-digest is being used in Mahout-MR, I believe its also packaged as part
of
Spark -> AddThis jar.
On Fri, May 1, 2015 at 12:11 PM, Pat Ferrel <[email protected]> wrote:
There is an assembly xml in
mahout/spark/src/main/assembly/dependency-reduced.xml. It contains
dependencies that are external to mahout but required for either the client
or backend executor distributed code.
Guava has recently been removed but scopt is still used by the client. For
some reason the following artifacts were added to the assembly and I’m not
sure why. This is only used with Spark.
