I think we shuld be fine with jackson 2.4.4 (its not fasterxml :)) But what's the reason for the massive downgrade? Elasticsearch 2.x and above need atleast Jackson 2.6.2 to function.
Downgrading Kryo, I have no opinion and would let others weigh in. On Wed, Jun 1, 2016 at 8:07 AM, Trevor Grant <[email protected]> wrote: > Can we use kryo v 2.21 (instead of 2.24) > > and fasterxml 2.4.4 instead of 2.7.2 (this is what worries me) > > I am also working on fishing the mahout jars directly out of > MAHOUT_HOME=..../ ... instead of including in the pom, so let's not get to > carried away with pruning dependencies. > > At this point its more of a, "can anyone one think of a specific reason > this shouldn't work". > > > Mahout > > com.esotericsoftware.kryo:kryo:jar:2.24:compile > com.fasterxml.jackson.core:jackson-core:jar:2.7.2:compile > > Spark (1.6) > > com.esotericsoftware.kryo:kryo:jar:2.21:compile > com.fasterxml.jackson.core:jackson-core:jar:2.4.4:compile > > > Trevor Grant > Data Scientist > https://github.com/rawkintrevo > http://stackexchange.com/users/3002022/rawkintrevo > http://trevorgrant.org > > *"Fortunate is he, who is able to know the causes of things." -Virgil* > > > On Tue, May 31, 2016 at 7:55 PM, Andrew Palumbo <[email protected]> > wrote: > > > which dependencies need to be removed? > > > > I saw the Kryo version on the PR was conflicting also, that may be a > flink > > thing. I think the version at one point was being enforced in the flink > > module at least. > > > > ________________________________________ > > From: Trevor Grant <[email protected]> > > Sent: Tuesday, May 31, 2016 8:41:15 PM > > To: [email protected] > > Subject: Re: Zeppelin Integration PR > > > > As a follow up to this- it would be nice to remove the dependencies from > > the pom.xml... > > > > All we REALLY need to do is make sure we can get to the required jars and > > load them. By including them in the pom we are ensuring they are > > available, but there is surely some other way to get ahold of them. > Since > > we have assumed that Mahout is installed on the system and > MAHOUT_HOME=... > > we can probably leverage that... > > > > > > > > Trevor Grant > > Data Scientist > > https://github.com/rawkintrevo > > http://stackexchange.com/users/3002022/rawkintrevo > > http://trevorgrant.org > > > > *"Fortunate is he, who is able to know the causes of things." -Virgil* > > > > > > On Tue, May 31, 2016 at 7:31 PM, Trevor Grant <[email protected]> > > wrote: > > > > > For what it is worth, simply removing the dependencies from pom.xml > > breaks > > > the Mahout interpreter. > > > > > > Upon a little further testing in cluster mode, so long as the > > dependencies > > > are included in pom.xml, the appropriate Mahout jars are shipped off to > > the > > > cluster and everything works swimmingly (in Zeppelin there is a local > > Spark > > > Interpretter internal to Zeppelin and then the 'real' cluster that > > > everything gets shipped off to. Sometimes you can make things work in > > local > > > mode that won't work in cluster mode) > > > > > > The moral of this story is that the patch DOES in fact work in local > and > > > cluster mode, so we just need to work out the dependencies and the > > > licensing (and a couple of fail safes to make sure the user is running > > > Spark version > 1.5.2) and we should be good to go. > > > > > > > > > Trevor Grant > > > Data Scientist > > > https://github.com/rawkintrevo > > > http://stackexchange.com/users/3002022/rawkintrevo > > > http://trevorgrant.org > > > > > > *"Fortunate is he, who is able to know the causes of things." -Virgil* > > > > > > > > > On Tue, May 31, 2016 at 4:22 PM, Trevor Grant < > [email protected]> > > > wrote: > > > > > >> Hey folks, > > >> > > >> looks like we're making progress on the Mahout-Zeppelin integration. > > >> > > >> Any who are interested check out: > > >> https://github.com/apache/incubator-zeppelin/pull/928 > > >> > > >> Regarding Moon's last comments: > > >> Does anyone know off hand if anything will break if we roll back the > > >> conflicting packages to the Spark 1.6 version? > > >> > > >> Also regarding the pom.xml and: > > >> "Packaging > > >> If mahout requires to be loaded in spark executor's classpath, then > > >> adding mahout dependency in pom.xml will not be enough to work with > > Spark > > >> cluster. Could you clarify if mahout need to be loaded in spark > > executor?" > > >> > > >> All we need to do is load the jars appropriate Mahout jars, I'm not > > >> familiar enough with the Spark Interpreter or Spark or Java to know > > exactly > > >> what would happen, any thoughts on this? > > >> > > >> Tonight I might just try removing mahout dependencies from pom.xml and > > >> seeing what happens? that would solve all of these problems I think. > As > > >> long as user has 'mvn install'ed Mahout, should be gtg? > > >> > > >> Trevor Grant > > >> Data Scientist > > >> https://github.com/rawkintrevo > > >> http://stackexchange.com/users/3002022/rawkintrevo > > >> http://trevorgrant.org > > >> > > >> *"Fortunate is he, who is able to know the causes of things." > -Virgil* > > >> > > >> > > > > > >
