On Sun, Jan 24, 2016 at 09:22AM, Evans Ye wrote: > I actually feels OK about having the dependency, if that's what spark want.
This isn't what Spark wants; this is what spark-hive integration needs. But my point is a bit different: let's suppose we all agree we need the Hive support in Spark (which I am not convinced, but don't care enough to make an argument). Now, without datanucleus libs spark-shell is broken and won't work. Perhaps other clients using spark-submit won't work either (again, I don't know nor care enough about Spark to figure out). Shall we simply get rid of the datanucleus package and install these libs everywhere now? Cause you know - having a package just for the sake of having the package doesn't sound very optimal to me. See where I am going with it? > BTW, during the test I found that the spark installation dir are under > /usr/lib/spark instead of /var/lib/spark. > I expect we should put all the component libs under /var/lib for > consistency. Is it correct? No, I believe /usr/lib/<component> is what we've been doing for a long time. Look at /usr/lib/hadoop or hbase or anything else for that matter. Cos > 2016-01-23 4:27 GMT+08:00 Konstantin Boudnik <[email protected]>: > > > Hey all. > > > > I hate to pollute the list with this extra traffic, but there's a quite > > ugly > > issue that we are facing because of the infamous Hive-Spark collusion. I am > > talking about https://issues.apache.org/jira/browse/BIGTOP-2154 > > > > I have proposed a patch to work around it and let the 1.1 out of the door. > > I > > have also open BIGTOP-2268 so we don't forget to fix it properly in the > > next > > release. > > > > I'd appreciate someone else taking a look and sharing the feedback, so we > > can > > finally unblock the RC1. > > > > Thanks, > > Cos > >
signature.asc
Description: Digital signature
