Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

> On Jul 25, 2016, at 1:16 PM, Sangjin Lee wrote: > > Also: right now, the non-Linux and/or non-x86 platforms have to supply their > own leveldbjni jar (or at least the C level library?) in order to make YARN > even functional. How is that going to work with the class path

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

On Fri, Jul 22, 2016 at 5:15 PM, Allen Wittenauer wrote: > > But if I don't use ApplicationClassLoader, my java app is basically > screwed then, right? > If we start upgrading the libraries aggressively, then it would also mean that the ApplicationClassLoader should

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

> On Jul 22, 2016, at 5:47 PM, Zheng, Kai wrote: > > For the leveldb thing, wouldn't we have an alternative option in Java for the > platforms where leveldb isn't supported yet due to whatever reasons. IMO, > native library would be best to be used for optimization and

RE: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

For the leveldb thing, wouldn't we have an alternative option in Java for the platforms where leveldb isn't supported yet due to whatever reasons. IMO, native library would be best to be used for optimization and production for performance. For development and pure Java platform, by default

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

But if I don't use ApplicationClassLoader, my java app is basically screwed then, right? Also: right now, the non-Linux and/or non-x86 platforms have to supply their own leveldbjni jar (or at least the C level library?) in order to make YARN even functional. How is that going to work with

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

The work on HADOOP-13070 and the ApplicationClassLoader are generic and go beyond YARN. It can be used in any JVM that uses hadoop. The current use cases are MR containers, hadoop's RunJar (as in "hadoop jar"), and the YARN node manager auxiliary services. I'm not sure if that's what you were

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

Does any of this work actually help processes that sit outside of YARN? > On Jul 21, 2016, at 12:29 PM, Sean Busbey wrote: > > thanks for bringing this up! big +1 on upgrading dependencies for 3.0. > > I have an updated patch for HADOOP-11804 ready to post this week. I've

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

thanks for bringing this up! big +1 on upgrading dependencies for 3.0. I have an updated patch for HADOOP-11804 ready to post this week. I've been updating HBase's master branch to try to make use of it, but could use some other reviews. On Thu, Jul 21, 2016 at 4:30 AM, Tsuyoshi Ozawa

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

Thanks Tsuyoshi for opening the discussion. One benefit of the dependency/classpath isolation work is that it can open up a possibility of having diverging dependencies in a safe manner so that upgrading libraries may have less impact. I'll spend some more time on HADOOP-13070 to make some

[DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

Hi developers, I'd like to discuss how to make an advance towards dependency management in Apache Hadoop trunk code since there has been lots work about updating dependencies in parallel. Summarizing recent works and activities as follows: 0) Currently, we have merged minimum update dependencies