Does any of this work actually help processes that sit outside of YARN? > On Jul 21, 2016, at 12:29 PM, Sean Busbey <bus...@cloudera.com> wrote: > > thanks for bringing this up! big +1 on upgrading dependencies for 3.0. > > I have an updated patch for HADOOP-11804 ready to post this week. I've > been updating HBase's master branch to try to make use of it, but > could use some other reviews. > > On Thu, Jul 21, 2016 at 4:30 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote: >> Hi developers, >> >> I'd like to discuss how to make an advance towards dependency >> management in Apache Hadoop trunk code since there has been lots work >> about updating dependencies in parallel. Summarizing recent works and >> activities as follows: >> >> 0) Currently, we have merged minimum update dependencies for making >> Hadoop JDK-8 compatible(compilable and runnable on JDK-8). >> 1) After that, some people suggest that we should update the other >> dependencies on trunk(e.g. protobuf, netty, jackthon etc.). >> 2) In parallel, Sangjin and Sean are working on classpath isolation: >> HADOOP-13070, HADOOP-11804 and HADOOP-11656. >> >> Main problems we try to solve in the activities above is as follows: >> >> * 1) tries to solve dependency hell between user-level jar and >> system(Hadoop)-level jar. >> * 2) tries to solve updating old libraries. >> >> IIUC, 1) and 2) looks not related, but it's related in fact. 2) tries >> to separate class loader between client-side dependencies and >> server-side dependencies in Hadoop, so we can the change policy of >> updating libraries after doing 2). We can also decide which libraries >> can be shaded after 2). >> >> Hence, IMHO, a straight way we should go to is doing 2 at first. >> After that, we can update both client-side and server-side >> dependencies based on new policy(maybe we should discuss what kind of >> incompatibility is acceptable, and the others are not). >> >> Thoughts? >> >> Thanks, >> - Tsuyoshi >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org >> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org >> > > > > -- > busbey > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org >
--------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
