I checked the client jars that are supposed to contain shaded dependencies, and they don't look quite right:
$ tar -tzvf hadoop-3.0.0.tar.gz | grep hadoop-client-api-3.0.0.jar -rw-r--r-- 0 andrew andrew 44531 Nov 14 11:53 hadoop-3.0.0/share/hadoop/client/hadoop-client-api-3.0.0.jar $ tar -tzvf hadoop-3.0.0.tar.gz | grep hadoop-client-runtime-3.0.0.jar -rw-r--r-- 0 andrew andrew 45533 Nov 14 11:53 hadoop-3.0.0/share/hadoop/client/hadoop-client-runtime-3.0.0.jar $ tar -tzvf hadoop-3.0.0.tar.gz | grep hadoop-client-minicluster-3.0.0.jar -rw-r--r-- 0 andrew andrew 47015 Nov 14 11:53 hadoop-3.0.0/share/hadoop/client/hadoop-client-minicluster-3.0.0.jar When I look at what's inside those jar, they only seem to include pom-related files with no class files. Am I missing something? When I build from the source with -Pdist, I do get much bigger jars: total 113760 -rw-r--r-- 1 sangjinlee 120039211 17055399 Nov 20 17:17 hadoop-client-api-3.0.0.jar -rw-r--r-- 1 sangjinlee 120039211 20451447 Nov 20 17:19 hadoop-client-minicluster-3.0.0.jar -rw-r--r-- 1 sangjinlee 120039211 20730866 Nov 20 17:18 hadoop-client-runtime-3.0.0.jar Sangjin On Mon, Nov 20, 2017 at 5:52 PM, Sangjin Lee <sj...@apache.org> wrote: > > > On Mon, Nov 20, 2017 at 5:26 PM, Vinod Kumar Vavilapalli < > vino...@apache.org> wrote: > >> Thanks for all the push, Andrew! >> >> Looking at the RC. Went through my usual check-list. Here's my summary. >> Will cast my final vote after comparing and validating my findings with >> others. >> >> Verification >> >> - [Check] Successful recompilation from source tar-ball >> - [Check] Signature verification >> - [Check] Generating dist tarballs from source tar-ball >> - [Check] Testing >> -- Start NN, DN, RM, NM, JHS, Timeline Service >> -- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort, >> grep, pi >> -- Tested CLIs to print nodes, apps etc and also navigated UIs >> >> Issues found during testing >> >> Major >> - The previously supported way of being able to use different tar-balls >> for different sub-modules is completely broken - common and HDFS tar.gz are >> completely empty. >> - Cannot enable new UI in YARN because it is under a non-default >> compilation flag. It should be on by default. >> - One decommissioned node in YARN ResourceManager UI always appears to >> start with, even when there are no NodeManagers that are started yet: Info >> <IP>:-1, DECOMMISSIONED, null rack. It shows up only in the UI though, not >> in the CLI node -list >> >> Minor >> - resourcemanager-metrics.out is going into current directory instead of >> log directory >> - $HADOOP_YARN_HOME/sbin/yarn-daemon.sh start historyserver doesn't >> even work. Not just deprecated in favor of timelineserver as was advertised. >> - Spurious warnings on CLI >> 17/11/20 17:07:34 INFO conf.Configuration: resource-types.xml >> not found >> 17/11/20 17:07:34 INFO resource.ResourceUtils: Unable to find >> 'resource-types.xml'. >> >> Side notes >> >> - When did we stop putting CHANGES files into the source artifacts? >> - Even after "mvn install"ing once, shading is repeated again and again >> for every new 'mvn install' even though there are no source changes - we >> should see how this can be avoided. >> - Compatibility notes >> -- NM's env list is curtailed unlike in 2.x (For e.g, >> HADOOP_MAPRED_HOME is not automatically inherited. Correct behavior) >> -- Sleep is moved from hadoop-mapreduce-client-jobclient-3.0.0.jar >> into hadoop-mapreduce-client-jobclient-3.0.0-tests.jar >> > > Sleep has always been in the jobclient test jar as long as I can remember, > so it's not new for 3.0. > > >> >> Thanks >> +Vinod >> >> > On Nov 14, 2017, at 1:34 PM, Andrew Wang <andrew.w...@cloudera.com> >> wrote: >> > >> > Hi folks, >> > >> > Thanks as always to the many, many contributors who helped with this >> > release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are >> > available here: >> > >> > http://people.apache.org/~wang/3.0.0-RC0/ >> > >> > This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific. >> > >> > 3.0.0 GA contains 291 fixed JIRA issues since 3.0.0-beta1. Notable >> > additions include the merge of YARN resource types, API-based >> configuration >> > of the CapacityScheduler, and HDFS router-based federation. >> > >> > I've done my traditional testing with a pseudo cluster and a Pi job. My >> +1 >> > to start. >> > >> > Best, >> > Andrew >> >> >
