[
https://issues.apache.org/jira/browse/HBASE-5361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Charles updated HBASE-5361:
--------------------------------
Attachment: HBASE-5361-01-NotForInclusion.patch
Here's a patch for the pom that builds on 0.24-SNAPSHOT as default (not
intended for inclusion atm, just sharing the information).
Please note I used the agregator module org.apache.hadoop:hadoop-minicluster.
On one side, this is easy and it protects you from any change in the hadoop
modules.
One the other side, it brings unuseful test dependencies in the distribution
(see *-tests.jar in the following list):
$du -h lib/hadoop*
20K lib/hadoop-annotations-0.24.0-SNAPSHOT.jar
44K lib/hadoop-auth-0.24.0-SNAPSHOT.jar
980K lib/hadoop-common-0.24.0-SNAPSHOT-tests.jar
1.8M lib/hadoop-common-0.24.0-SNAPSHOT.jar
1.1M lib/hadoop-hdfs-0.24.0-SNAPSHOT-tests.jar
3.6M lib/hadoop-hdfs-0.24.0-SNAPSHOT.jar
412K lib/hadoop-mapreduce-client-app-0.24.0-SNAPSHOT.jar
580K lib/hadoop-mapreduce-client-common-0.24.0-SNAPSHOT.jar
1.3M lib/hadoop-mapreduce-client-core-0.24.0-SNAPSHOT.jar
100K lib/hadoop-mapreduce-client-hs-0.24.0-SNAPSHOT.jar
1.2M lib/hadoop-mapreduce-client-jobclient-0.24.0-SNAPSHOT-tests.jar
36K lib/hadoop-mapreduce-client-jobclient-0.24.0-SNAPSHOT.jar
16K lib/hadoop-mapreduce-client-shuffle-0.24.0-SNAPSHOT.jar
4.0K lib/hadoop-minicluster-0.24.0-SNAPSHOT.jar
872K lib/hadoop-yarn-api-0.24.0-SNAPSHOT.jar
1.0M lib/hadoop-yarn-common-0.24.0-SNAPSHOT.jar
132K lib/hadoop-yarn-server-common-0.24.0-SNAPSHOT.jar
380K lib/hadoop-yarn-server-nodemanager-0.24.0-SNAPSHOT.jar
504K lib/hadoop-yarn-server-resourcemanager-0.24.0-SNAPSHOT.jar
36K lib/hadoop-yarn-server-tests-0.24.0-SNAPSHOT-tests.jar
24K lib/hadoop-yarn-server-web-proxy-0.24.0-SNAPSHOT.jar
I am also a bit suprised with the 'test' scope of some dependencies (walk the
following tree..), prolly to be discussed on hadoop ml.
[INFO] \- org.apache.hadoop:hadoop-minicluster:jar:0.24.0-SNAPSHOT:compile
[INFO] +-
org.apache.hadoop:hadoop-common:test-jar:tests:0.24.0-SNAPSHOT:compile
[INFO] | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | +- commons-net:commons-net:jar:1.4.1:compile
[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:compile
[INFO] | +- commons-el:commons-el:jar:1.0:compile
[INFO] | +- commons-logging:commons-logging-api:jar:1.1:compile
[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile
[INFO] | +- hsqldb:hsqldb:jar:1.8.0.7:compile
[INFO] | +- oro:oro:jar:2.0.8:compile
[INFO] | +- org.aspectj:aspectjrt:jar:1.6.5:compile
[INFO] | +- net.sf.kosmosfs:kfs:jar:0.3:compile
[INFO] | +- org.apache.hadoop:hadoop-auth:jar:0.24.0-SNAPSHOT:compile
[INFO] | \- com.googlecode.json-simple:json-simple:jar:1.1:compile
[INFO] +-
org.apache.hadoop:hadoop-hdfs:test-jar:tests:0.24.0-SNAPSHOT:compile
[INFO] | \- commons-daemon:commons-daemon:jar:1.0.3:compile
[INFO] +-
org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:0.24.0-SNAPSHOT:compile
[INFO] | +-
org.apache.hadoop:hadoop-yarn-server-common:jar:0.24.0-SNAPSHOT:compile
[INFO] | +-
org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:0.24.0-SNAPSHOT:compile
[INFO] | +-
org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:0.24.0-SNAPSHOT:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:0.24.0-SNAPSHOT:compile
[INFO] | | \- jdiff:jdiff:jar:1.0.9:compile
[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile
[INFO] | +- com.cenqua.clover:clover:jar:3.0.2:compile
[INFO] | +- com.google.inject:guice:jar:3.0:compile
[INFO] | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile
[INFO] | +-
com.sun.jersey.jersey-test-framework:jersey-test-framework-grizzly2:jar:1.8:compile
[INFO] | | +-
com.sun.jersey.jersey-test-framework:jersey-test-framework-core:jar:1.8:compile
[INFO] | | | +- org.glassfish:javax.servlet:jar:3.0:compile
[INFO] | | | \- com.sun.jersey:jersey-client:jar:1.8:compile
[INFO] | | \- com.sun.jersey:jersey-grizzly2:jar:1.8:compile
[INFO] | | +- org.glassfish.grizzly:grizzly-http:jar:2.1.1:compile
[INFO] | | | \-
org.glassfish.grizzly:grizzly-framework:jar:2.1.1:compile
[INFO] | | | \-
org.glassfish.gmbal:gmbal-api-only:jar:3.0.0-b023:compile
[INFO] | | | \-
org.glassfish.external:management-api:jar:3.0.0-b012:compile
[INFO] | | +-
org.glassfish.grizzly:grizzly-http-server:jar:2.1.1:compile
[INFO] | | | \- org.glassfish.grizzly:grizzly-rcm:jar:2.1.1:compile
[INFO] | | \-
org.glassfish.grizzly:grizzly-http-servlet:jar:2.1.1:compile
[INFO] | | \-
org.glassfish.grizzly:grizzly-framework:jar:tests:2.1.1:compile
[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.8:compile
[INFO] +-
org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:0.24.0-SNAPSHOT:compile
[INFO] | +-
org.apache.hadoop:hadoop-mapreduce-client-common:jar:0.24.0-SNAPSHOT:compile
[INFO] | \-
org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:0.24.0-SNAPSHOT:compile
[INFO] +- org.apache.hadoop:hadoop-common:jar:0.24.0-SNAPSHOT:compile
[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:0.24.0-SNAPSHOT:compile
[INFO] +-
org.apache.hadoop:hadoop-mapreduce-client-app:jar:0.24.0-SNAPSHOT:compile
[INFO] | \-
org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:0.24.0-SNAPSHOT:compile
[INFO] +- org.apache.hadoop:hadoop-yarn-api:jar:0.24.0-SNAPSHOT:compile
[INFO] +-
org.apache.hadoop:hadoop-mapreduce-client-core:jar:0.24.0-SNAPSHOT:compile
[INFO] | \- org.apache.hadoop:hadoop-yarn-common:jar:0.24.0-SNAPSHOT:compile
[INFO] +-
org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:0.24.0-SNAPSHOT:compile
[INFO] \-
org.apache.hadoop:hadoop-mapreduce-client-hs:jar:0.24.0-SNAPSHOT:compile
> Build and test on Hadoop 0.24.0-SNAPSHOT
> ----------------------------------------
>
> Key: HBASE-5361
> URL: https://issues.apache.org/jira/browse/HBASE-5361
> Project: HBase
> Issue Type: New Feature
> Components: build
> Reporter: Eric Charles
> Priority: Minor
> Attachments: HBASE-5361-01-NotForInclusion.patch
>
>
> I updated hbase pom to let it depend on hadoop 0.24.0-SNAPSHOT.
> mvn test -PrunMediumTests is nearly successful... (TestImportExport,
> TestImportTsv and TestStore fail).
> I run the process on my laptop (in 'fully' distributed mode), and interaction
> via console is fine.
> 70508 NameNode
> 70719 SecondaryNameNode
> 70601 DataNode
> 70896 NodeManager
> 70796 ResourceManager
> 72715 HMaster
> 72777 HRegionServer
> 72677 HQuorumPeer
> I will further look for the test failure and will try hbase mapreduce
> functions (the 0.24 hadoop bigbang is on mapreduce, not on hdfs).
> Thx, Eric
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
