[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170412238 ## File path: pom.xml ## @@ -128,7 +128,9 @@ 3.0.5 1 +2 Review comment: There is a build extension mentioned in the comments of that issue which could be a good workaround to creating a third profile. It provides a mechanism to add a scripted expression to determine if a profile should be active or not. It looks promising, but I'd want to test it in Eclipse M2E (and maybe other IDEs which support Maven) first to see if the correct profile activates by default there also. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170411383 ## File path: core/pom.xml ## @@ -282,5 +277,50 @@ + + hadoop2 + + + Review comment: Not sure this comment makes sense without some explanation as to why we were expecting activeByDefault to behave differently than it was documented to behave. Might be better to just say that 'this profile is used when not otherwise specified'. (EDIT: might also want to borrow from previous wording when we had Hadoop 1 and 2 profile) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the
hadoop3 shaded jars
URL: https://github.com/apache/accumulo/pull/332#discussion_r170411239
##
File path: assemble/bin/bootstrap_config.sh
##
@@ -383,6 +394,25 @@ elif [[ "${HADOOP_VERSION}" == "IOP4.1" ]]; then
sed -e 's///-->/' \
"${CONF_DIR}/$ACCUMULO_SITE" > temp
+ sed -e 's///-->/' \
+ "${CONF_DIR}/$ACCUMULO_SITE" > temp
+ mv temp "${CONF_DIR}/$ACCUMULO_SITE"
+elif [[ "${HADOOP_VERSION}" == "3" ]]; then
+ sed -e 's///-->/' \
+ "${CONF_DIR}/$ACCUMULO_SITE" > temp
+ sed -e 's///-->/' \
+ "${CONF_DIR}/$ACCUMULO_SITE" > temp
Review comment:
This will clobber the contents of temp from the previous sed execution. It's
too bad we didn't just move the file once, and do all the sed calls with the
in-place `-i` flag (but not worth changing in this PR). (EDIT: that is... not
worth changing to use `-i`; the clobbering still needs to be fixed)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170411152 ## File path: .travis.yml ## @@ -34,4 +34,6 @@ jdk: before_script: - rm -rf $HOME/.m2/repository/org/apache/accumulo - echo "secure-centralhttps://repo.maven.apache.org/maven2central" > $HOME/settings-custom.xml -script: mvn --settings $HOME/settings-custom.xml clean verify javadoc:jar -DskipITs +script: + - mvn --settings $HOME/settings-custom.xml clean verify javadoc:jar -DskipITs + - mvn --settings $HOME/settings-custom.xml clean verify javadoc:jar -DskipITs -Dhadoop.profile=3 Review comment: From the Travis docs, this looks like it will always run both, but fail the build if either task fails. That sounds like what we want... it's just unfortunate that Travis runs these sequentially instead of concurrently, and that you have to scroll down half the page to find where one stops and the other begins. Not much we can do about that, I guess. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170411664 ## File path: assemble/pom.xml ## @@ -323,5 +323,67 @@ + + hadoop2 + + + + !hadoop.profile + + + + commons-math3,commons-vfs2,gson,guava,htrace-core,javax.servlet-api,jcommander,jetty-http,jetty-io,jetty-security,jetty-server,jetty-servlet,jetty-util,jline,libthrift,protobuf-java,slf4j-api,slf4j-log4j12 + src/main/assemblies/binary-release.xml + + + + hadoop3 + + + hadoop.profile + 3 + + + + commons-cli,commons-codec,commons-collections,commons-configuration,commons-io,commons-lang,commons-logging,commons-math3,commons-vfs2,gson,guava,htrace-core,htrace-core4,javax.servlet-api,jcommander,jetty-http,jetty-io,jetty-security,jetty-server,jetty-servlet,jetty-util,jline,libthrift,protobuf-java,slf4j-api,slf4j-log4j12 + src/main/assemblies/binary-release-hadoop3.xml + + Review comment: Do these additional bundled jars affect the LICENSE/NOTICE files of our binary tarball in any way? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170411518 ## File path: minicluster/pom.xml ## @@ -154,4 +145,51 @@ + + + hadoop2 + + + + !hadoop.profile + + + + + org.apache.hadoop + hadoop-client + + + org.apache.hadoop + hadoop-minicluster + true + + + + + hadoop3 + + + hadoop.profile + 3 + + + + + org.apache.hadoop + hadoop-client-api + + + org.apache.hadoop + hadoop-client-minicluster + true Review comment: I'm actually very curious whether this optional has any effect, since it's not clear to me how transitive dependency resolution actually happens for dependencies declared in profiles (and we're not generating "effective" POMs based on profiles). It shouldn't hurt, though. I'll make a personal note to investigate this, unless somebody else knows and can educate me. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170411608 ## File path: pom.xml ## @@ -128,7 +128,9 @@ 3.0.5 1 +2 Review comment: I think this will actually disable the hadoop2 profile, because of `!hadoop.profile`. Because of https://issues.apache.org/jira/browse/MNG-3328 , when we had a 1 and 2 profile, we had to create a third hadoop default profile which had `!hadoop.profile` and was identical (duplicated) to the hadoop2 profile. It was tedious, but I think it might be the only workaround to MNG-3328. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170411271 ## File path: assemble/pom.xml ## @@ -256,6 +234,28 @@ + +org.apache.maven.plugins +maven-dependency-plugin Review comment: Not a huge deal, but this plugin was reordered, making the diff bigger than it needs to be. Might make sense to move it back above exec-maven-plugin. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170411383 ## File path: core/pom.xml ## @@ -282,5 +277,50 @@ + + hadoop2 + + + Review comment: Not sure this comment makes sense without some explanation as to why we were expecting activeByDefault to behave differently than it was documented to behave. Might be better to just say that 'this profile is used when not otherwise specified'. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the
hadoop3 shaded jars
URL: https://github.com/apache/accumulo/pull/332#discussion_r170411239
##
File path: assemble/bin/bootstrap_config.sh
##
@@ -383,6 +394,25 @@ elif [[ "${HADOOP_VERSION}" == "IOP4.1" ]]; then
sed -e 's///-->/' \
"${CONF_DIR}/$ACCUMULO_SITE" > temp
+ sed -e 's///-->/' \
+ "${CONF_DIR}/$ACCUMULO_SITE" > temp
+ mv temp "${CONF_DIR}/$ACCUMULO_SITE"
+elif [[ "${HADOOP_VERSION}" == "3" ]]; then
+ sed -e 's///-->/' \
+ "${CONF_DIR}/$ACCUMULO_SITE" > temp
+ sed -e 's///-->/' \
+ "${CONF_DIR}/$ACCUMULO_SITE" > temp
Review comment:
This will clobber the contents of temp from the previous sed execution. It's
too bad we didn't just move the file once, and do all the sed calls with the
in-place `-i` flag (but not worth changing in this PR).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
Accumulo-Pull-Requests - Build # 1062 - Fixed
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #1062) Status: Fixed Check console output at https://builds.apache.org/job/Accumulo-Pull-Requests/1062/ to view the results.
Accumulo-Pull-Requests - Build # 1061 - Unstable
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #1061) Status: Unstable Check console output at https://builds.apache.org/job/Accumulo-Pull-Requests/1061/ to view the results.
[GitHub] joshelser commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
joshelser commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170408222 ## File path: server/gc/pom.xml ## @@ -87,4 +83,40 @@ test + + + hadoop2 + + + hadoop.profile + 2 + + + + + org.apache.hadoop + hadoop-client + + + + + hadoop3 + + + hadoop.profile + 3 + + + + + org.apache.hadoop + hadoop-client-api + + + org.apache.hadoop + hadoop-client-runtime Review comment: The only thing in the hadoop-client-runtime jar is relocated dependency classes -- no Hadoop code at all. So, no, it's presently 100% guaranteed that we're not using anything from it. Runtime sounds correct. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] joshelser commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
joshelser commented on a change in pull request #332: ACCUMULO-4826 Use the
hadoop3 shaded jars
URL: https://github.com/apache/accumulo/pull/332#discussion_r170407578
##
File path: assemble/pom.xml
##
@@ -323,5 +283,147 @@
+
+ hadoop2
+
+
+ hadoop.profile
+ 2
+
+
+
+
+
+org.apache.maven.plugins
+maven-dependency-plugin
+
+
+
+generate-dependencies-raw
+
+ list
+
+compile
+
+
${project.build.directory}/dependencies.raw.txt
+ false
+ true
+
+
commons-math3,commons-vfs2,gson,guava,htrace-core,javax.servlet-api,jcommander,jetty-http,jetty-io,jetty-security,jetty-server,jetty-servlet,jetty-util,jline,libthrift,protobuf-java,slf4j-api,slf4j-log4j12
+ true
+
+
+
+
+
+org.apache.maven.plugins
+maven-assembly-plugin
+
+
+binary-assembly
+
+ single
+
+package
+
+
+
src/main/assemblies/binary-release.xml
+
+
+
+
+
+
+
+
+
+ hadoop3
+
+
+ hadoop.profile
+ 3
+
+
+
+
+
+ commons-cli
+ commons-cli
+
+
+ commons-codec
+ commons-codec
+
+
+ commons-collections
+ commons-collections
+
+
+ commons-configuration
+ commons-configuration
+
+
+ commons-io
+ commons-io
+
+
+ commons-lang
+ commons-lang
+
+
+ commons-logging
+ commons-logging
+
+
+ org.apache.htrace
+ htrace-core4
+
+
+
+
+
+org.apache.maven.plugins
+maven-dependency-plugin
+
+
+
+generate-dependencies-raw
Review comment:
> I think it would make more sense to put the included artifact list and the
assembly descriptor in a property set by the profile
Easy enough
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
Accumulo-1.8 - Build # 272 - Unstable
The Apache Jenkins build system has built Accumulo-1.8 (build #272) Status: Unstable Check console output at https://builds.apache.org/job/Accumulo-1.8/272/ to view the results.
[GitHub] ctubbsii commented on issue #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on issue #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#issuecomment-368159110 We could update the travis configuration to execute once with hadoop2, and once with hadoop3. It'd be useful to see both. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170385390 ## File path: server/gc/pom.xml ## @@ -87,4 +83,40 @@ test + + + hadoop2 + + + hadoop.profile + 2 + + + + + org.apache.hadoop + hadoop-client + + + + + hadoop3 + + + hadoop.profile + 3 + + + + + org.apache.hadoop + hadoop-client-api + + + org.apache.hadoop + hadoop-client-runtime Review comment: Should these have scope runtime? I wonder if we're using stuff that's now in this jar... if so, that seems bad and we should avoid that if possible. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170382302 ## File path: assemble/bin/bootstrap_config.sh ## @@ -24,7 +24,7 @@ where options include (long options not available on all platforms): -n, --native Configure to use native libraries -j, --jvmConfigure to use the jvm -o, --overwrite Overwrite the default config directory --v, --versionSpecify the Apache Hadoop version supported versions: '1' '2' +-v, --versionSpecify the Apache Hadoop version supported versions: '1' '2' '3' Review comment: Ha! We don't actually support version 1 anymore. Probably need to remove that from the list. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the
hadoop3 shaded jars
URL: https://github.com/apache/accumulo/pull/332#discussion_r170384404
##
File path: assemble/pom.xml
##
@@ -323,5 +283,147 @@
+
+ hadoop2
+
+
+ hadoop.profile
+ 2
+
+
+
+
+
+org.apache.maven.plugins
+maven-dependency-plugin
+
+
+
+generate-dependencies-raw
+
+ list
+
+compile
+
+
${project.build.directory}/dependencies.raw.txt
+ false
+ true
+
+
commons-math3,commons-vfs2,gson,guava,htrace-core,javax.servlet-api,jcommander,jetty-http,jetty-io,jetty-security,jetty-server,jetty-servlet,jetty-util,jline,libthrift,protobuf-java,slf4j-api,slf4j-log4j12
+ true
+
+
+
+
+
+org.apache.maven.plugins
+maven-assembly-plugin
+
+
+binary-assembly
+
+ single
+
+package
+
+
+
src/main/assemblies/binary-release.xml
+
+
+
+
+
+
+
+
+
+ hadoop3
+
+
+ hadoop.profile
+ 3
+
+
+
+
+
+ commons-cli
+ commons-cli
+
+
+ commons-codec
+ commons-codec
+
+
+ commons-collections
+ commons-collections
+
+
+ commons-configuration
+ commons-configuration
+
+
+ commons-io
+ commons-io
+
+
+ commons-lang
+ commons-lang
+
+
+ commons-logging
+ commons-logging
+
+
+ org.apache.htrace
+ htrace-core4
+
+
+
+
+
+org.apache.maven.plugins
+maven-dependency-plugin
+
+
+
+generate-dependencies-raw
Review comment:
Since this task, and the assembly plugin are both executed regardless... but
with different config, I think it would make more sense to put the included
artifact list and the assembly descriptor in a property set by the profile.
That would reduce the redundancy in the plugin configuration.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars
ctubbsii commented on a change in pull request #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#discussion_r170384652 ## File path: core/pom.xml ## @@ -282,5 +277,50 @@ + + hadoop2 + +true Review comment: These still have the activeByDefault flag, which must be removed, because it does not do what you think. :) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (ACCUMULO-4826) Support Hadoop 3
[ https://issues.apache.org/jira/browse/ACCUMULO-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-4826: Fix Version/s: 2.0.0 > Support Hadoop 3 > > > Key: ACCUMULO-4826 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4826 > Project: Accumulo > Issue Type: New Feature >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] joshelser commented on issue #332: ACCUMULO-4826 Use the hadoop3 shaded jars
joshelser commented on issue #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#issuecomment-368140983 Hrm, I broke hadoop2 compilation? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (ACCUMULO-4826) Support Hadoop 3
[ https://issues.apache.org/jira/browse/ACCUMULO-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ACCUMULO-4826: - Labels: pull-request-available (was: ) > Support Hadoop 3 > > > Key: ACCUMULO-4826 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4826 > Project: Accumulo > Issue Type: New Feature >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Accumulo-Pull-Requests - Build # 1060 - Still Failing
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #1060) Status: Still Failing Check console output at https://builds.apache.org/job/Accumulo-Pull-Requests/1060/ to view the results.
[GitHub] joshelser commented on issue #332: ACCUMULO-4826 Use the hadoop3 shaded jars
joshelser commented on issue #332: ACCUMULO-4826 Use the hadoop3 shaded jars URL: https://github.com/apache/accumulo/pull/332#issuecomment-368138112 This seems to be OK for me. Ran TestIngest locally against Apache Hadoop 3.0.0. Big changes that were required: * Start bundling all of the commons-\* jars we already depending on but expected Hadoop to give us (yay, wart removal) * Also have to include htrace-core4 as this one isn't shaded by Hadoop as it would break. htrace3 and 4 are currently bundled because it's going to be a re-working of our entire tracing infra to support htrace4 (I think). I've punted on it. * bootstrap_configuration.sh is updated, as is the template for Hadoop3 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (ACCUMULO-4826) Support Hadoop 3
[ https://issues.apache.org/jira/browse/ACCUMULO-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374952#comment-16374952 ] Josh Elser commented on ACCUMULO-4826: -- ACCUMULO-4171 could also happen now, but this is a pretty brutally invasive change to do. > Support Hadoop 3 > > > Key: ACCUMULO-4826 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4826 > Project: Accumulo > Issue Type: New Feature >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ACCUMULO-4826) Support Hadoop 3
Josh Elser created ACCUMULO-4826: Summary: Support Hadoop 3 Key: ACCUMULO-4826 URL: https://issues.apache.org/jira/browse/ACCUMULO-4826 Project: Accumulo Issue Type: New Feature Reporter: Josh Elser Assignee: Josh Elser Fix For: 1.9.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ACCUMULO-4825) Combine fate/core modules (and maybe start)
Christopher Tubbs created ACCUMULO-4825: --- Summary: Combine fate/core modules (and maybe start) Key: ACCUMULO-4825 URL: https://issues.apache.org/jira/browse/ACCUMULO-4825 Project: Accumulo Issue Type: Improvement Components: build, core, fate, start Reporter: Christopher Tubbs Assignee: Christopher Tubbs Fix For: 2.0.0 The fate jar, and possibly the start jar, could be merged into the core jar. Neither can really be used on their own for anything, are not public API, cannot be substituted with any pluggable alternative, should not be exposed directly to users, and aren't substantively different than other common utility code throughout the core jar. There might be some classloader issues or other problems which would prevent the start jar from being included, but the fate jar should be fine to roll in. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ACCUMULO-4611) Provide a migration path to commons-configuration2 to better support Hadoop 3
[
https://issues.apache.org/jira/browse/ACCUMULO-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374670#comment-16374670
]
Christopher Tubbs commented on ACCUMULO-4611:
-
bq. With the change you have already committed, things appear to be fine for
H3. Just assembly-descriptor tweaks are needed.
Great. Please let me know if there's any way I can be of help. As you may have
seen, I reopened the duplicate ticket (ACCUMULO-4753) to track the remaining
follow-on work.
> Provide a migration path to commons-configuration2 to better support Hadoop 3
> -
>
> Key: ACCUMULO-4611
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4611
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.7.2, 1.8.1
> Environment: CentOS 7
> Accumulo 1.7.x, 1.8.x
> Hadoop 3.0.0-alpha 2
> Zookeeper 3.4.9
>Reporter: Michael Hogue
>Assignee: Christopher Tubbs
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.9.0, 2.0.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> I was investigating running either Accumulo 1.7.x or 1.8.x on Hadoop
> 3.0.0-alpha 2 and ran into a couple of issues. Since Accumulo assumes some
> dependencies will be provided by Hadoop (per ACCUMULO-1244), if those
> dependencies change then Accumulo might also need to change.
> HADOOP-13660 bumped {{commons-configuration}} from 1.6 -> 2.1 for Hadoop
> 3.0.0 and so NoClassDefFoundErrors are thrown per the below stack trace on
> {{accumulo init}}:
> {noformat}
> [mike@localhost cloud]$ accumulo init
> 2017-03-22 15:38:34,106 [start.Main] ERROR: Uncaught exception
> java.util.ServiceConfigurationError:
> org.apache.accumulo.start.spi.KeywordExecutable: Provider
> org.apache.accumulo.proxy.Proxy could not be instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at org.apache.accumulo.start.Main.checkDuplicates(Main.java:223)
> at org.apache.accumulo.start.Main.getExecutables(Main.java:215)
> at org.apache.accumulo.start.Main.main(Main.java:78)
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/commons/configuration/Configuration
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.newInstance(Class.java:412)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.configuration.Configuration
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
> org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:284)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 10 more
> {noformat}
> I worked around this by dropping the {{commons-configuration}} 1.6 jar on
> Accumulo's classpath, but this should either be provided by Accumulo now or
> Accumulo should be bumped to {{commons-configuration2}} as well. Note that
> the latter change would cause problems for Hadoop 2 installations.
> There was actually one more thing that needed added to the accumulo-site.xml
> in order to get Accumulo to run in Hadoop 3:
> {noformat}
> $HADOOP_PREFIX/share/hadoop/client/[^.].*.jar
> {noformat}
> The reason for this is because {{Hdfs.java}} moved from the hadoop-hdfs jar
> to the hadoop-client-api jar.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ACCUMULO-4611) Provide a migration path to commons-configuration2 to better support Hadoop 3
[
https://issues.apache.org/jira/browse/ACCUMULO-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374650#comment-16374650
]
Josh Elser commented on ACCUMULO-4611:
--
bq. Whether or not it is important for Hadoop 3 compatibility depends on
whether Hadoop is shipping with shaded jars or not, and whether or not Josh
Elser's experiment with Maven profiles in
https://github.com/apache/accumulo/pull/332 is successful at supporting Hadoop
3, and I haven't had any time to follow up with any of that.
I've picked that back up which is why I was trying to figure out what exactly
you had done here.
bq. If it turns out that the migration to commons-configuration2 is essential
for Hadoop 3 compatibility, then I can re-prioritize to get this done, so it
won't hold up any future 2.0 releases
With the change you have already committed, things appear to be fine for H3.
Just assembly-descriptor tweaks are needed.
> Provide a migration path to commons-configuration2 to better support Hadoop 3
> -
>
> Key: ACCUMULO-4611
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4611
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.7.2, 1.8.1
> Environment: CentOS 7
> Accumulo 1.7.x, 1.8.x
> Hadoop 3.0.0-alpha 2
> Zookeeper 3.4.9
>Reporter: Michael Hogue
>Assignee: Christopher Tubbs
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.9.0, 2.0.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> I was investigating running either Accumulo 1.7.x or 1.8.x on Hadoop
> 3.0.0-alpha 2 and ran into a couple of issues. Since Accumulo assumes some
> dependencies will be provided by Hadoop (per ACCUMULO-1244), if those
> dependencies change then Accumulo might also need to change.
> HADOOP-13660 bumped {{commons-configuration}} from 1.6 -> 2.1 for Hadoop
> 3.0.0 and so NoClassDefFoundErrors are thrown per the below stack trace on
> {{accumulo init}}:
> {noformat}
> [mike@localhost cloud]$ accumulo init
> 2017-03-22 15:38:34,106 [start.Main] ERROR: Uncaught exception
> java.util.ServiceConfigurationError:
> org.apache.accumulo.start.spi.KeywordExecutable: Provider
> org.apache.accumulo.proxy.Proxy could not be instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at org.apache.accumulo.start.Main.checkDuplicates(Main.java:223)
> at org.apache.accumulo.start.Main.getExecutables(Main.java:215)
> at org.apache.accumulo.start.Main.main(Main.java:78)
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/commons/configuration/Configuration
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.newInstance(Class.java:412)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.configuration.Configuration
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
> org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:284)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 10 more
> {noformat}
> I worked around this by dropping the {{commons-configuration}} 1.6 jar on
> Accumulo's classpath, but this should either be provided by Accumulo now or
> Accumulo should be bumped to {{commons-configuration2}} as well. Note that
> the latter change would cause problems for Hadoop 2 installations.
> There was actually one more thing that needed added to the accumulo-site.xml
> in order to get Accumulo to run in Hadoop 3:
> {noformat}
> $HADOOP_PREFIX/share/hadoop/client/[^.].*.jar
> {noformat}
> The reason for this is because {{Hdfs.java}} moved from the hadoop-hdfs jar
> to the hadoop-client-api jar.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Accumulo-Master - Build # 2272 - Fixed
The Apache Jenkins build system has built Accumulo-Master (build #2272) Status: Fixed Check console output at https://builds.apache.org/job/Accumulo-Master/2272/ to view the results.
Accumulo-Pull-Requests - Build # 1059 - Still Failing
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #1059) Status: Still Failing Check console output at https://builds.apache.org/job/Accumulo-Pull-Requests/1059/ to view the results.
[GitHub] ghajos commented on issue #381: ACCUMULO-4413 Avoid Copy/Sort column on WAL recovery exceeds 100%
ghajos commented on issue #381: ACCUMULO-4413 Avoid Copy/Sort column on WAL recovery exceeds 100% URL: https://github.com/apache/accumulo/pull/381#issuecomment-367972087 Changed max to min. Unfortunately ACCUMULO-4413 is a duplicate of ACCUMULO-3807. Should I create a new pull request for that ticket? Also there are some concerns about this issue under ACCUMULO-3807 this modification should follow, but don't know how yet. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
