[jira] [Commented] (HBASE-28260) Possible data loss in WAL after RegionServer crash
[
https://issues.apache.org/jira/browse/HBASE-28260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796845#comment-17796845
]
Sean Busbey commented on HBASE-28260:
-
I don't think HDFS defines when a reader in the same client can expect to see
writes into file, so we're already relying on some shady business in that part
of replication. but yeah I would expect an increase in the latency between RS
acks a write and the replication system
can see it.
bq. I think that's exactly what this change will do. During reconstruction,
the NameNode uses one of the existing blocks as the primary. So if we set this
flag, the local DataNode should not be an option there.
Looking at this definition, I think that's only indirectly true because HDFS
will "try" to not have any of the blocks be stored on the DN that's on the same
host as the RS:
https://hadoop.apache.org/docs/r3.3.6/api/org/apache/hadoop/fs/CreateFlag.html#NO_LOCAL_WRITE
I think in the case where we have count of DNs <= WAL Replication factor we're
still going to get a local write and then the recovery process is still going
to choose that write for primary since it's local.
bq. Maybe you are speaking to the later phase of reconstruction when the new
block is replicated to 2 other datanodes? Not sure we can control that.
I'm talking about when reconstruction is choosing which which existing block is
primary it'd be nice if we could hint to HDFS that it should avoid picking that
block as primary. rather than having to avoid a local write all together.
> Possible data loss in WAL after RegionServer crash
> --
>
> Key: HBASE-28260
> URL: https://issues.apache.org/jira/browse/HBASE-28260
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
>
> We recently had a production incident:
> # RegionServer crashes, but local DataNode lives on
> # WAL lease recovery kicks in
> # Namenode reconstructs the block during lease recovery (which results in a
> new genstamp). It chooses the replica on the local DataNode as the primary.
> # Local DataNode reconstructs the block, so NameNode registers the new
> genstamp.
> # Local DataNode and the underlying host dies, before the new block could be
> replicated to other replicas.
> This leaves us with a missing block, because the new genstamp block has no
> replicas. The old replicas still remain, but are considered corrupt due to
> GENSTAMP_MISMATCH.
> Thankfully we were able to confirm that the length of the corrupt blocks were
> identical to the newly constructed and lost block. Further, the file in
> question was only 1 block. So we downloaded one of those corrupt block files
> and hdfs {{hdfs dfs -put -f}} to force that block to replace the file in
> hdfs. So in this case we had no actual data loss, but it could have happened
> easily if the file was more than 1 block or the replicas weren't fully in
> sync prior to reconstruction.
> In order to avoid this issue, we should avoid writing WAL blocks too the
> local datanode. We can use CreateFlag.NO_WRITE_LOCAL for this. Hat tip to
> [~weichiu] for pointing this out.
> During reading of WALs we already reorder blocks so as to avoid reading from
> the local datanode, but avoiding writing there altogether would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28260) Possible data loss in WAL after RegionServer crash
[
https://issues.apache.org/jira/browse/HBASE-28260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796819#comment-17796819
]
Sean Busbey commented on HBASE-28260:
-
I assume there's no way to tell HDFS on write pipeline creation that it should
also favor avoiding use of the specific DataNode that's colocated when doing
block reconstruction?
> Possible data loss in WAL after RegionServer crash
> --
>
> Key: HBASE-28260
> URL: https://issues.apache.org/jira/browse/HBASE-28260
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
>
> We recently had a production incident:
> # RegionServer crashes, but local DataNode lives on
> # WAL lease recovery kicks in
> # Namenode reconstructs the block during lease recovery (which results in a
> new genstamp). It chooses the replica on the local DataNode as the primary.
> # Local DataNode reconstructs the block, so NameNode registers the new
> genstamp.
> # Local DataNode and the underlying host dies, before the new block could be
> replicated to other replicas.
> This leaves us with a missing block, because the new genstamp block has no
> replicas. The old replicas still remain, but are considered corrupt due to
> GENSTAMP_MISMATCH.
> Thankfully we were able to confirm that the length of the corrupt blocks were
> identical to the newly constructed and lost block. Further, the file in
> question was only 1 block. So we downloaded one of those corrupt block files
> and hdfs {{hdfs dfs -put -f}} to force that block to replace the file in
> hdfs. So in this case we had no actual data loss, but it could have happened
> easily if the file was more than 1 block or the replicas weren't fully in
> sync prior to reconstruction.
> In order to avoid this issue, we should avoid writing WAL blocks too the
> local datanode. We can use CreateFlag.NO_WRITE_LOCAL for this. Hat tip to
> [~weichiu] for pointing this out.
> During reading of WALs we already reorder blocks so as to avoid reading from
> the local datanode, but avoiding writing there altogether would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-20448) update ref guide to expressly use shaded clients for examples
[ https://issues.apache.org/jira/browse/HBASE-20448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned HBASE-20448: --- Assignee: (was: Sean Busbey) > update ref guide to expressly use shaded clients for examples > - > > Key: HBASE-20448 > URL: https://issues.apache.org/jira/browse/HBASE-20448 > Project: HBase > Issue Type: Improvement > Components: Client, documentation, mapreduce >Reporter: Sean Busbey >Priority: Blocker > Fix For: 3.0.0-alpha-4 > > > the whole mapreduce section, especially, should be using the shaded version. > Specifically include an example of running the ITBLL we ship -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-22924) GitHUB PR job should use when clause to filter to just PRs.
[
https://issues.apache.org/jira/browse/HBASE-22924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643430#comment-17643430
]
Sean Busbey commented on HBASE-22924:
-
that interpretation is correct Duo. this ticket is for a) adding to the
jenkinsfile the needed configs to skip non-PR builds AND b) remove the current
shell script clause that skips non-PR builds.
this ticket would also be obsolete if someone were to combine our build
pipelines for PRs with the ones for nightlies. But that would be a lot more
work.
> GitHUB PR job should use when clause to filter to just PRs.
> ---
>
> Key: HBASE-22924
> URL: https://issues.apache.org/jira/browse/HBASE-22924
> Project: HBase
> Issue Type: Improvement
> Components: build, community
>Reporter: Sean Busbey
>Assignee: Shanky Sharma
>Priority: Minor
> Labels: beginner
>
> Right now the GitHub PR checking job uses a Jenkins environment variable to
> stop branch builds:
> {code}
> # If CHANGE_URL is set (e.g., Github Branch Source
> plugin), process it.
> # Otherwise exit, because we don't want HBase to do a
> # full build. We wouldn't normally do this check for
> smaller
> # projects. :)
> if [[ -z "${CHANGE_URL}" ]]; then
> echo "Full build skipped" >
> "${WORKSPACE}/${PATCHDIR}/report.html"
> exit 0
> fi
> {code}
> instead we should use a Jenkinsfile {{when}} clause
> ([ref|https://jenkins.io/doc/book/pipeline/syntax/#when]) and the
> {{changeRequest()}} filter to say we want to build PRs. Assuming the eventual
> adaptation of the JIRA precommit job properly sets that it's testing a change
> this should also help reuse there.
> {code}
> stage ('precommit-run') {
> when { changeRequest() }
> steps {
> ...
> }
> }
> {code}
> it would also be nice to fold things into a single stage so that we skip the
> yetus checkout when we're not going to run tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27425) Run flaky test job more often
[ https://issues.apache.org/jira/browse/HBASE-27425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17622331#comment-17622331 ] Sean Busbey commented on HBASE-27425: - thanks for pushing the change Duo! > Run flaky test job more often > - > > Key: HBASE-27425 > URL: https://issues.apache.org/jira/browse/HBASE-27425 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.2, 2.4.16 > > Attachments: ci-hbase-hbase-flaky-test-master-time-png.png, > ci-hbase-hbase-load-chart-long.png > > > We currently run the flaky tests once per 4 hours and keep a backlog of 50 > runs. That means it takes a perfect test a bit over 8 days to get back out of > being penalized. > we regularly are leaving a bunch of executors idle (see attached PNG) and I > think we should run the tests more often. > we use an executor per branch for a flaky test run. the duration depends on > the specific tests that are flagged, but it has a timeout of 2 hours. > as an example the master branch tests currently take < 30 minutes (see other > attached PNG). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27425) Run flaky test job more often
[ https://issues.apache.org/jira/browse/HBASE-27425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-27425: Status: Patch Available (was: In Progress) > Run flaky test job more often > - > > Key: HBASE-27425 > URL: https://issues.apache.org/jira/browse/HBASE-27425 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Attachments: ci-hbase-hbase-flaky-test-master-time-png.png, > ci-hbase-hbase-load-chart-long.png > > > We currently run the flaky tests once per 4 hours and keep a backlog of 50 > runs. That means it takes a perfect test a bit over 8 days to get back out of > being penalized. > we regularly are leaving a bunch of executors idle (see attached PNG) and I > think we should run the tests more often. > we use an executor per branch for a flaky test run. the duration depends on > the specific tests that are flagged, but it has a timeout of 2 hours. > as an example the master branch tests currently take < 30 minutes (see other > attached PNG). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-27425) Run flaky test job more often
[ https://issues.apache.org/jira/browse/HBASE-27425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-27425 started by Sean Busbey. --- > Run flaky test job more often > - > > Key: HBASE-27425 > URL: https://issues.apache.org/jira/browse/HBASE-27425 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Attachments: ci-hbase-hbase-flaky-test-master-time-png.png, > ci-hbase-hbase-load-chart-long.png > > > We currently run the flaky tests once per 4 hours and keep a backlog of 50 > runs. That means it takes a perfect test a bit over 8 days to get back out of > being penalized. > we regularly are leaving a bunch of executors idle (see attached PNG) and I > think we should run the tests more often. > we use an executor per branch for a flaky test run. the duration depends on > the specific tests that are flagged, but it has a timeout of 2 hours. > as an example the master branch tests currently take < 30 minutes (see other > attached PNG). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27425) Run flaky test job more often
[ https://issues.apache.org/jira/browse/HBASE-27425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17616640#comment-17616640 ] Sean Busbey commented on HBASE-27425: - let's try once an hour and see how the executor load goes. that would give us a floor for test clearing of just over 2 days instead of 8. > Run flaky test job more often > - > > Key: HBASE-27425 > URL: https://issues.apache.org/jira/browse/HBASE-27425 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Attachments: ci-hbase-hbase-flaky-test-master-time-png.png, > ci-hbase-hbase-load-chart-long.png > > > We currently run the flaky tests once per 4 hours and keep a backlog of 50 > runs. That means it takes a perfect test a bit over 8 days to get back out of > being penalized. > we regularly are leaving a bunch of executors idle (see attached PNG) and I > think we should run the tests more often. > we use an executor per branch for a flaky test run. the duration depends on > the specific tests that are flagged, but it has a timeout of 2 hours. > as an example the master branch tests currently take < 30 minutes (see other > attached PNG). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-27425) Run flaky test job more often
[ https://issues.apache.org/jira/browse/HBASE-27425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned HBASE-27425: --- Assignee: Sean Busbey > Run flaky test job more often > - > > Key: HBASE-27425 > URL: https://issues.apache.org/jira/browse/HBASE-27425 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Attachments: ci-hbase-hbase-flaky-test-master-time-png.png, > ci-hbase-hbase-load-chart-long.png > > > We currently run the flaky tests once per 4 hours and keep a backlog of 50 > runs. That means it takes a perfect test a bit over 8 days to get back out of > being penalized. > we regularly are leaving a bunch of executors idle (see attached PNG) and I > think we should run the tests more often. > we use an executor per branch for a flaky test run. the duration depends on > the specific tests that are flagged, but it has a timeout of 2 hours. > as an example the master branch tests currently take < 30 minutes (see other > attached PNG). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27425) Run flaky test job more often
Sean Busbey created HBASE-27425: --- Summary: Run flaky test job more often Key: HBASE-27425 URL: https://issues.apache.org/jira/browse/HBASE-27425 Project: HBase Issue Type: Task Components: test Reporter: Sean Busbey Attachments: ci-hbase-hbase-flaky-test-master-time-png.png, ci-hbase-hbase-load-chart-long.png We currently run the flaky tests once per 4 hours and keep a backlog of 50 runs. That means it takes a perfect test a bit over 8 days to get back out of being penalized. we regularly are leaving a bunch of executors idle (see attached PNG) and I think we should run the tests more often. we use an executor per branch for a flaky test run. the duration depends on the specific tests that are flagged, but it has a timeout of 2 hours. as an example the master branch tests currently take < 30 minutes (see other attached PNG). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27328) Enforcer phase EvaluateBeanShell fails occasionally
[
https://issues.apache.org/jira/browse/HBASE-27328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600817#comment-17600817
]
Sean Busbey commented on HBASE-27328:
-
(the enforcer plugin has been marked threadsafe since 1.0.1 and it looks like
the beanshell evaluator is threadsafe.)
> Enforcer phase EvaluateBeanShell fails occasionally
> ---
>
> Key: HBASE-27328
> URL: https://issues.apache.org/jira/browse/HBASE-27328
> Project: HBase
> Issue Type: Task
> Components: build
>Reporter: Nick Dimiduk
>Priority: Major
>
> For example, from
> https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/412/General_20Nightly_20Build_20Report/
> {noformat}
> [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @
> hbase-shaded-client-byo-hadoop ---
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.EvaluateBeanshell failed
> with message:
> Couldn't evaluate condition: File license = new
> File("/home/jenkins/jenkins-home/workspace/HBase_Nightly_branch-2.4/component/hbase-shaded/hbase-shaded-client/target/maven-shared-archive-resources/META-INF/LICENSE");
> // Beanshell does not support try-with-resources,
> // so we must close this scanner manually
> Scanner scanner = new Scanner(license);
> while (scanner.hasNextLine()) {
> if (scanner.nextLine().startsWith("ERROR:")) {
> scanner.close();
> return false;
> }
> }
> scanner.close();
> return true;
> [INFO] No sources to compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27328) Enforcer phase EvaluateBeanShell fails occasionally
[
https://issues.apache.org/jira/browse/HBASE-27328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600815#comment-17600815
]
Sean Busbey commented on HBASE-27328:
-
can we reproduce this with a `-X` on maven? the source for the beanshell
evaluation should include the exception that caused the failure:
https://github.com/apache/maven-enforcer/blob/master/enforcer-rules/src/main/java/org/apache/maven/plugins/enforcer/EvaluateBeanshell.java#L114
> Enforcer phase EvaluateBeanShell fails occasionally
> ---
>
> Key: HBASE-27328
> URL: https://issues.apache.org/jira/browse/HBASE-27328
> Project: HBase
> Issue Type: Task
> Components: build
>Reporter: Nick Dimiduk
>Priority: Major
>
> For example, from
> https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/412/General_20Nightly_20Build_20Report/
> {noformat}
> [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @
> hbase-shaded-client-byo-hadoop ---
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.EvaluateBeanshell failed
> with message:
> Couldn't evaluate condition: File license = new
> File("/home/jenkins/jenkins-home/workspace/HBase_Nightly_branch-2.4/component/hbase-shaded/hbase-shaded-client/target/maven-shared-archive-resources/META-INF/LICENSE");
> // Beanshell does not support try-with-resources,
> // so we must close this scanner manually
> Scanner scanner = new Scanner(license);
> while (scanner.hasNextLine()) {
> if (scanner.nextLine().startsWith("ERROR:")) {
> scanner.close();
> return false;
> }
> }
> scanner.close();
> return true;
> [INFO] No sources to compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27328) Enforcer phase EvaluateBeanShell fails occasionally
[
https://issues.apache.org/jira/browse/HBASE-27328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600813#comment-17600813
]
Sean Busbey commented on HBASE-27328:
-
the license validator failing to run should fail the build, because it means
our failsafe on obviously bad license info couldn't run.
the logging situation wrt multithreaded builds is pretty bad, sorry this
particular issue is kind of a nightmare with it. Are we sure both the enforcer
plugin and the plugin we use to run beanshell are threadsafe?
> Enforcer phase EvaluateBeanShell fails occasionally
> ---
>
> Key: HBASE-27328
> URL: https://issues.apache.org/jira/browse/HBASE-27328
> Project: HBase
> Issue Type: Task
> Components: build
>Reporter: Nick Dimiduk
>Priority: Major
>
> For example, from
> https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/412/General_20Nightly_20Build_20Report/
> {noformat}
> [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @
> hbase-shaded-client-byo-hadoop ---
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.EvaluateBeanshell failed
> with message:
> Couldn't evaluate condition: File license = new
> File("/home/jenkins/jenkins-home/workspace/HBase_Nightly_branch-2.4/component/hbase-shaded/hbase-shaded-client/target/maven-shared-archive-resources/META-INF/LICENSE");
> // Beanshell does not support try-with-resources,
> // so we must close this scanner manually
> Scanner scanner = new Scanner(license);
> while (scanner.hasNextLine()) {
> if (scanner.nextLine().startsWith("ERROR:")) {
> scanner.close();
> return false;
> }
> }
> scanner.close();
> return true;
> [INFO] No sources to compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-24466) Release scripts in docker mode should be able to use a named volume for maven repo
[ https://issues.apache.org/jira/browse/HBASE-24466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17597260#comment-17597260 ] Sean Busbey commented on HBASE-24466: - forgot to add. named volumes usually have better performance on macos than bind mounts (a mount that is visible both on the host OS and in the docker containers) because the docker daemon doesn't have to translate between the host and the container environment. > Release scripts in docker mode should be able to use a named volume for maven > repo > -- > > Key: HBASE-24466 > URL: https://issues.apache.org/jira/browse/HBASE-24466 > Project: HBase > Issue Type: Improvement > Components: build, community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > Allow optional volume name to use to mount the maven repo so we get reuse of > third party artifacts between runs. > Forcefully remove artifacts we build from it (assuming we can't make a > read-only one that's shared for third party dependencies and an isolated one > that's for what we're building) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-24466) Release scripts in docker mode should be able to use a named volume for maven repo
[ https://issues.apache.org/jira/browse/HBASE-24466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17597257#comment-17597257 ] Sean Busbey commented on HBASE-24466: - picking this back up again. for folks not familiar with docker and container stuff a bit of background. A "named volume" is a way to consistently refer to a storage location that's available to one or more containers but not the host OS. in k8s terminology they're like a ReadWriteMany PVC but only for the containers on the one host handling the docker daemon. The idea here would be that for a given release line (e.g. 2.5.0) you would reuse the named volume across release candidates to store all the third party artifacts since those usually don't change between candidates. Maybe we could reuse it across patch releases in the same minor release line. Importantly we would need to make sure we aren't reusing any of the artifacts we create in the build. This is close to what happens if you use the release scripts without the docker container and set REPO to point at your local maven repository. > Release scripts in docker mode should be able to use a named volume for maven > repo > -- > > Key: HBASE-24466 > URL: https://issues.apache.org/jira/browse/HBASE-24466 > Project: HBase > Issue Type: Improvement > Components: build, community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > Allow optional volume name to use to mount the maven repo so we get reuse of > third party artifacts between runs. > Forcefully remove artifacts we build from it (assuming we can't make a > read-only one that's shared for third party dependencies and an isolated one > that's for what we're building) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-24466) Release scripts in docker mode should be able to use a named volume for maven repo
[ https://issues.apache.org/jira/browse/HBASE-24466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-24466: Status: In Progress (was: Patch Available) > Release scripts in docker mode should be able to use a named volume for maven > repo > -- > > Key: HBASE-24466 > URL: https://issues.apache.org/jira/browse/HBASE-24466 > Project: HBase > Issue Type: Improvement > Components: build, community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > Allow optional volume name to use to mount the maven repo so we get reuse of > third party artifacts between runs. > Forcefully remove artifacts we build from it (assuming we can't make a > read-only one that's shared for third party dependencies and an isolated one > that's for what we're building) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-24457) release scripts on mac os are too slow
[ https://issues.apache.org/jira/browse/HBASE-24457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-24457. - Fix Version/s: 3.0.0-alpha-1 (was: 3.0.0-alpha-4) Resolution: Fixed > release scripts on mac os are too slow > -- > > Key: HBASE-24457 > URL: https://issues.apache.org/jira/browse/HBASE-24457 > Project: HBase > Issue Type: Umbrella > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0-alpha-1 > > > related changes to try to bring the time down to something I can use -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-24457) release scripts on mac os are too slow
[ https://issues.apache.org/jira/browse/HBASE-24457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17597252#comment-17597252 ] Sean Busbey commented on HBASE-24457: - Okay I'll go ahead and close this out. probably worth revisiting HBASE-24466 to see if that can still get us back ~15% since it looks like that would be a significant amount of time if a full RC build is 5 hours. > release scripts on mac os are too slow > -- > > Key: HBASE-24457 > URL: https://issues.apache.org/jira/browse/HBASE-24457 > Project: HBase > Issue Type: Umbrella > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0-alpha-4 > > > related changes to try to bring the time down to something I can use -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-24457) release scripts on mac os are too slow
[ https://issues.apache.org/jira/browse/HBASE-24457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17594047#comment-17594047 ] Sean Busbey commented on HBASE-24457: - > Still in progress? this is a good question. I have stopped looking at this and I don't know if it's a pressing concern anymore. My personal build machine is windows 11 + wls ubuntu. [~ndimiduk] I think you were trying to run a release off of mac os recently, how does build time look now? > release scripts on mac os are too slow > -- > > Key: HBASE-24457 > URL: https://issues.apache.org/jira/browse/HBASE-24457 > Project: HBase > Issue Type: Umbrella > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0-alpha-4 > > > related changes to try to bring the time down to something I can use -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-24457) release scripts on mac os are too slow
[ https://issues.apache.org/jira/browse/HBASE-24457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17594004#comment-17594004 ] Sean Busbey commented on HBASE-24457: - Best to ask on the dev@hbase mailing list about the status of the hbase 3 release. > release scripts on mac os are too slow > -- > > Key: HBASE-24457 > URL: https://issues.apache.org/jira/browse/HBASE-24457 > Project: HBase > Issue Type: Umbrella > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0-alpha-4 > > > related changes to try to bring the time down to something I can use -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-24457) release scripts on mac os are too slow
[ https://issues.apache.org/jira/browse/HBASE-24457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-24457: Priority: Critical (was: Blocker) > release scripts on mac os are too slow > -- > > Key: HBASE-24457 > URL: https://issues.apache.org/jira/browse/HBASE-24457 > Project: HBase > Issue Type: Umbrella > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0-alpha-4 > > > related changes to try to bring the time down to something I can use -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-21771) Cluster key in Master UI is incorrect
[
https://issues.apache.org/jira/browse/HBASE-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-21771 started by Sean Busbey.
---
> Cluster key in Master UI is incorrect
> -
>
> Key: HBASE-21771
> URL: https://issues.apache.org/jira/browse/HBASE-21771
> Project: HBase
> Issue Type: Bug
> Components: Replication, UI, Usability
>Affects Versions: 3.0.0-alpha-1, 2.1.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: ClusterKey_afterPatch.png, hbase-21771.master.001.patch,
> hbase-21771.master.002.patch
>
>
> The master UI is supposed to give us a cluster key we can copy/paste to add a
> replication peer in the hbase shell. the shell explains that it should look
> like this:
> {quote}
> {code}
> For a HBase cluster peer, a cluster key must be provided and is composed like
> this:
> hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent
> This gives a full path for HBase to connect to another HBase cluster.
> ...
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase"
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase", STATE =>
> "ENABLED"
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase", STATE =>
> "DISABLED"
> hbase> add_peer '2', CLUSTER_KEY => "zk1,zk2,zk3:2182:/hbase-prod",
> TABLE_CFS => { "table1" => [], "table2" => ["cf1"], "table3" => ["cf1",
> "cf2"] }
> hbase> add_peer '2', CLUSTER_KEY => "zk1,zk2,zk3:2182:/hbase-prod",
> NAMESPACES => ["ns1", "ns2", "ns3"]
> {code}
> {quote}
> However, on my example cluster with ZK quorum with 3 servers, the master ui
> gives this:
> {quote}
> {code}
> Cluster Key busbey-training-1.gce.cloudera.com:2181
> busbey-training-2.gce.cloudera.com:2181
> busbey-training-3.gce.cloudera.com:2181:/hbaseKey to add this cluster
> as a peer for replication. Use 'help "add_peer"' in the shell for details.
> {code}
> {quote}
> Note that the quorum members are newline separated instead of comma and that
> the port appears on each member instead of after the set of host names.
> Workaround is to manually construct the cluster key from the details in the
> field. :(
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-21771) Cluster key in Master UI is incorrect
[
https://issues.apache.org/jira/browse/HBASE-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553671#comment-17553671
]
Sean Busbey commented on HBASE-21771:
-
Sakthi hasn't been around in quite some time, but I could take a stab at
getting this turned into a PR sure.
> Cluster key in Master UI is incorrect
> -
>
> Key: HBASE-21771
> URL: https://issues.apache.org/jira/browse/HBASE-21771
> Project: HBase
> Issue Type: Bug
> Components: Replication, UI, Usability
>Affects Versions: 3.0.0-alpha-1, 2.1.0, 2.0.0
>Reporter: Sean Busbey
>Priority: Major
> Attachments: ClusterKey_afterPatch.png, hbase-21771.master.001.patch,
> hbase-21771.master.002.patch
>
>
> The master UI is supposed to give us a cluster key we can copy/paste to add a
> replication peer in the hbase shell. the shell explains that it should look
> like this:
> {quote}
> {code}
> For a HBase cluster peer, a cluster key must be provided and is composed like
> this:
> hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent
> This gives a full path for HBase to connect to another HBase cluster.
> ...
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase"
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase", STATE =>
> "ENABLED"
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase", STATE =>
> "DISABLED"
> hbase> add_peer '2', CLUSTER_KEY => "zk1,zk2,zk3:2182:/hbase-prod",
> TABLE_CFS => { "table1" => [], "table2" => ["cf1"], "table3" => ["cf1",
> "cf2"] }
> hbase> add_peer '2', CLUSTER_KEY => "zk1,zk2,zk3:2182:/hbase-prod",
> NAMESPACES => ["ns1", "ns2", "ns3"]
> {code}
> {quote}
> However, on my example cluster with ZK quorum with 3 servers, the master ui
> gives this:
> {quote}
> {code}
> Cluster Key busbey-training-1.gce.cloudera.com:2181
> busbey-training-2.gce.cloudera.com:2181
> busbey-training-3.gce.cloudera.com:2181:/hbaseKey to add this cluster
> as a peer for replication. Use 'help "add_peer"' in the shell for details.
> {code}
> {quote}
> Note that the quorum members are newline separated instead of comma and that
> the port appears on each member instead of after the set of host names.
> Workaround is to manually construct the cluster key from the details in the
> field. :(
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Assigned] (HBASE-21771) Cluster key in Master UI is incorrect
[
https://issues.apache.org/jira/browse/HBASE-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reassigned HBASE-21771:
---
Assignee: Sean Busbey
> Cluster key in Master UI is incorrect
> -
>
> Key: HBASE-21771
> URL: https://issues.apache.org/jira/browse/HBASE-21771
> Project: HBase
> Issue Type: Bug
> Components: Replication, UI, Usability
>Affects Versions: 3.0.0-alpha-1, 2.1.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: ClusterKey_afterPatch.png, hbase-21771.master.001.patch,
> hbase-21771.master.002.patch
>
>
> The master UI is supposed to give us a cluster key we can copy/paste to add a
> replication peer in the hbase shell. the shell explains that it should look
> like this:
> {quote}
> {code}
> For a HBase cluster peer, a cluster key must be provided and is composed like
> this:
> hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent
> This gives a full path for HBase to connect to another HBase cluster.
> ...
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase"
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase", STATE =>
> "ENABLED"
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase", STATE =>
> "DISABLED"
> hbase> add_peer '2', CLUSTER_KEY => "zk1,zk2,zk3:2182:/hbase-prod",
> TABLE_CFS => { "table1" => [], "table2" => ["cf1"], "table3" => ["cf1",
> "cf2"] }
> hbase> add_peer '2', CLUSTER_KEY => "zk1,zk2,zk3:2182:/hbase-prod",
> NAMESPACES => ["ns1", "ns2", "ns3"]
> {code}
> {quote}
> However, on my example cluster with ZK quorum with 3 servers, the master ui
> gives this:
> {quote}
> {code}
> Cluster Key busbey-training-1.gce.cloudera.com:2181
> busbey-training-2.gce.cloudera.com:2181
> busbey-training-3.gce.cloudera.com:2181:/hbaseKey to add this cluster
> as a peer for replication. Use 'help "add_peer"' in the shell for details.
> {code}
> {quote}
> Note that the quorum members are newline separated instead of comma and that
> the port appears on each member instead of after the set of host names.
> Workaround is to manually construct the cluster key from the details in the
> field. :(
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27114) Upgrade scalatest maven plugin for thread-safety
[
https://issues.apache.org/jira/browse/HBASE-27114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553629#comment-17553629
]
Sean Busbey commented on HBASE-27114:
-
the abase-connectors assembly relies on this config in
{{hbase-connectors-bin.xml}} to get jars like the {{hbase-spark}} module:
{code}
true
{code}
This doesn't work when telling maven to use multiple threads because the
assembly module only lists two dependencies:
{code}
org.apache.hbase.connectors.kafka
hbase-kafka-proxy
${revision}
org.apache.hbase.connectors.kafka
hbase-kafka-model
${revision}
{code}
If you want to do multithreading then we have to tell maven that the assembly
plugin needs the other modules to be built prior to it running by adding them
as dependencies.
> Upgrade scalatest maven plugin for thread-safety
>
>
> Key: HBASE-27114
> URL: https://issues.apache.org/jira/browse/HBASE-27114
> Project: HBase
> Issue Type: Task
> Components: build, spark
>Affects Versions: hbase-connectors-1.0.1, hbase-connectors-1.1.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> The {{master}} branch on the connectors repo warns when {{--threads}} is
> issued, the complaint being the scalatest-maven-plugin. Looks like the latest
> version resolves the complaint. Let's upgrade.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (HBASE-27114) Upgrade scalatest maven plugin for thread-safety
[
https://issues.apache.org/jira/browse/HBASE-27114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553629#comment-17553629
]
Sean Busbey edited comment on HBASE-27114 at 6/13/22 2:58 PM:
--
the hbase-connectors assembly relies on this config in
{{hbase-connectors-bin.xml}} to get jars like the {{hbase-spark}} module:
{code}
true
{code}
This doesn't work when telling maven to use multiple threads because the
assembly module only lists two dependencies:
{code}
org.apache.hbase.connectors.kafka
hbase-kafka-proxy
${revision}
org.apache.hbase.connectors.kafka
hbase-kafka-model
${revision}
{code}
If you want to do multithreading then we have to tell maven that the assembly
plugin needs the other modules to be built prior to it running by adding them
as dependencies.
was (Author: busbey):
the abase-connectors assembly relies on this config in
{{hbase-connectors-bin.xml}} to get jars like the {{hbase-spark}} module:
{code}
true
{code}
This doesn't work when telling maven to use multiple threads because the
assembly module only lists two dependencies:
{code}
org.apache.hbase.connectors.kafka
hbase-kafka-proxy
${revision}
org.apache.hbase.connectors.kafka
hbase-kafka-model
${revision}
{code}
If you want to do multithreading then we have to tell maven that the assembly
plugin needs the other modules to be built prior to it running by adding them
as dependencies.
> Upgrade scalatest maven plugin for thread-safety
>
>
> Key: HBASE-27114
> URL: https://issues.apache.org/jira/browse/HBASE-27114
> Project: HBase
> Issue Type: Task
> Components: build, spark
>Affects Versions: hbase-connectors-1.0.1, hbase-connectors-1.1.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> The {{master}} branch on the connectors repo warns when {{--threads}} is
> issued, the complaint being the scalatest-maven-plugin. Looks like the latest
> version resolves the complaint. Let's upgrade.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-27106) HBase site generation should use its own JRuby
[
https://issues.apache.org/jira/browse/HBASE-27106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-27106:
Status: Patch Available (was: In Progress)
> HBase site generation should use its own JRuby
> --
>
> Key: HBASE-27106
> URL: https://issues.apache.org/jira/browse/HBASE-27106
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> The maven plugin we use to generate the reference guide,
> {{asciidoctor-maven-plugin}}, does not yet work with JRuby 9.3.
> Due to a dependency error we currently get a mix of JRuby 9.2 and 9.3
> libraries when building the site goal, which fails without an actionable
> error message. Correcting the dependency to consistently get JRuby 9.3 also
> results in failure, but due to gem resolution.
> We originally started overriding the JRuby version to get Aarch64 support.
> The version that ships with the plugin has this support already.
> Falling back to JRuby 9.2 for the plugin will mean our reference guide
> building won't work on M1 systems, but I don't see how we can change that
> until the plugin updates for JRuby 9.3 changes.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved HBASE-26983.
-
Resolution: Fixed
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work started] (HBASE-27106) HBase site generation should use its own JRuby
[
https://issues.apache.org/jira/browse/HBASE-27106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-27106 started by Sean Busbey.
---
> HBase site generation should use its own JRuby
> --
>
> Key: HBASE-27106
> URL: https://issues.apache.org/jira/browse/HBASE-27106
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> The maven plugin we use to generate the reference guide,
> {{asciidoctor-maven-plugin}}, does not yet work with JRuby 9.3.
> Due to a dependency error we currently get a mix of JRuby 9.2 and 9.3
> libraries when building the site goal, which fails without an actionable
> error message. Correcting the dependency to consistently get JRuby 9.3 also
> results in failure, but due to gem resolution.
> We originally started overriding the JRuby version to get Aarch64 support.
> The version that ships with the plugin has this support already.
> Falling back to JRuby 9.2 for the plugin will mean our reference guide
> building won't work on M1 systems, but I don't see how we can change that
> until the plugin updates for JRuby 9.3 changes.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (HBASE-27106) HBase site generation should use its own JRuby
Sean Busbey created HBASE-27106:
---
Summary: HBase site generation should use its own JRuby
Key: HBASE-27106
URL: https://issues.apache.org/jira/browse/HBASE-27106
Project: HBase
Issue Type: Bug
Components: website
Reporter: Sean Busbey
Assignee: Sean Busbey
The maven plugin we use to generate the reference guide,
{{asciidoctor-maven-plugin}}, does not yet work with JRuby 9.3.
Due to a dependency error we currently get a mix of JRuby 9.2 and 9.3 libraries
when building the site goal, which fails without an actionable error message.
Correcting the dependency to consistently get JRuby 9.3 also results in
failure, but due to gem resolution.
We originally started overriding the JRuby version to get Aarch64 support. The
version that ships with the plugin has this support already.
Falling back to JRuby 9.2 for the plugin will mean our reference guide building
won't work on M1 systems, but I don't see how we can change that until the
plugin updates for JRuby 9.3 changes.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553024#comment-17553024
]
Sean Busbey edited comment on HBASE-26983 at 6/11/22 5:12 AM:
--
okay I figured it out. We started overriding the version of JRuby used by the
plugin to get aarch64 support in HBASE-19547. In version 2.0.0 of the plugin
they switched from using the jruby-complete dependency to the jruby dependency,
but we coincidentally happened to both be on the same version.
Let me file a follow on to put up one of the possible fixes.
was (Author: busbey):
okay I figured it out. We started overriding the version of JRuby used by the
plugin to get aarch64 support in HBASE-19547. In version 2.0.0 of the plugin
they switched from using the jruby-complete dependency to the jruby dependency,
but we coincidentally happened to both be on the same version.
Let me file a follow on to put up one of the two possible fixes.
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553024#comment-17553024
]
Sean Busbey edited comment on HBASE-26983 at 6/11/22 5:03 AM:
--
okay I figured it out. We started overriding the version of JRuby used by the
plugin to get aarch64 support in HBASE-19547. In version 2.0.0 of the plugin
they switched from using the jruby-complete dependency to the jruby dependency,
but we coincidentally happened to both be on the same version.
Let me file a follow on to put up one of the two possible fixes.
was (Author: busbey):
okay I figured it out. We started overriding the version of JRuby used by the
plugin to get aarch64 support in HBASE-19547. In version 2.1.0 of the plugin
they switched from using the jruby-complete dependency to the jruby dependency,
but we coincidentally happened to both be on the same version.
Let me file a follow on to put up one of the two possible fixes.
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553024#comment-17553024
]
Sean Busbey commented on HBASE-26983:
-
okay I figured it out. We started overriding the version of JRuby used by the
plugin to get aarch64 support in HBASE-19547. In version 2.2.0 of the plugin
they switched from using the jruby-complete dependency to the jruby dependency,
but we coincidentally happened to both be on the same version.
Let me file a follow on to put up one of the two possible fixes.
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553024#comment-17553024
]
Sean Busbey edited comment on HBASE-26983 at 6/11/22 5:02 AM:
--
okay I figured it out. We started overriding the version of JRuby used by the
plugin to get aarch64 support in HBASE-19547. In version 2.1.0 of the plugin
they switched from using the jruby-complete dependency to the jruby dependency,
but we coincidentally happened to both be on the same version.
Let me file a follow on to put up one of the two possible fixes.
was (Author: busbey):
okay I figured it out. We started overriding the version of JRuby used by the
plugin to get aarch64 support in HBASE-19547. In version 2.2.0 of the plugin
they switched from using the jruby-complete dependency to the jruby dependency,
but we coincidentally happened to both be on the same version.
Let me file a follow on to put up one of the two possible fixes.
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553022#comment-17553022
]
Sean Busbey edited comment on HBASE-26983 at 6/11/22 4:09 AM:
--
this is odd. in our pom we add in our jruby version as the jruby-complete jar
to the set of dependencies for the asciidoctor plugin, but then the plugin is
pulling in the individual libraries. I can't be certain that this is the cause
of the failure, but it seems likely to be a source of trouble.
was (Author: busbey):
this is odd. in our pom we add in our jruby version as the jruby-complete jar,
but then the plugin is pulling in the individual libraries. I can't be certain
that this is the cause of the failure, but it seems likely to be a source of
trouble.
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553022#comment-17553022
]
Sean Busbey commented on HBASE-26983:
-
this is odd. in our pom we add in our jruby version as the jruby-complete jar,
but then the plugin is pulling in the individual libraries. I can't be certain
that this is the cause of the failure, but it seems likely to be a source of
trouble.
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552324#comment-17552324
]
Sean Busbey commented on HBASE-26983:
-
right, but the plugin should be grabbing its own dependency version:
{code}
[INFO] Downloaded from central:
https://repo.maven.apache.org/maven2/org/jruby/jruby-core/9.2.13.0/jruby-core-9.2.13.0.jar
(10 MB at 26 MB/s) [INFO] Downloaded from central:
https://repo.maven.apache.org/maven2/org/jruby/jruby-stdlib/9.2.13.0/jruby-stdlib-9.2.13.0.jar
(12 MB at 27 MB/s)
{code}
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552324#comment-17552324
]
Sean Busbey edited comment on HBASE-26983 at 6/9/22 4:57 PM:
-
right, but the plugin should be grabbing its own dependency version:
{code}
[INFO] Downloaded from central:
https://repo.maven.apache.org/maven2/org/jruby/jruby-core/9.2.13.0/jruby-core-9.2.13.0.jar
(10 MB at 26 MB/s)
[INFO] Downloaded from central:
https://repo.maven.apache.org/maven2/org/jruby/jruby-stdlib/9.2.13.0/jruby-stdlib-9.2.13.0.jar
(12 MB at 27 MB/s)
{code}
was (Author: busbey):
right, but the plugin should be grabbing its own dependency version:
{code}
[INFO] Downloaded from central:
https://repo.maven.apache.org/maven2/org/jruby/jruby-core/9.2.13.0/jruby-core-9.2.13.0.jar
(10 MB at 26 MB/s) [INFO] Downloaded from central:
https://repo.maven.apache.org/maven2/org/jruby/jruby-stdlib/9.2.13.0/jruby-stdlib-9.2.13.0.jar
(12 MB at 27 MB/s)
{code}
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Reopened] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reopened HBASE-26983:
-
reopened to evaluate impact on site build
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552298#comment-17552298
]
Sean Busbey commented on HBASE-26983:
-
huh. good catch. certainly looks guilty. let me go dig in.
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved HBASE-26983.
-
Release Note:
Updates the version of JRuby that ships with HBase for the HBase shell to
9.3.4.0.
Note that this changes the supported version of Ruby for HBase shell
integration to Ruby 2.6.
Resolution: Fixed
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-26983) Upgrade JRuby to 9.3.4.0
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26983:
Summary: Upgrade JRuby to 9.3.4.0 (was: HBase shell should work for
dev/test runtime on M1 systems)
> Upgrade JRuby to 9.3.4.0
>
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (HBASE-27006) cordon off large ci worker nodes
[ https://issues.apache.org/jira/browse/HBASE-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-27006. - Fix Version/s: 2.5.0 2.6.0 3.0.0-alpha-3 2.4.13 Assignee: Sean Busbey Resolution: Fixed > cordon off large ci worker nodes > > > Key: HBASE-27006 > URL: https://issues.apache.org/jira/browse/HBASE-27006 > Project: HBase > Issue Type: Sub-task >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13 > > > Make sure our node labeling keeps test workloads that off of the new large > spec nodes so that they're only used for the nightly k8s based tests -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-27024) The User API and Developer API links are broken on hbase.apache.org
[ https://issues.apache.org/jira/browse/HBASE-27024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17535003#comment-17535003 ] Sean Busbey commented on HBASE-27024: - did something change for the website generation job? > The User API and Developer API links are broken on hbase.apache.org > --- > > Key: HBASE-27024 > URL: https://issues.apache.org/jira/browse/HBASE-27024 > Project: HBase > Issue Type: Sub-task > Components: website >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-27023) Add protobuf to NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-27023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534983#comment-17534983 ] Sean Busbey commented on HBASE-27023: - any files with a third party notice, yes. > Add protobuf to NOTICE file > --- > > Key: HBASE-27023 > URL: https://issues.apache.org/jira/browse/HBASE-27023 > Project: HBase > Issue Type: Task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > > The spotless formatting removed the protobuf credit from the > AbstractByteRange. It is currently not included in the NOTICE file. > https://github.com/apache/hbase/commit/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443#diff-f5806f14849a23b9265b022f3f330b80d08bcc10fcf69d8ee2e1b0d5af266d52 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-27023) Add protobuf to NOTICE file
[ https://issues.apache.org/jira/browse/HBASE-27023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534939#comment-17534939 ] Sean Busbey commented on HBASE-27023: - copying some comments over from the PR. 3-Clause BSD licensed works only get entered into relevant LICENSE files. Our inclusion of protocol buffers source is currently included in the existing LICENSE files. we should add the protobuf credit back to the AbstractByteRange source file in order to remain in compliance with the [ASF policy on third party works included in our project source|https://www.apache.org/legal/src-headers.html#3party]. > Add protobuf to NOTICE file > --- > > Key: HBASE-27023 > URL: https://issues.apache.org/jira/browse/HBASE-27023 > Project: HBase > Issue Type: Task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > > The spotless formatting removed the protobuf credit from the > AbstractByteRange. It is currently not included in the NOTICE file. > https://github.com/apache/hbase/commit/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443#diff-f5806f14849a23b9265b022f3f330b80d08bcc10fcf69d8ee2e1b0d5af266d52 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27012) nightly ITBLL
Sean Busbey created HBASE-27012: --- Summary: nightly ITBLL Key: HBASE-27012 URL: https://issues.apache.org/jira/browse/HBASE-27012 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27011) migrate existing nightly test
Sean Busbey created HBASE-27011: --- Summary: migrate existing nightly test Key: HBASE-27011 URL: https://issues.apache.org/jira/browse/HBASE-27011 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey once we can run a cluster on k8s in the nightly worker node, migrate our existing test to use it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27010) container images for HBase
Sean Busbey created HBASE-27010: --- Summary: container images for HBase Key: HBASE-27010 URL: https://issues.apache.org/jira/browse/HBASE-27010 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey we need container images of the build that got made during the nightly run -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27008) container images for Hadoop
Sean Busbey created HBASE-27008: --- Summary: container images for Hadoop Key: HBASE-27008 URL: https://issues.apache.org/jira/browse/HBASE-27008 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey we need image(s) for the hadoop cluster we run the nightly tests on top of -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27009) container images for ZooKeeper
Sean Busbey created HBASE-27009: --- Summary: container images for ZooKeeper Key: HBASE-27009 URL: https://issues.apache.org/jira/browse/HBASE-27009 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey we need image(s) for a zk cluster for our nightly cluster run -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27007) nightly support scripts for install/setup of single-node k8s env
Sean Busbey created HBASE-27007: --- Summary: nightly support scripts for install/setup of single-node k8s env Key: HBASE-27007 URL: https://issues.apache.org/jira/browse/HBASE-27007 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey minikube , kind, something like that. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27006) cordon off large ci worker nodes
Sean Busbey created HBASE-27006: --- Summary: cordon off large ci worker nodes Key: HBASE-27006 URL: https://issues.apache.org/jira/browse/HBASE-27006 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey Make sure our node labeling keeps test workloads that off of the new large spec nodes so that they're only used for the nightly k8s based tests -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27005) move nightly cluster tests to run on k8s
Sean Busbey created HBASE-27005: --- Summary: move nightly cluster tests to run on k8s Key: HBASE-27005 URL: https://issues.apache.org/jira/browse/HBASE-27005 Project: HBase Issue Type: Umbrella Components: integration tests Reporter: Sean Busbey -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-26995) Remove ref guide check in pre commit for branches other than master
[ https://issues.apache.org/jira/browse/HBASE-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530871#comment-17530871 ] Sean Busbey commented on HBASE-26995: - Given the change in process for the ref guide, precommit on these branches should be checking that we're expressly not including ref guide updates > Remove ref guide check in pre commit for branches other than master > --- > > Key: HBASE-26995 > URL: https://issues.apache.org/jira/browse/HBASE-26995 > Project: HBase > Issue Type: Sub-task > Components: build, scripts >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Our nightly builds are failing after HBASE-26932. Let's fix it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-26983) HBase shell should work for dev/test runtime on M1 systems
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17529011#comment-17529011
]
Sean Busbey commented on HBASE-26983:
-
> Added Ruby 2.6 support (but didn't change default?)
I take this back. it looks like JRuby works off of a given Ruby compat for a
minor release line. so we'll need to release note the move from Ruby 2.5 to
Ruby 2.6.
I think this also means for sure this should only go in our next minor release.
> HBase shell should work for dev/test runtime on M1 systems
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work started] (HBASE-26983) HBase shell should work for dev/test runtime on M1 systems
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-26983 started by Sean Busbey.
---
> HBase shell should work for dev/test runtime on M1 systems
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26983) HBase shell should work for dev/test runtime on M1 systems
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17529008#comment-17529008
]
Sean Busbey commented on HBASE-26983:
-
reading the [JRuby 9.3 release
notes|https://github.com/jruby/jruby/wiki/JRuby-9.3.0.0-Release-Notes], things
to be mindful of:
* "require/load/autoload subsystem has been rewritten to better match Ruby
behavior."
* Added Ruby 2.6 support (but didn't change default?)
* Expects Java 8 - 17 (some rumblings about upcoming drop of 8)
* warning about avoiding a direct dependency on {{jruby-core}} (should be
either {{jruby-base}} or {{jruby-complete}}
> HBase shell should work for dev/test runtime on M1 systems
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-26983) HBase shell should work for dev/test runtime on M1 systems
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26983:
Summary: HBase shell should work for dev/test runtime on M1 systems (was:
Dev/test runtime support on M1 systems)
> HBase shell should work for dev/test runtime on M1 systems
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-26983) Dev/test runtime support on M1 systems
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26983:
Fix Version/s: 3.0.0-alpha-3
> Dev/test runtime support on M1 systems
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Priority: Major
> Fix For: 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-26983) Dev/test runtime support on M1 systems
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26983:
Fix Version/s: 2.6.0
> Dev/test runtime support on M1 systems
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Assigned] (HBASE-26983) Dev/test runtime support on M1 systems
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reassigned HBASE-26983:
---
Assignee: Sean Busbey
> Dev/test runtime support on M1 systems
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-26983) Hbase shell fails to start on Apple M1 mac osx
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26983:
Issue Type: Improvement (was: Bug)
> Hbase shell fails to start on Apple M1 mac osx
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Priority: Major
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-26983) Dev/test runtime support on M1 systems
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26983:
Summary: Dev/test runtime support on M1 systems (was: Hbase shell fails to
start on Apple M1 mac osx)
> Dev/test runtime support on M1 systems
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Priority: Major
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26983) Hbase shell fails to start on Apple M1 mac osx
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17529006#comment-17529006
]
Sean Busbey commented on HBASE-26983:
-
Looks like we need to update our JRuby version. It looks like M1 support is
being built up in the 9.3 release line and we're currently on 9.2.
> Hbase shell fails to start on Apple M1 mac osx
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Priority: Major
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-26983) Hbase shell fails to start on Apple M1 mac osx
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26983:
Component/s: shell
> Hbase shell fails to start on Apple M1 mac osx
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Priority: Major
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-26983) Hbase shell fails to start on Apple M1 mac osx
[
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26983:
Description:
Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
*Error:*
{code}
Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15 10:31:00
PDT 2022
Took 0.0010 seconds
NotImplementedError: fstat unimplemented unsupported or native support failed
to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
initialize at org/jruby/RubyIO.java:1015
open at org/jruby/RubyIO.java:1156
initialize at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
initialize at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
initialize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
initialize at uri:classloader:/irb/hirb.rb:49
at classpath:/jar-bootstrap.rb:223
{code}
{*}Uname output{*}:
{code}
Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32
PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
{code}
was:
Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
*Error:*
Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15 10:31:00
PDT 2022
Took 0.0010 seconds
NotImplementedError: fstat unimplemented unsupported or native support failed
to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
initialize at org/jruby/RubyIO.java:1015
open at org/jruby/RubyIO.java:1156
initialize at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
initialize at
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
initialize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
initialize at uri:classloader:/irb/hirb.rb:49
at classpath:/jar-bootstrap.rb:223
{*}Uname output{*}:
Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32
PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> Hbase shell fails to start on Apple M1 mac osx
> --
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Priority: Major
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
> initialize at org/jruby/RubyIO.java:1015
> open at org/jruby/RubyIO.java:1156
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
> initialize at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
> initialize at uri:classloader:/irb/hirb.rb:49
> at classpath:/jar-bootstrap.rb:223
> {code}
>
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26954) Compilation of master vs hadoop-3.3.2 fails
[
https://issues.apache.org/jira/browse/HBASE-26954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522449#comment-17522449
]
Sean Busbey commented on HBASE-26954:
-
whoever works on this please figure out why our import ban didnt' catch this:
https://github.com/apache/hbase/blob/master/pom.xml#L2388
{code}
true
512
Do not use htrace
org.htrace.**
org.apache.htrace.**
{code}
> Compilation of master vs hadoop-3.3.2 fails
> ---
>
> Key: HBASE-26954
> URL: https://issues.apache.org/jira/browse/HBASE-26954
> Project: HBase
> Issue Type: Task
> Components: build, hadoop3
>Affects Versions: 3.0.0-alpha-2
>Reporter: Nick Dimiduk
>Priority: Major
>
> We noticed over on HBASE-26907 while adding Hadoop 3.3.2 to our nightly build
> matrix that we get a compilation failure when setting this as the
> hadoop.three version.
> {noformat}
> [INFO] -
> [ERROR] COMPILATION ERROR :
> [INFO] -
> [ERROR]
> /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4305/yetus-general-check/src/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java:[832,29]
> package org.apache.htrace.core does not exist
> [INFO] 1 error
> [INFO] -
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26954) Compilation of master vs hadoop-3.3.2 fails
[
https://issues.apache.org/jira/browse/HBASE-26954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522447#comment-17522447
]
Sean Busbey commented on HBASE-26954:
-
master doesn't have our noop htrace implementation because we removed our own
use.
must be a reference that got missed in that process.
> Compilation of master vs hadoop-3.3.2 fails
> ---
>
> Key: HBASE-26954
> URL: https://issues.apache.org/jira/browse/HBASE-26954
> Project: HBase
> Issue Type: Task
> Components: build, hadoop3
>Affects Versions: 3.0.0-alpha-2
>Reporter: Nick Dimiduk
>Priority: Major
>
> We noticed over on HBASE-26907 while adding Hadoop 3.3.2 to our nightly build
> matrix that we get a compilation failure when setting this as the
> hadoop.three version.
> {noformat}
> [INFO] -
> [ERROR] COMPILATION ERROR :
> [INFO] -
> [ERROR]
> /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4305/yetus-general-check/src/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java:[832,29]
> package org.apache.htrace.core does not exist
> [INFO] 1 error
> [INFO] -
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26936) project Jenkins nodes need a fail-safe clean up of surefire deferred output
[ https://issues.apache.org/jira/browse/HBASE-26936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-26936: Summary: project Jenkins nodes need a fail-safe clean up of surefire deferred output (was: project Jenkins node needs a fail-safe clean up of surefire deferred output) > project Jenkins nodes need a fail-safe clean up of surefire deferred output > --- > > Key: HBASE-26936 > URL: https://issues.apache.org/jira/browse/HBASE-26936 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > > We've had build nodes run out of space a few times now caused by surefire > deferred stdout / stderr filling a disk. > > Come up with a strategy to make recovery from this automated. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26936) project Jenkins node needs a fail-safe clean up of surefire deferred output
Sean Busbey created HBASE-26936: --- Summary: project Jenkins node needs a fail-safe clean up of surefire deferred output Key: HBASE-26936 URL: https://issues.apache.org/jira/browse/HBASE-26936 Project: HBase Issue Type: Task Components: test Reporter: Sean Busbey Assignee: Sean Busbey We've had build nodes run out of space a few times now caused by surefire deferred stdout / stderr filling a disk. Come up with a strategy to make recovery from this automated. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26880) Misspelling commands in hbase shell will crash the shell
[ https://issues.apache.org/jira/browse/HBASE-26880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518173#comment-17518173 ] Sean Busbey commented on HBASE-26880: - in non-interactive mode we should crash. Does that get maintained? > Misspelling commands in hbase shell will crash the shell > > > Key: HBASE-26880 > URL: https://issues.apache.org/jira/browse/HBASE-26880 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.10 >Reporter: Evan Krettek >Assignee: Peter Somogyi >Priority: Minor > Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.12 > > > Misspelling commands in hbase shell will crash the shell, e.g. > hbase:001:0> hlep > NameError: undefined local variable or method `hlep' for > # > Did you mean? help > evaluate at (hbase):1 > eval at org/jruby/RubyKernel.java:1048 > evaluate at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/workspace.rb:85 > evaluate at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:380 > eval_input at uri:classloader:/irb/hirb.rb:112 > signal_status at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:623 > eval_input at uri:classloader:/irb/hirb.rb:109 > each_top_level_statement at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:246 > loop at org/jruby/RubyKernel.java:1442 > each_top_level_statement at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:232 > catch at org/jruby/RubyKernel.java:1189 > each_top_level_statement at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:231 > eval_input at uri:classloader:/irb/hirb.rb:108 > run at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:428 > catch at org/jruby/RubyKernel.java:1189 > run at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 > at classpath:/jar-bootstrap.rb:221 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514920#comment-17514920
]
Sean Busbey commented on HBASE-26871:
-
I merged the branch-2 change, but have now hit an unrelated logistics of life
thing that's going to delay me from the rest by several hours. If anyone needs
the branch-2.5 or master stuff faster feel free to step in.
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26909) hbase-shaded-mapreduce and hbase-shaded-client expose some of the same classes
[ https://issues.apache.org/jira/browse/HBASE-26909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514915#comment-17514915 ] Sean Busbey edited comment on HBASE-26909 at 3/30/22, 7:46 PM: --- that's an important distinction. if we want a refactoring to pull out the duplicate classes, hbase-shaded-mapreduce will need to show a runtime dependency on the hbase-shaded-client-byo-hadoop artifact and not the one that include hadoop classes. the intended use of hbase-shaded-mapreduce is always with an existing hadoop; the hadoop classes should come from that existing install. was (Author: busbey): that's an important distinction. if we want a refactoring to pull out the duplicate classes, hbase-shaded-mapreduce will need to show a runtime dependency on the hbase-shaded-client-byo-hadoop artifact and not the one that include hadoop classes. the intended use of abase-shaded-mapreduce is always with an existing hadoop; the hadoop classes should come from that existing install. > hbase-shaded-mapreduce and hbase-shaded-client expose some of the same classes > -- > > Key: HBASE-26909 > URL: https://issues.apache.org/jira/browse/HBASE-26909 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Priority: Major > > We supply 2 primary artifacts for end-users to consume: > * hbase-shaded-client, which is for general use > * hbase-shaded-mapreduce, which is for use when you need to connect to hbase > via mapreduce. For example, TableInputFormat > The problem is that these artifacts expose tons of duplicate classes. One > example (among many) is org.apache.hadoop.hbase.Cell, which appears in both > jars. > This may not be a problem if your projects are always very isolated – either > doing mapreduce, or not. In that case you just depend in the one you need. > Many users might exist in much more complicated environments where > dependencies tend to bleed along more between projects. Here's an > illustration: > * Imagine a project FooService, which includes two modules FooServiceRestWeb > (for the rest http resources) and FooServiceData (which includes DAOs for > accessing data). FooServiceRestWeb depends on FooServiceData to access hbase. > In this case, FooServiceData should depend on hbase-shaded-client. > * Now imagine another project FooPipeline, which has modules > FooPipelineHadoop (with M/R jobs for processing data) and FooPipelineData > (which has some DAOs for accessing data). In this case, FooPipelineData might > depend on hbase-shaded-mapreduce since the context is intended for M/R. > * The problem arises when suddenly we want to include some data from > FooService into our pipeline. The most straightforward way to achieve this is > by depending on FooServiceData, which has all of he DAOs for that data but > also depends on hbase-shaded-client. At this point you have a problem, > because FooPipelineHadoop now depends on both hbase-shaded-mapreduce and > hbase-shaded-client. > (Note, this obviously skirts around potential microservice solutions like > only accessing FooService's data through the API... it's just for > illustration, and it does come up.) > From a plain java perspective, having these 2 jars on the classpath is > somewhat wasteful but not a huge issue since the implementations are all the > same. > From a maven perspective, it's problematic because the maven dependency > plugin will complain about the conflicting classes. > One potential fix is to add exclusions to the FooServiceData dependency, to > avoid pulling in hbase-shaded-client. This works on a one-off basis but is > much more painful in a large and complicated environment where this may come > up hundreds of times. > A better fix in my opinion is to make hbase-shaded-mapreduce depend on > hbase-shaded-client and then only expose the classes that aren't already > exposed by the shaded client. > [~busbey] also mentioned a BOM being a potential solution, but I don't have > experience with that. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26909) hbase-shaded-mapreduce and hbase-shaded-client expose some of the same classes
[ https://issues.apache.org/jira/browse/HBASE-26909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514915#comment-17514915 ] Sean Busbey commented on HBASE-26909: - that's an important distinction. if we want a refactoring to pull out the duplicate classes, hbase-shaded-mapreduce will need to show a runtime dependency on the hbase-shaded-client-byo-hadoop artifact and not the one that include hadoop classes. the intended use of abase-shaded-mapreduce is always with an existing hadoop; the hadoop classes should come from that existing install. > hbase-shaded-mapreduce and hbase-shaded-client expose some of the same classes > -- > > Key: HBASE-26909 > URL: https://issues.apache.org/jira/browse/HBASE-26909 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Priority: Major > > We supply 2 primary artifacts for end-users to consume: > * hbase-shaded-client, which is for general use > * hbase-shaded-mapreduce, which is for use when you need to connect to hbase > via mapreduce. For example, TableInputFormat > The problem is that these artifacts expose tons of duplicate classes. One > example (among many) is org.apache.hadoop.hbase.Cell, which appears in both > jars. > This may not be a problem if your projects are always very isolated – either > doing mapreduce, or not. In that case you just depend in the one you need. > Many users might exist in much more complicated environments where > dependencies tend to bleed along more between projects. Here's an > illustration: > * Imagine a project FooService, which includes two modules FooServiceRestWeb > (for the rest http resources) and FooServiceData (which includes DAOs for > accessing data). FooServiceRestWeb depends on FooServiceData to access hbase. > In this case, FooServiceData should depend on hbase-shaded-client. > * Now imagine another project FooPipeline, which has modules > FooPipelineHadoop (with M/R jobs for processing data) and FooPipelineData > (which has some DAOs for accessing data). In this case, FooPipelineData might > depend on hbase-shaded-mapreduce since the context is intended for M/R. > * The problem arises when suddenly we want to include some data from > FooService into our pipeline. The most straightforward way to achieve this is > by depending on FooServiceData, which has all of he DAOs for that data but > also depends on hbase-shaded-client. At this point you have a problem, > because FooPipelineHadoop now depends on both hbase-shaded-mapreduce and > hbase-shaded-client. > (Note, this obviously skirts around potential microservice solutions like > only accessing FooService's data through the API... it's just for > illustration, and it does come up.) > From a plain java perspective, having these 2 jars on the classpath is > somewhat wasteful but not a huge issue since the implementations are all the > same. > From a maven perspective, it's problematic because the maven dependency > plugin will complain about the conflicting classes. > One potential fix is to add exclusions to the FooServiceData dependency, to > avoid pulling in hbase-shaded-client. This works on a one-off basis but is > much more painful in a large and complicated environment where this may come > up hundreds of times. > A better fix in my opinion is to make hbase-shaded-mapreduce depend on > hbase-shaded-client and then only expose the classes that aren't already > exposed by the shaded client. > [~busbey] also mentioned a BOM being a potential solution, but I don't have > experience with that. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514896#comment-17514896
]
Sean Busbey commented on HBASE-26871:
-
I think the posted results on [https://github.com/apache/hbase/pull/4297] show
the addendum working?
Could someone give me a quick review and I'll push to branch-2, branch-2.5 and
forward port the test change to master.
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514337#comment-17514337
]
Sean Busbey commented on HBASE-26871:
-
while updating the script to use JAVA_HOME/bin/jar I rememebered that this
script was already using the jar command to inspect the classes present. so
it's extra confusing that this is failing now?
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514131#comment-17514131
]
Sean Busbey commented on HBASE-26871:
-
I don't think yetus sets JAVA_HOME onto the path, and we rely on yetus's
mechanism for setting java home.
Let me put up a patch that switches to expressly using the jar command out of
JAVA_HOME.
will need a follow on to figure out why this didn't fail in the test branch
nightly and pr.
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26871:
Status: Patch Available (was: In Progress)
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512689#comment-17512689
]
Sean Busbey commented on HBASE-26871:
-
filed jiras:
* HBASE-26887 nightly integration test claims success while showing error
parsing shell output
* HBASE-26888 Update branch-specific nightly test handling to allow for earlier
release line feature branches
* HBASE-26889 nightly yetus tests don't reflect failure when required
environment variables are missing.
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (HBASE-26889) nightly yetus tests don't reflect failure when required environment variables are missing.
Sean Busbey created HBASE-26889:
---
Summary: nightly yetus tests don't reflect failure when required
environment variables are missing.
Key: HBASE-26889
URL: https://issues.apache.org/jira/browse/HBASE-26889
Project: HBase
Issue Type: Bug
Components: test
Reporter: Sean Busbey
the nightly yetus wrapper shows an error when we're missing some needed
environment variables, but the stage shows success still.
e.g
{code:java}
[2022-03-24T22:35:39.450Z] [ERROR] Required environment variable 'DEBUG' is not
set.
[2022-03-24T22:35:39.451Z] [ERROR] Required environment variable
'USE_YETUS_PRERELEASE' is not set.
[2022-03-24T22:35:39.451Z] [ERROR] Please set the required environment
variables before invoking. If this error is on Jenkins, then please file a
JIRA about the error. {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (HBASE-26888) Update branch-specific nightly test handling to allow for earlier release line feature branches
Sean Busbey created HBASE-26888:
---
Summary: Update branch-specific nightly test handling to allow for
earlier release line feature branches
Key: HBASE-26888
URL: https://issues.apache.org/jira/browse/HBASE-26888
Project: HBase
Issue Type: Task
Components: integration tests, test
Reporter: Sean Busbey
due to limitations in ASF infra our branch protection rules prohibit force
pushes or deletions on branches that start with "master" or "branch-".
Our current nightly tests assume that feature branches specific to branch-2 or
branch-1 will start with the release branch name rather than the feature Jira
key. However, doing so would mean we need to file INFRA jiras to clean up
afterwards.
We can't change the limitations of branch protection rules, so we should update
the nightly tests to allow for feature branches named like
{{{}HBASE-X-branch-2{}}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (HBASE-26887) nightly integration test claims success while showing error parsing shell output
Sean Busbey created HBASE-26887:
---
Summary: nightly integration test claims success while showing
error parsing shell output
Key: HBASE-26887
URL: https://issues.apache.org/jira/browse/HBASE-26887
Project: HBase
Issue Type: Bug
Components: integration tests, shell
Reporter: Sean Busbey
current master branch claims that the integration test is succeeding, but
looking at the detailed output shows a failure to parse the shell output:
{code:java}
Starting up Hadoop
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
Verifying configs
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/output-integration/hadoop-3/hbase-conf/core-site.xml:
valid
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/output-integration/hadoop-3/hbase-conf/hbase-site.xml:
valid
OK
Listing HDFS contents
Starting up HBase
running master, logging to
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/hbase-install/bin/../logs/hbase-jenkins-master-jenkins-hbase12.out
retry waiting for hbase to come up.
Setting up table 'test:example' with 1,000 regions
writing out example TSV to example.tsv
uploading example.tsv to HDFS
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
Importing TSV via shaded client artifact for HBase - MapReduce integration.
Verifying row count from import.
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/component/dev-support/hbase_nightly_pseudo-distributed-test.sh:
line 418: [: hbase:002:0> : integer expression expected
Hadoop client jars not given; getting them from 'hadoop classpath' for the
example.
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
Building shaded client example.
Running shaded client example. It'll fetch the set of regions, round-trip them
to a file in HDFS, then write them one-per-row into the test table.
Checking on results of example program.
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
Verifying row count from example.
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/component/dev-support/hbase_nightly_pseudo-distributed-test.sh:
line 531: [: hbase:002:0> : integer expression expected
ERROR: Only found hbase:002:0> rows.
Shutting down HBase {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512686#comment-17512686
]
Sean Busbey commented on HBASE-26871:
-
I take that back. this same claim of success but failure that shows us failing
to grab the row count is currently present on master as well. I don't think
it's related to the failure we're trying to fix here.
the other output files show we're properly getting things in and out of hbase
using the shaded clients.
I'll put up a pr.
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512678#comment-17512678
]
Sean Busbey commented on HBASE-26871:
-
still no joy. the integration test ran against hadoop 2 and reported success,
but the output shows failure:
{code:java}
Setting up table 'test:example' with 1,000 regions
writing out example TSV to example.tsv
uploading example.tsv to HDFS
Importing TSV via shaded client artifact for HBase - MapReduce integration.
Verifying row count from import.
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26871-branch-2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh:
line 414: [: hbase:002:0> : integer expression expected
Hadoop client jars not given; getting them from 'hadoop classpath' for the
example.
Building shaded client example.
Running shaded client example. It'll fetch the set of regions, round-trip them
to a file in HDFS, then write them one-per-row into the test table.
Checking on results of example program.
Verifying row count from example.
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26871-branch-2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh:
line 527: [: hbase:002:0> : integer expression expected
ERROR: Only found hbase:002:0> rows.
Shutting down HBase
stopping hbase...
Shutdown: listing HDFS contents
Shutting down Hadoop {code}
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512185#comment-17512185
]
Sean Busbey commented on HBASE-26871:
-
also looks like a) we still fail to get parameter defaults on first jenkins run
and b) despite the yetus wrapper including error text we don't pick up the
stage as failing.
I'll file jiras for the second one too.
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512181#comment-17512181
]
Sean Busbey commented on HBASE-26871:
-
the test run of nightly didn't run the integration test against hadoop 2. looks
like the branch mapping never got updated for the change to feature branch
naming required by how asf infra does branch protection. I'll push an update on
the branch and make sure we have a Jira long term.
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512180#comment-17512180
]
Sean Busbey commented on HBASE-26871:
-
I believe the appropriate indentation for pom files is 2 spaces, but I haven't
gone looking anywhere to confirm.The pom file changes here were merely reverts
of HBASE-26802 for those files because there were no log4j 2 relevant changes
to them. If you want to file a dedicated issue for fixing indent or evaluating
the exclusions I am not opposed to that in principle.
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512084#comment-17512084
]
Sean Busbey commented on HBASE-26871:
-
pushed up a branch so that nightly can run on it:
https://github.com/apache/hbase/tree/HBASE-26871-branch-2
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512062#comment-17512062
]
Sean Busbey commented on HBASE-26871:
-
the shaded client for bring-your-own hadoop also is broken in the same way:
{code:java}
(base) sbusbey@Seans-MBP hbase % jar tf
hbase-shaded/hbase-shaded-mapreduce/target/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar|
wc -l
10
(base) sbusbey@Seans-MBP hbase % jar tf
hbase-shaded/hbase-shaded-client/target/hbase-shaded-client-2.6.0-SNAPSHOT.jar|
wc -l
28827
(base) sbusbey@Seans-MBP hbase % jar tf
hbase-shaded/hbase-shaded-client-byo-hadoop/target/hbase-shaded-client-byo-hadoop-2.6.0-SNAPSHOT.jar
| wc -l
10
(base) sbusbey@Seans-MBP hbase % jar tf
hbase-shaded/hbase-shaded-testing-util/target/hbase-shaded-testing-util-2.6.0-SNAPSHOT.jar|
wc -l
64340
(base) sbusbey@Seans-MBP hbase % {code}
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26871) shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26871:
Summary: shaded mapreduce and shaded byo-hadoop client artifacts contains
no classes (was: shaded mapreduce artifact contains no classes)
> shaded mapreduce and shaded byo-hadoop client artifacts contains no classes
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work started] (HBASE-26871) shaded mapreduce artifact contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-26871 started by Sean Busbey.
---
> shaded mapreduce artifact contains no classes
> -
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26871) Client integration test fails on branch-2.x with ClassNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26871:
Fix Version/s: 2.5.0
2.6.0
> Client integration test fails on branch-2.x with ClassNotFoundException
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26871) shaded mapreduce artifact contains no classes
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26871:
Summary: shaded mapreduce artifact contains no classes (was: Client
integration test fails on branch-2.x with ClassNotFoundException)
> shaded mapreduce artifact contains no classes
> -
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26871) Client integration test fails on branch-2.x with ClassNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26871:
Priority: Blocker (was: Major)
> Client integration test fails on branch-2.x with ClassNotFoundException
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26871) Client integration test fails on branch-2.x with ClassNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-26871:
Affects Version/s: 2.5.0
2.6.0
> Client integration test fails on branch-2.x with ClassNotFoundException
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Blocker
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) Client integration test fails on branch-2.x with ClassNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512033#comment-17512033
]
Sean Busbey commented on HBASE-26871:
-
looks like the backport of HBASE-26802 to branch-2 mistakenly removed the
dependency on hbase-mapreduce so we just make a noop jar without any contents
> Client integration test fails on branch-2.x with ClassNotFoundException
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Major
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) Client integration test fails on branch-2.x with ClassNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512032#comment-17512032
]
Sean Busbey commented on HBASE-26871:
-
huh. no classes at all.
{code:java}
(base) sbusbey@Seans-MBP hbase % jar tf
hbase-shaded/hbase-shaded-mapreduce/target/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
| wc -l
10
(base) sbusbey@Seans-MBP hbase % jar tf
hbase-shaded/hbase-shaded-mapreduce/target/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
META-INF/
META-INF/MANIFEST.MF
META-INF/DEPENDENCIES
META-INF/LICENSE
META-INF/maven/
META-INF/maven/org.apache.hbase/
META-INF/maven/org.apache.hbase/hbase-shaded-mapreduce/
META-INF/maven/org.apache.hbase/hbase-shaded-mapreduce/pom.xml
META-INF/maven/org.apache.hbase/hbase-shaded-mapreduce/pom.properties
META-INF/NOTICE
(base) sbusbey@Seans-MBP hbase % jar tf
hbase-shaded/hbase-shaded-client/target/hbase-shaded-client-2.6.0-SNAPSHOT.jar
| wc -l
28827 {code}
> Client integration test fails on branch-2.x with ClassNotFoundException
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Major
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26871) Client integration test fails on branch-2.x with ClassNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512027#comment-17512027
]
Sean Busbey commented on HBASE-26871:
-
sure enough, a build on branch-2 shows that the driver class is not present in
the hbase-shaded-mapreduce artifact.
digging in.
> Client integration test fails on branch-2.x with ClassNotFoundException
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Major
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (HBASE-26871) Client integration test fails on branch-2.x with ClassNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reassigned HBASE-26871:
---
Assignee: Sean Busbey
> Client integration test fails on branch-2.x with ClassNotFoundException
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: integration tests, jenkins, mapreduce
>Reporter: Duo Zhang
>Assignee: Sean Busbey
>Priority: Major
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
