[GitHub] [hbase] ramkrish86 commented on pull request #2663: HBASE-24637 - Reseek regression related to filter SKIP hinting
ramkrish86 commented on pull request #2663: URL: https://github.com/apache/hbase/pull/2663#issuecomment-742334015 @infraio I have updated the PR with a new config 'hbase.switchto.next.bytes.read' . It defaults to preadMaxBytes though. also if you configure this value to be < 0 then we don't honour this optimization. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247063#comment-17247063 ] Viraj Jasani commented on HBASE-25329: -- [~caroliney14] If you could create PRs including backport changes, that would be great. patch file support is almost on the verge of deprecation (or maybe already deprecated). Thanks > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22404) Open/Close region request may be executed twice when master restart
[
https://issues.apache.org/jira/browse/HBASE-22404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247055#comment-17247055
]
Sanjeet Nishad commented on HBASE-22404:
Hi [~zhangduo] & [~zghao], we are using HBase-2.2.3. Recently we also faced
similar problem where regionserver ignored a procedure (closeRegionProcedure)
due to duplicate pid which lead region to stuck in RIT.
Analysis:
1. After Hmaster failover, master in-memory proc-id was reset.
2. Upon new DisableTable client request, Master dispatched a
closeRegionProcedure to RS and suspended the proc.
3. But RS ignored the current CloseRegionProcedure request without doing
anything since RS had already executed a procedure with same id.
Since no UnAssignRegionHandler was created at Step-3, so RS did not send any
reportRegionStateTransition to HM. And at HMaster side the procedure remain in
suspended state because we awake the suspended procedure on
reportRegionStateTransition. So region stuck in RIT forever until unless we
restart HM & RS.
> Open/Close region request may be executed twice when master restart
> ---
>
> Key: HBASE-22404
> URL: https://issues.apache.org/jira/browse/HBASE-22404
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.2.0, 2.3.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.2.0, 2.3.0
>
>
> We found this problem when run ITBLL for our internal branch which based
> branch-2.2.
> # Master A schedule a TRSP which will reopen region1. And this TRSP firstly
> schdule a sub remote procedure: CloseRegionProcedure and send the close
> region request to RS.
> # Master A shutdown and Master B is the new active master. And restore this
> TRSP and the remote procedure CloseRegionProcedure.
> # RS reported to the new Master B and the CloseRegionProcedure finished.
> Then the TRSP schdule a new OpenRegionProcedure and send open region request
> to RS.
> # {color:#FF}But meanwhile Master B send the close region request to RS
> again{color}.
> # The open region request finished firstly and report to master succeed. The
> master thought the region was opened on RS. But the RS excuted the close
> region request again and closed the region1.
> # The Master thought the region opened but the RS closed the region. Then
> the new TRSP will stuck forever.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caroline updated HBASE-25329: - Attachment: HBASE-25329.branch-1.000.patch HBASE-25329.branch-2.000.patch HBASE-25329.master.000.patch Status: Patch Available (was: Open) > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2748: HBASE-25373 Remove HTrace completely in code base and try to make use…
Apache-HBase commented on pull request #2748: URL: https://github.com/apache/hbase/pull/2748#issuecomment-742287290 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 6s | master passed | | +1 :green_heart: | compile | 2m 55s | master passed | | +1 :green_heart: | shadedjars | 8m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 7m 11s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 36s | the patch passed | | +1 :green_heart: | compile | 3m 15s | the patch passed | | +1 :green_heart: | javac | 3m 15s | the patch passed | | +1 :green_heart: | shadedjars | 9m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 30s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 350m 38s | root in the patch failed. | | | | 403m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2748 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3630b1c97a3d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7851438379 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/testReport/ | | Max. process+thread count | 3296 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2745: [HBASE-25246] Backup/Restore hbase cell tags.
Apache-HBase commented on pull request #2745: URL: https://github.com/apache/hbase/pull/2745#issuecomment-742284543 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | checkstyle | 2m 3s | master passed | | +1 :green_heart: | spotbugs | 4m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | checkstyle | 1m 59s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 21s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 50m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2745 | | JIRA Issue | HBASE-25246 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b40041f4f5cf 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7a532f8328 | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-mapreduce U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2753: HBASE-25368 Filter out more invalid encoded name in isEncodedRegionNa…
Apache-HBase commented on pull request #2753: URL: https://github.com/apache/hbase/pull/2753#issuecomment-742273859 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 7s | master passed | | +1 :green_heart: | checkstyle | 1m 39s | master passed | | +1 :green_heart: | spotbugs | 3m 10s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 46s | the patch passed | | +1 :green_heart: | checkstyle | 1m 39s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 40s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 47m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2753/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2753 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 43ef7efeecd7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7a532f8328 | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2753/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2753: HBASE-25368 Filter out more invalid encoded name in isEncodedRegionNa…
saintstack commented on pull request #2753: URL: https://github.com/apache/hbase/pull/2753#issuecomment-742254863 Failure seems unrelated. Rerunning anyways while Huaxiang sleeps. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2753: HBASE-25368 Filter out more invalid encoded name in isEncodedRegionNa…
saintstack commented on a change in pull request #2753:
URL: https://github.com/apache/hbase/pull/2753#discussion_r539859613
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
##
@@ -363,7 +363,23 @@ static TableName getTable(final byte [] regionName) {
@InterfaceAudience.Private // For use by internals only.
public static boolean isEncodedRegionName(byte[] regionName) {
// If not parseable as region name, presume encoded. TODO: add stringency;
e.g. if hex.
-return parseRegionNameOrReturnNull(regionName) == null &&
regionName.length <= MD5_HEX_LENGTH;
+if (parseRegionNameOrReturnNull(regionName) == null) {
+ if (regionName.length > MD5_HEX_LENGTH) {
+return false;
+ } else if (regionName.length == MD5_HEX_LENGTH) {
+return true;
Review comment:
THanks for explanation. Shove this up in the JIRA description? Its good.
Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
saintstack commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742245463 Tried upping the vagrantfile ubuntu... removed the old image first. vagrant@rmvm:~$ more /etc/issue Ubuntu 20.04.1 LTS \n \l vagrant@rmvm:~$ gpg --sign repos/hbase/pom.xml gpg: no default secret key: No secret key gpg: signing failed: No secret key vagrant@rmvm:~$ ps auxwww|grep gpg-agent vagrant21315 0.0 0.0 82948 3496 ?SLs 05:08 0:00 /usr/bin/gpg-agent --supervised I didn't check to see if the gpg-agent process was present before I went about my business. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25378) Legacy comparator in Hfile trailer will fail to load
[ https://issues.apache.org/jira/browse/HBASE-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247006#comment-17247006 ] Pankaj Kumar commented on HBASE-25378: -- Np [~ram_krish], yeah it's better to handle in getComparatorClass() > Legacy comparator in Hfile trailer will fail to load > > > Key: HBASE-25378 > URL: https://issues.apache.org/jira/browse/HBASE-25378 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.3.2 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the > deadlock issue. But this introduced compatibility issue, old hfile with > comparator class as > "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open > due to ClassNotFoundException. > Also we should also handle the case when comparator class is > "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was > case before HBASE-24968. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25378) Legacy comparator in Hfile trailer will fail to load
[ https://issues.apache.org/jira/browse/HBASE-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247005#comment-17247005 ] Pankaj Kumar commented on HBASE-25378: -- [~anoop.hbase] I've verified in my local env, 2.2.3 generated hfiles can be read by RS after applying HBASE-24968 changes. So no data compatibility break. Thanks for the pointer. > Legacy comparator in Hfile trailer will fail to load > > > Key: HBASE-25378 > URL: https://issues.apache.org/jira/browse/HBASE-25378 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.3.2 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the > deadlock issue. But this introduced compatibility issue, old hfile with > comparator class as > "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open > due to ClassNotFoundException. > Also we should also handle the case when comparator class is > "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was > case before HBASE-24968. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2748: HBASE-25373 Remove HTrace completely in code base and try to make use…
Apache-HBase commented on pull request #2748: URL: https://github.com/apache/hbase/pull/2748#issuecomment-742213897 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 0s | master passed | | +1 :green_heart: | compile | 2m 48s | master passed | | +1 :green_heart: | shadedjars | 6m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 33s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 1s | the patch passed | | +1 :green_heart: | compile | 2m 47s | the patch passed | | +1 :green_heart: | javac | 2m 47s | the patch passed | | +1 :green_heart: | shadedjars | 7m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 39s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 142m 29s | root in the patch failed. | | | | 188m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2748 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a9b9dbbafa32 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7851438379 | | Default Java | AdoptOpenJDK-11.0.6+10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/testReport/ | | Max. process+thread count | 3974 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25346) hbase2.x the performance is lower than hbase 1.x ？
[
https://issues.apache.org/jira/browse/HBASE-25346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246984#comment-17246984
]
nilonealex commented on HBASE-25346:
[^test_for_randomWrite.log]
> hbase2.x the performance is lower than hbase 1.x ？
> ---
>
> Key: HBASE-25346
> URL: https://issues.apache.org/jira/browse/HBASE-25346
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.2
>Reporter: nilonealex
>Priority: Critical
> Attachments: hbase-pe-performace-test.log, hbase-site.xml,
> test_for_randomWrite.log
>
>
> Recently we found that the newly built production hbase cluster is running a
> bit slow , the hadoop version is Hbase2.0.2 ( HDP3.1.1) and it has 100
> nodes.Then we begin to do load & query performance verification between
> Hbase2.0.2 ( HDP3.1.1) & Hbase1.2.0 ( CDH5.13.3 ) test environment （4nodes）,
> found that : put data based on hbase2.0 is much slower than hbase1.x (the
> former is almost half of the latter), I use BufferedMutator and
> BufferedMutatorParams term for batch put to improve efficiency. More
> confusing is the performance of the production environment is worse than my
> test environment
> Some of the codes are as follows:
> ---
> {color:#4C9AFF}List mutator = new ArrayList<>();
> BufferedMutator table = null;
> BufferedMutatorParams params = new
> BufferedMutatorParams(TableName.valueOf(fileHbRule.getHbaseTableName()));
> params.writeBufferSize(fileHbRule.getFlushBuffer().intValue()*1024*1024);
> table = connection.getBufferedMutator(params);
>
> mutator.add(p);
> if(totalCnts % 5000 == 0 ) {
> table.mutate(mutator);
> mutator.clear();
> }{color}
> ---
> The file to put is a text format file: 2 million rows comma-separated text
> file, each row records 110 columns, total size is about 1G. In addition to
> the main parameter configuration such as heap memory, I kept the default
> parameter values ??for most of the hbase services.
> The load program is designed for single thread.
> The following is the progress information :
> --- Hbase1.2.0 ( CDH5.13.3 )
>
> 2020-12-01 16:48:18 inserted: 10
> 2020-12-01 16:48:36 inserted: 20
> 2020-12-01 16:48:52 inserted: 30
> 2020-12-01 16:49:08 inserted: 40
> 2020-12-01 16:49:23 inserted: 50
> 2020-12-01 16:49:39 inserted: 60
> 2020-12-01 16:49:56 inserted: 70
> 2020-12-01 16:50:12 inserted: 80
> 2020-12-01 16:50:29 inserted: 90
> 2020-12-01 16:50:45 inserted: 100
> 2020-12-01 16:51:01 inserted: 110
> 2020-12-01 16:51:17 inserted: 120
> 2020-12-01 16:51:34 inserted: 130
> 2020-12-01 16:51:49 inserted: 140
> 2020-12-01 16:52:05 inserted: 150
> 2020-12-01 16:52:21 inserted: 160
> 2020-12-01 16:52:40 inserted: 170
> 2020-12-01 16:52:57 inserted: 180
> 2020-12-01 16:53:19 inserted: 190
> 2020-12-01 16:53:42 inserted: 200
> 2020-12-01 16:53:48 inserted: 200
> imp finished ok!
> --job finished--
> ---Hbase.2.0.2 (
> HDP3.1.1)-
> 2020-12-01 17:25:24 inserted: 10
> 2020-12-01 17:26:03 inserted: 20
> 2020-12-01 17:26:39 inserted: 30
> 2020-12-01 17:27:13 inserted: 40
> 2020-12-01 17:27:47 inserted: 50
> 2020-12-01 17:28:23 inserted: 60
> 2020-12-01 17:29:03 inserted: 70
> 2020-12-01 17:29:40 inserted: 80
> 2020-12-01 17:30:15 inserted: 90
> 2020-12-01 17:30:51 inserted: 100
> 2020-12-01 17:31:27 inserted: 110
> 2020-12-01 17:32:03 inserted: 120
> 2020-12-01 17:32:39 inserted: 130
> 2020-12-01 17:33:14 inserted: 140
> 2020-12-01 17:33:50 inserted: 150
> 2020-12-01 17:34:25 inserted: 160
> 2020-12-01 17:35:01 inserted: 170
> 2020-12-01 17:35:38 inserted: 180
> 2020-12-01 17:36:14 inserted: 190
> 2020-12-01 17:36:51 inserted: 200
> 2020-12-01 17:36:55 inserted: 200
> imp finished ok!
> --job finished--
> returnCode=0
> In addition, we also did some benchmark tests on the production cluster.The
> delay is seem to be a bit high. The detailed report is in the attachment.
> Are there any key points that I have not done configuration? or，, this
> version has performance defects ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25346) hbase2.x the performance is lower than hbase 1.x ？
[
https://issues.apache.org/jira/browse/HBASE-25346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nilonealex updated HBASE-25346:
---
Attachment: test_for_randomWrite.log
> hbase2.x the performance is lower than hbase 1.x ？
> ---
>
> Key: HBASE-25346
> URL: https://issues.apache.org/jira/browse/HBASE-25346
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.2
>Reporter: nilonealex
>Priority: Critical
> Attachments: hbase-pe-performace-test.log, hbase-site.xml,
> test_for_randomWrite.log
>
>
> Recently we found that the newly built production hbase cluster is running a
> bit slow , the hadoop version is Hbase2.0.2 ( HDP3.1.1) and it has 100
> nodes.Then we begin to do load & query performance verification between
> Hbase2.0.2 ( HDP3.1.1) & Hbase1.2.0 ( CDH5.13.3 ) test environment （4nodes）,
> found that : put data based on hbase2.0 is much slower than hbase1.x (the
> former is almost half of the latter), I use BufferedMutator and
> BufferedMutatorParams term for batch put to improve efficiency. More
> confusing is the performance of the production environment is worse than my
> test environment
> Some of the codes are as follows:
> ---
> {color:#4C9AFF}List mutator = new ArrayList<>();
> BufferedMutator table = null;
> BufferedMutatorParams params = new
> BufferedMutatorParams(TableName.valueOf(fileHbRule.getHbaseTableName()));
> params.writeBufferSize(fileHbRule.getFlushBuffer().intValue()*1024*1024);
> table = connection.getBufferedMutator(params);
>
> mutator.add(p);
> if(totalCnts % 5000 == 0 ) {
> table.mutate(mutator);
> mutator.clear();
> }{color}
> ---
> The file to put is a text format file: 2 million rows comma-separated text
> file, each row records 110 columns, total size is about 1G. In addition to
> the main parameter configuration such as heap memory, I kept the default
> parameter values ??for most of the hbase services.
> The load program is designed for single thread.
> The following is the progress information :
> --- Hbase1.2.0 ( CDH5.13.3 )
>
> 2020-12-01 16:48:18 inserted: 10
> 2020-12-01 16:48:36 inserted: 20
> 2020-12-01 16:48:52 inserted: 30
> 2020-12-01 16:49:08 inserted: 40
> 2020-12-01 16:49:23 inserted: 50
> 2020-12-01 16:49:39 inserted: 60
> 2020-12-01 16:49:56 inserted: 70
> 2020-12-01 16:50:12 inserted: 80
> 2020-12-01 16:50:29 inserted: 90
> 2020-12-01 16:50:45 inserted: 100
> 2020-12-01 16:51:01 inserted: 110
> 2020-12-01 16:51:17 inserted: 120
> 2020-12-01 16:51:34 inserted: 130
> 2020-12-01 16:51:49 inserted: 140
> 2020-12-01 16:52:05 inserted: 150
> 2020-12-01 16:52:21 inserted: 160
> 2020-12-01 16:52:40 inserted: 170
> 2020-12-01 16:52:57 inserted: 180
> 2020-12-01 16:53:19 inserted: 190
> 2020-12-01 16:53:42 inserted: 200
> 2020-12-01 16:53:48 inserted: 200
> imp finished ok!
> --job finished--
> ---Hbase.2.0.2 (
> HDP3.1.1)-
> 2020-12-01 17:25:24 inserted: 10
> 2020-12-01 17:26:03 inserted: 20
> 2020-12-01 17:26:39 inserted: 30
> 2020-12-01 17:27:13 inserted: 40
> 2020-12-01 17:27:47 inserted: 50
> 2020-12-01 17:28:23 inserted: 60
> 2020-12-01 17:29:03 inserted: 70
> 2020-12-01 17:29:40 inserted: 80
> 2020-12-01 17:30:15 inserted: 90
> 2020-12-01 17:30:51 inserted: 100
> 2020-12-01 17:31:27 inserted: 110
> 2020-12-01 17:32:03 inserted: 120
> 2020-12-01 17:32:39 inserted: 130
> 2020-12-01 17:33:14 inserted: 140
> 2020-12-01 17:33:50 inserted: 150
> 2020-12-01 17:34:25 inserted: 160
> 2020-12-01 17:35:01 inserted: 170
> 2020-12-01 17:35:38 inserted: 180
> 2020-12-01 17:36:14 inserted: 190
> 2020-12-01 17:36:51 inserted: 200
> 2020-12-01 17:36:55 inserted: 200
> imp finished ok!
> --job finished--
> returnCode=0
> In addition, we also did some benchmark tests on the production cluster.The
> delay is seem to be a bit high. The detailed report is in the attachment.
> Are there any key points that I have not done configuration? or，, this
> version has performance defects ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25372) Fix typo in ban-jersey section of the enforcer plugin in pom.xml
[
https://issues.apache.org/jira/browse/HBASE-25372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246980#comment-17246980
]
Hudson commented on HBASE-25372:
Results for branch branch-2.4
[build #5 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/5/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/5/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/5/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/5/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/5/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix typo in ban-jersey section of the enforcer plugin in pom.xml
>
>
> Key: HBASE-25372
> URL: https://issues.apache.org/jira/browse/HBASE-25372
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: lixiaobao
>Priority: Major
> Labels: trivial
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1
>
>
> {code}
>
> >org.glassfish.jersey.containers:**
> >org.glassfish.jersey.core:**
>
> {code}
> Need to remove the extra '>'.
> This was introduced when we shaded jetty and jersey.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2760: HBASE-25293 Followup jira to address the client handling issue when c…
Apache-HBase commented on pull request #2760: URL: https://github.com/apache/hbase/pull/2760#issuecomment-742198364 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 34s | branch-2 passed | | +1 :green_heart: | compile | 1m 24s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 14s | the patch passed | | +1 :green_heart: | compile | 1m 23s | the patch passed | | +1 :green_heart: | javac | 1m 23s | the patch passed | | +1 :green_heart: | shadedjars | 5m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 23s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 139m 43s | hbase-server in the patch passed. | | | | 169m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2760/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2760 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 483f246f3191 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 81868e328a | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2760/1/testReport/ | | Max. process+thread count | 4031 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2760/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2759: HBASE-25356 HBaseAdmin#getRegion() needs to filter out non-regionName…
Apache-HBase commented on pull request #2759: URL: https://github.com/apache/hbase/pull/2759#issuecomment-742188057 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 43s | branch-2 passed | | +1 :green_heart: | compile | 1m 48s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 19s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 50s | the patch passed | | +1 :green_heart: | compile | 1m 53s | the patch passed | | +1 :green_heart: | javac | 1m 53s | the patch passed | | +1 :green_heart: | shadedjars | 7m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 40s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 225m 39s | hbase-server in the patch passed. | | | | 263m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2759 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a4a9a07cab7e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 81868e328a | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/testReport/ | | Max. process+thread count | 2490 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
saintstack commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742179447 On Mac its $ gpg --version gpg (GnuPG) 2.2.25 Trying later ubuntu... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on pull request #2759: HBASE-25356 HBaseAdmin#getRegion() needs to filter out non-regionName…
huaxiangsun commented on pull request #2759: URL: https://github.com/apache/hbase/pull/2759#issuecomment-742178536 checking the client error at my local machine now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2748: HBASE-25373 Remove HTrace completely in code base and try to make use…
Apache-HBase commented on pull request #2748: URL: https://github.com/apache/hbase/pull/2748#issuecomment-742177170 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | master passed | | +1 :green_heart: | checkstyle | 2m 11s | master passed | | +1 :green_heart: | spotbugs | 18m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 21s | the patch passed | | +1 :green_heart: | checkstyle | 2m 12s | root: The patch generated 0 new + 562 unchanged - 23 fixed = 562 total (was 585) | | -0 :warning: | rubocop | 0m 3s | The patch generated 4 new + 3 unchanged - 8 fixed = 7 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 15s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 17m 12s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 19m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 2m 54s | The patch does not generate ASF License warnings. | | | | 81m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2748 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle xml rubocop | | uname | Linux c222ccfd610f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7851438379 | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 137 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2745: [HBASE-25246] Backup/Restore hbase cell tags.
Apache-HBase commented on pull request #2745: URL: https://github.com/apache/hbase/pull/2745#issuecomment-742167846 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 21s | master passed | | +1 :green_heart: | compile | 2m 9s | master passed | | +1 :green_heart: | shadedjars | 7m 24s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 24s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 6s | the patch passed | | +1 :green_heart: | compile | 2m 2s | the patch passed | | +1 :green_heart: | javac | 2m 2s | the patch passed | | +1 :green_heart: | shadedjars | 7m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 21s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 230m 23s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 14m 47s | hbase-mapreduce in the patch passed. | | | | 281m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2745 | | JIRA Issue | HBASE-25246 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 45329940c8c0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7851438379 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/3/testReport/ | | Max. process+thread count | 3427 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-mapreduce U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25376) [create-release] Fix double .asc
[ https://issues.apache.org/jira/browse/HBASE-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-25376: -- Fix Version/s: 3.0.0-alpha-1 > [create-release] Fix double .asc > > > Key: HBASE-25376 > URL: https://issues.apache.org/jira/browse/HBASE-25376 > Project: HBase > Issue Type: Bug > Components: create-release >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1 > > Attachments: Screen Shot 2020-12-09 at 8.24.07 AM.png, Screen Shot > 2020-12-09 at 8.24.56 AM.png > > > We are signing the signature file when we run the create-release which > results in *.jar.asc.asc files up in maven repository. It looks to be this > line in release-util.sh maven_deploy: > - declare -a mvn_goals=(clean install) > + declare -a mvn_goals=(clean) > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25376) [create-release] Fix double .asc
[ https://issues.apache.org/jira/browse/HBASE-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25376. --- Hadoop Flags: Reviewed Resolution: Fixed Merge. Thanks for review [~apurtell] > [create-release] Fix double .asc > > > Key: HBASE-25376 > URL: https://issues.apache.org/jira/browse/HBASE-25376 > Project: HBase > Issue Type: Bug > Components: create-release >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1 > > Attachments: Screen Shot 2020-12-09 at 8.24.07 AM.png, Screen Shot > 2020-12-09 at 8.24.56 AM.png > > > We are signing the signature file when we run the create-release which > results in *.jar.asc.asc files up in maven repository. It looks to be this > line in release-util.sh maven_deploy: > - declare -a mvn_goals=(clean install) > + declare -a mvn_goals=(clean) > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25380) [create-release] Add timestamping to log output
[ https://issues.apache.org/jira/browse/HBASE-25380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25380. --- Fix Version/s: 3.0.0-alpha-1 Hadoop Flags: Reviewed Assignee: Michael Stack Resolution: Fixed Merged (thanks for review [~apurtell] ) > [create-release] Add timestamping to log output > --- > > Key: HBASE-25380 > URL: https://issues.apache.org/jira/browse/HBASE-25380 > Project: HBase > Issue Type: Sub-task > Components: create-release >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Add a timestamp when we output a status message. Without it is more difficult > figuring where the time is being spent building. > Updated the README too adding some setup text copied from the head of the > script. The README is better place for this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #2758: HBASE-25380 [create-release] Add timestamping to log output
saintstack merged pull request #2758: URL: https://github.com/apache/hbase/pull/2758 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2758: HBASE-25380 [create-release] Add timestamping to log output
saintstack commented on pull request #2758: URL: https://github.com/apache/hbase/pull/2758#issuecomment-742160868 Signed-off-by: Andrew Purtell This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2758: HBASE-25380 [create-release] Add timestamping to log output
saintstack commented on pull request #2758: URL: https://github.com/apache/hbase/pull/2758#issuecomment-742160489 Thanks @apurtell This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
ndimiduk commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742159172 Head's up though -- if the issue here is indeed gpg agent version mismatch, create-release in docker mode will have the same problems because it's also using an ubuntu bionic base for it's agent forwarding tests... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
ndimiduk commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742157507 I think this project has bit-rot since I used it last (which has been several months). My current best guess is that gpg version miss-match is preventing agent forwarding to work. My host machine has gpg 2.2.20 while the guest bionic vm install gpg 2.2.4. According to [packages.u.o](https://packages.ubuntu.com/search?keywords=gnupg), I can get gpg 2.2.19 if I upgrade the vm to Ubuntu focal (20.04LTS). The alternative is to downgrade the gpg version running on the host OS. I don't think that's a good long-term solution though. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2759: HBASE-25356 HBaseAdmin#getRegion() needs to filter out non-regionName…
Apache-HBase commented on pull request #2759: URL: https://github.com/apache/hbase/pull/2759#issuecomment-742153293 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 33s | branch-2 passed | | +1 :green_heart: | compile | 1m 24s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 16s | the patch passed | | +1 :green_heart: | compile | 1m 24s | the patch passed | | +1 :green_heart: | javac | 1m 24s | the patch passed | | +1 :green_heart: | shadedjars | 6m 2s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 0m 55s | hbase-client in the patch failed. | | +1 :green_heart: | unit | 138m 51s | hbase-server in the patch passed. | | | | 167m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2759 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 609f4fb06765 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 81868e328a | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-client.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/testReport/ | | Max. process+thread count | 4134 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2760: HBASE-25293 Followup jira to address the client handling issue when c…
Apache-HBase commented on pull request #2760: URL: https://github.com/apache/hbase/pull/2760#issuecomment-742153201 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 14s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 52s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 54s | branch-2 passed | | +1 :green_heart: | spotbugs | 3m 17s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | -0 :warning: | checkstyle | 1m 13s | hbase-server: The patch generated 10 new + 0 unchanged - 0 fixed = 10 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 45s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | | | | 41m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2760/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2760 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux c9fd5bb3789c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 81868e328a | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2760/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2760/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun opened a new pull request #2760: HBASE-25293 Followup jira to address the client handling issue when c…
huaxiangsun opened a new pull request #2760: URL: https://github.com/apache/hbase/pull/2760 …haning from meta replica to non-meta-replica at the server side. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
saintstack commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742136556 Ok. Sorted out the virtualbox and vagrant stuff... Thats all good. Just stuck now on the gpg secret passing bit (Nick has been helping me out offline). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2745: [HBASE-25246] Backup/Restore hbase cell tags.
Apache-HBase commented on pull request #2745: URL: https://github.com/apache/hbase/pull/2745#issuecomment-742131236 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 2m 2s | master passed | | +1 :green_heart: | shadedjars | 6m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 29s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 2m 4s | the patch passed | | +1 :green_heart: | javac | 2m 4s | the patch passed | | +1 :green_heart: | shadedjars | 6m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 27s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 8s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 137m 22s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 11m 3s | hbase-mapreduce in the patch passed. | | | | 182m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2745 | | JIRA Issue | HBASE-25246 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8d8a4f8dfdf5 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7851438379 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/3/testReport/ | | Max. process+thread count | 4087 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-mapreduce U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-25356) HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName
[ https://issues.apache.org/jira/browse/HBASE-25356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25356 started by Huaxiang Sun. > HBaseAdmin#getRegion() needs to filter out non-regionName and > non-encodedRegionName > --- > > Key: HBASE-25356 > URL: https://issues.apache.org/jira/browse/HBASE-25356 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.0 >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > > I was running shell command to major compact meta table. The implementation > is wrong because it tries to search the meta table with meta table name. This > also results in an unnecessary scan of meta table. > > majorCompactRegion() is calling HBaseAdmin#getRegion() which basically scan > meta table itself. > This command is being used by operator quite often, we need to correct it. > > This applies to split/flush command as well, which calls getRegion() with > tableName as an input. > > The solution is that getRegion() needs to filter out non-regionName and > non-encodedRegionName, this will save a query of meta table and a heavy scan > of meta table. If meta table size is large, the overhead is huge. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-25368) Filter out more invalid encoded name in isEncodedRegionName(byte[] regionName)
[
https://issues.apache.org/jira/browse/HBASE-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-25368 started by Huaxiang Sun.
> Filter out more invalid encoded name in isEncodedRegionName(byte[]
> regionName)
> ---
>
> Key: HBASE-25368
> URL: https://issues.apache.org/jira/browse/HBASE-25368
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
>
> {code:java}
> public static boolean isEncodedRegionName(byte[] regionName) {
> // If not parseable as region name, presume encoded. TODO: add stringency;
> e.g. if hex.
> return parseRegionNameOrReturnNull(regionName) == null && regionName.length
> <= MD5_HEX_LENGTH;
> }
> Right now, if it passes in an table name, it still thinks it is a encoded
> region name and will result in unnecessary registry query for meta regions.
> This can be avoided if table names can be filtered out early in this
> method.{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2759: HBASE-25356 HBaseAdmin#getRegion() needs to filter out non-regionName…
Apache-HBase commented on pull request #2759: URL: https://github.com/apache/hbase/pull/2759#issuecomment-742106469 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 57s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 46s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 7s | branch-2 passed | | +1 :green_heart: | spotbugs | 3m 58s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 13s | the patch passed | | -0 :warning: | checkstyle | 0m 41s | hbase-client: The patch generated 1 new + 124 unchanged - 0 fixed = 125 total (was 124) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 50s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | | | | 47m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2759 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 9794ac38a1f8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 81868e328a | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2759/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
saintstack commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742105721 Build fails for me because no gpg key in vagrant env... $ gpg --sign repos/hbase/pom.xml gpg: no default secret key: No secret key gpg: signing failed: No secret key When I run vagrant, there is a gpg-agent up that works.. has cached key... Vagrant supposed to hoist it up into the vm (the key may not have registered in agent when I did 'vagrant up'... maybe it has to be then?) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun opened a new pull request #2759: HBASE-25356 HBaseAdmin#getRegion() needs to filter out non-regionName…
huaxiangsun opened a new pull request #2759: URL: https://github.com/apache/hbase/pull/2759 … and non-encodedRegionName This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25356) HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName
[ https://issues.apache.org/jira/browse/HBASE-25356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25356: - Affects Version/s: (was: 1.6.0) > HBaseAdmin#getRegion() needs to filter out non-regionName and > non-encodedRegionName > --- > > Key: HBASE-25356 > URL: https://issues.apache.org/jira/browse/HBASE-25356 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.0 >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > > I was running shell command to major compact meta table. The implementation > is wrong because it tries to search the meta table with meta table name. This > also results in an unnecessary scan of meta table. > > majorCompactRegion() is calling HBaseAdmin#getRegion() which basically scan > meta table itself. > This command is being used by operator quite often, we need to correct it. > > This applies to split/flush command as well, which calls getRegion() with > tableName as an input. > > The solution is that getRegion() needs to filter out non-regionName and > non-encodedRegionName, this will save a query of meta table and a heavy scan > of meta table. If meta table size is large, the overhead is huge. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25356) HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName
[ https://issues.apache.org/jira/browse/HBASE-25356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25356: - Description: I was running shell command to major compact meta table. The implementation is wrong because it tries to search the meta table with meta table name. This also results in an unnecessary scan of meta table. majorCompactRegion() is calling HBaseAdmin#getRegion() which basically scan meta table itself. This command is being used by operator quite often, we need to correct it. This applies to split/flush command as well, which calls getRegion() with tableName as an input. The solution is that getRegion() needs to filter out non-regionName and non-encodedRegionName, this will save a query of meta table and a heavy scan of meta table. If meta table size is large, the overhead is huge. was: I was running shell command to major compact meta table. The implementation is wrong because it tries to search the meta table with meta table name. This also results in an unnecessary scan of meta table. majorCompactRegion() is calling getRegion() which basically scan meta table itself. This command is being used by operator quite often, we need to correct it. This applies to split/flush command as well, which calls getRegion() with tableName as an input. > HBaseAdmin#getRegion() needs to filter out non-regionName and > non-encodedRegionName > --- > > Key: HBASE-25356 > URL: https://issues.apache.org/jira/browse/HBASE-25356 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 1.6.0, 2.4.0 >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > > I was running shell command to major compact meta table. The implementation > is wrong because it tries to search the meta table with meta table name. This > also results in an unnecessary scan of meta table. > > majorCompactRegion() is calling HBaseAdmin#getRegion() which basically scan > meta table itself. > This command is being used by operator quite often, we need to correct it. > > This applies to split/flush command as well, which calls getRegion() with > tableName as an input. > > The solution is that getRegion() needs to filter out non-regionName and > non-encodedRegionName, this will save a query of meta table and a heavy scan > of meta table. If meta table size is large, the overhead is huge. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25356) HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName
[ https://issues.apache.org/jira/browse/HBASE-25356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25356: - Summary: HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName (was: shell command major_compact misbehave for hbase:meta) > HBaseAdmin#getRegion() needs to filter out non-regionName and > non-encodedRegionName > --- > > Key: HBASE-25356 > URL: https://issues.apache.org/jira/browse/HBASE-25356 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 1.6.0, 2.4.0 >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > > I was running shell command to major compact meta table. The implementation > is wrong because it tries to search the meta table with meta table name. This > also results in an unnecessary scan of meta table. > > majorCompactRegion() is calling getRegion() which basically scan meta table > itself. > This command is being used by operator quite often, we need to correct it. > > This applies to split/flush command as well, which calls getRegion() with > tableName as an input. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25356) shell command major_compact misbehave for hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-25356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25356: - Description: I was running shell command to major compact meta table. The implementation is wrong because it tries to search the meta table with meta table name. This also results in an unnecessary scan of meta table. majorCompactRegion() is calling getRegion() which basically scan meta table itself. This command is being used by operator quite often, we need to correct it. This applies to split/flush command as well, which calls getRegion() with tableName as an input. was: I was running shell command to major compact meta table. The implementation is wrong because it tries to search the meta table with meta table name. This also results in an unnecessary scan of meta table. majorCompactRegion() is calling getRegion() which basically scan meta table itself. This command is being used by operator quite often, we need to correct it. > shell command major_compact misbehave for hbase:meta > > > Key: HBASE-25356 > URL: https://issues.apache.org/jira/browse/HBASE-25356 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 1.6.0, 2.4.0 >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > > I was running shell command to major compact meta table. The implementation > is wrong because it tries to search the meta table with meta table name. This > also results in an unnecessary scan of meta table. > > majorCompactRegion() is calling getRegion() which basically scan meta table > itself. > This command is being used by operator quite often, we need to correct it. > > This applies to split/flush command as well, which calls getRegion() with > tableName as an input. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2745: [HBASE-25246] Backup/Restore hbase cell tags.
Apache-HBase commented on pull request #2745: URL: https://github.com/apache/hbase/pull/2745#issuecomment-742065444 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 33s | master passed | | +1 :green_heart: | checkstyle | 1m 55s | master passed | | +1 :green_heart: | spotbugs | 3m 38s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | checkstyle | 1m 55s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 12s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 45m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2745 | | JIRA Issue | HBASE-25246 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 6c4fa894bec6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7851438379 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-mapreduce U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2745/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #2745: [HBASE-25246] Backup/Restore hbase cell tags.
shahrs87 commented on pull request #2745: URL: https://github.com/apache/hbase/pull/2745#issuecomment-742036969 Thank you @anoopsjohn and @virajjasani for your offline help. Could you guys please review the changes. Thank you ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2753: HBASE-25368 Filter out more invalid encoded name in isEncodedRegionNa…
huaxiangsun commented on a change in pull request #2753:
URL: https://github.com/apache/hbase/pull/2753#discussion_r539631859
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin1.java
##
@@ -116,6 +130,8 @@ public void testCompactionTimestamps() throws Exception {
assertEquals(0, ts);
ADMIN.flush(tableName);
+
//ADMIN.majorCompactRegion(Bytes.toBytes("abcdefghijklmnopqrstuvwxyzabcdedgeghijklmnopqrs"));
+ADMIN.majorCompactRegion(Bytes.toBytes("abcd"));
Review comment:
Will remove these two lines, they are used for testing before the new
test case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
Apache-HBase commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742029557 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +0 :ok: | yamllint | 0m 0s | yamllint was not available. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 3m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2754 | | Optional Tests | dupname asflicense shellcheck shelldocs yamllint markdownlint | | uname | Linux 54e3523ac62d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7851438379 | | Max. process+thread count | 47 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/2/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
Apache-HBase commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742029108 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2754 | | Optional Tests | | | uname | Linux 6c77c877b27a 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7851438379 | | Max. process+thread count | 58 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
Apache-HBase commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742028698 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 1m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2754 | | Optional Tests | | | uname | Linux f5f34bf955d2 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7851438379 | | Max. process+thread count | 52 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
saintstack commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742027391 On another host (my build machine as it happens), it seems to just work. Trying test build... Looked like this when I logged in... ``` $ vagrant ssh Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-72-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support:https://ubuntu.com/advantage System information as of Wed Dec 9 19:47:22 UTC 2020 System load: 0.22 Processes: 110 Usage of /: 29.8% of 9.63GB Users logged in:0 Memory usage: 7%IP address for enp0s3: 10.0.2.15 Swap usage: 0%IP address for docker0: XXX.17.0.1 168 packages can be updated. 121 updates are security updates. New release '20.04.1 LTS' available. Run 'do-release-upgrade' to upgrade to it. *** System restart required *** ``` Seemed good bug gpg was not present in env so closed down... cleaned that up and then on restart got this. ``` stack:release-env stack$ vagrant up Bringing machine 'rmvm' up with 'virtualbox' provider... ==> rmvm: Checking if box 'ubuntu/bionic64' version '20191218.0.0' is up to date... ==> rmvm: Machine already provisioned. Run `vagrant provision` or use the `--provision` ==> rmvm: flag to force provisioning. Provisioners marked to run always will still run. ==> rmvm: Running provisioner: puppet... ==> rmvm: Running Puppet with environment production... ==> rmvm: Info: Loading facts ==> rmvm: Info: Loading facts ==> rmvm: Info: Loading facts ==> rmvm: Info: Loading facts ==> rmvm: Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5 ==> rmvm:(file: /etc/puppet/hiera.yaml) ==> rmvm: Notice: VersionRanges will always be strict when using non-vendored SemanticPuppet gem, version 1.0.2 ==> rmvm: Warning: The function 'hiera_array' is deprecated in favor of using 'lookup'. See https://docs.puppet.com/puppet/5.4/reference/deprecated_language.html\n (file & line not available) ==> rmvm: Warning: The function 'hiera_hash' is deprecated in favor of using 'lookup'. See https://docs.puppet.com/puppet/5.4/reference/deprecated_language.html\n (file & line not available) ==> rmvm: Notice: Compiled catalog for rmvm.apple.com in environment production in 2.01 seconds ==> rmvm: Info: Applying configuration version '1607545060' ==> rmvm: Notice: /Stage[main]/Main/Node[rmvm]/Vcsrepo[/home/vagrant/repos/hbase]/ensure: Updating to latest 'master' revision ==> rmvm: Error: Execution of '/usr/bin/git stash pop' returned 1: No stash entries found. ==> rmvm: Error: /Stage[main]/Main/Node[rmvm]/Vcsrepo[/home/vagrant/repos/hbase]/ensure: change from 'present' to 'latest' failed: Execution of '/usr/bin/git stash pop' returned 1: No stash entries found. ==> rmvm: Notice: Applied catalog in 10.30 seconds The SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed. The output for this command should be in the log above. Please read the output to determine what went wrong. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
ndimiduk commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742024258 Oh, one other idea: sometimes VirtualBox gets stuck in such a way that the only way to resolve it is by launching it's GUI and clicking through whatever error messages it displays. Sometimes I have to delete a VM from the GUI and then resend vagrant in order to get state cleared out of both systems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #2755: HBASE-25376 [create-release] Fix double .asc
saintstack merged pull request #2755: URL: https://github.com/apache/hbase/pull/2755 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
ndimiduk commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-742022909 VirtualBox is really painful to use. It doesn't help that Vagrant and VirtualBox don't always update together. If it helps at all, the vm is working for my with this version pair, ``` $ VBoxManage --version 6.1.16r140961 $ vagrant --version Vagrant 2.2.14 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-19352) Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags
[ https://issues.apache.org/jira/browse/HBASE-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar reassigned HBASE-19352: Assignee: Pankaj Kumar (was: Esteban Gutierrez) > Port HADOOP-10379: Protect authentication cookies with the HttpOnly and > Secure flags > > > Key: HBASE-19352 > URL: https://issues.apache.org/jira/browse/HBASE-19352 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Esteban Gutierrez >Assignee: Pankaj Kumar >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 > > Attachments: HBASE-19352.master.v0.patch > > > This came via a security scanner, since we have a fork of HttpServer2 in > HBase we should include it too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-19352) Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags
[ https://issues.apache.org/jira/browse/HBASE-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar reassigned HBASE-19352: Assignee: Esteban Gutierrez (was: Pankaj Kumar) > Port HADOOP-10379: Protect authentication cookies with the HttpOnly and > Secure flags > > > Key: HBASE-19352 > URL: https://issues.apache.org/jira/browse/HBASE-19352 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 > > Attachments: HBASE-19352.master.v0.patch > > > This came via a security scanner, since we have a fork of HttpServer2 in > HBase we should include it too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25362) Quoting in hbase-vote script fails build with empty ""
[ https://issues.apache.org/jira/browse/HBASE-25362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York resolved HBASE-25362. --- Fix Version/s: 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed > Quoting in hbase-vote script fails build with empty "" > -- > > Key: HBASE-25362 > URL: https://issues.apache.org/jira/browse/HBASE-25362 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.4.1 >Reporter: Zach York >Assignee: Zach York >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > HBASE-25085 added support for maven options to be passed into the hbase-vote > script. However, with the quoting on my machine it fails if I do not pass in > any maven opts: > * Signature: ok > * Checksum : ok > * Rat check (1.8.0_92): failed > - mvn clean apache-rat:check "" > * Built from source (1.8.0_92): failed > - mvn clean install -DskipTests "" > * Unit tests pass (1.8.0_92): failed > - mvn package -P runAllTests "" -Dsurefire.rerunFailingTestsCount=3 > Notice the "" and the build failed due to unknown lifecycle "" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
saintstack commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-741997273 FYI @ndimiduk ``` $ vagrant up Bringing machine 'rmvm' up with 'virtualbox' provider... ==> rmvm: Checking if box 'ubuntu/bionic64' version '20201207.0.0' is up to date... ==> rmvm: Clearing any previously set forwarded ports... ==> rmvm: Clearing any previously set network interfaces... ==> rmvm: Preparing network interfaces based on configuration... rmvm: Adapter 1: nat ==> rmvm: Forwarding ports... rmvm: 22 (guest) => (host) (adapter 1) ==> rmvm: Running 'pre-boot' VM customizations... ==> rmvm: Booting VM... There was an error while executing `VBoxManage`, a CLI used by Vagrant for controlling VirtualBox. The command and stderr is shown below. Command: ["startvm", "e3d1d940-168e-4eb4-9fc8-ff4e46d02203", "--type", "headless"] Stderr: VBoxManage: error: The virtual machine 'rmvm' has terminated unexpectedly during startup with exit code 1 (0x1) VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MachineWrap, interface IMachine ``` I'd started virtualbox ... I updated it, and got this: $ vagrant up The provider 'virtualbox' that was requested to back the machine 'rmvm' is reporting that it isn't usable on this system. The reason is shown below: Vagrant has detected that you have a version of VirtualBox installed that is not supported by this version of Vagrant. Please install one of the supported versions listed below to use Vagrant: 4.0, 4.1, 4.2, 4.3, 5.0, 5.1, 5.2, 6.0 A Vagrant update may also be available that adds support for the version you specified. Please check www.vagrantup.com/downloads.html to download the latest version. (I'd started it and see the rmvm as stopped in listing of machines). Updated vagrant... brew upgrade vagrant. 2.2.4 to 2.2.14. Then it did this again... ``` There was an error while executing `VBoxManage`, a CLI used by Vagrant for controlling VirtualBox. The command and stderr is shown below. Command: ["startvm", "e3d1d940-168e-4eb4-9fc8-ff4e46d02203", "--type", "headless"] Stderr: VBoxManage: error: The virtual machine 'rmvm' has terminated unexpectedly during startup with exit code 1 (0x1) VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MachineWrap, interface IMachine ``` Reinstalled (because net says might work...) brew reinstall virtualbox ... hmm... still same complaint. This is enough comment for one text box. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2748: HBASE-25373 Remove HTrace completely in code base and try to make use…
Apache-HBase commented on pull request #2748: URL: https://github.com/apache/hbase/pull/2748#issuecomment-741985767 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | compile | 2m 49s | master passed | | +1 :green_heart: | shadedjars | 6m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 39s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 0s | the patch passed | | +1 :green_heart: | compile | 2m 49s | the patch passed | | +1 :green_heart: | javac | 2m 49s | the patch passed | | +1 :green_heart: | shadedjars | 6m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 171m 8s | root in the patch passed. | | | | 218m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2748 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9d667e2c005b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c853c99b20 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/3/testReport/ | | Max. process+thread count | 6533 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25277) postScannerFilterRow impacts Scan performance a lot in HBase 2.x
[ https://issues.apache.org/jira/browse/HBASE-25277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246745#comment-17246745 ] Andrew Kyle Purtell commented on HBASE-25277: - +1 Please merge to all branch-2.x. This is a perf regression / bug fix that does not change API semantics if a CP implements those hooks. > postScannerFilterRow impacts Scan performance a lot in HBase 2.x > > > Key: HBASE-25277 > URL: https://issues.apache.org/jira/browse/HBASE-25277 > Project: HBase > Issue Type: Bug > Components: Coprocessors, scan >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Labels: perfomance, scanning > Fix For: 3.0.0-alpha-1, 2.5.0 > > Attachments: test_report.png > > > In our test we observed Scan performance is degraded by more than 60% in > HBase-2.x as compared to 1.3.x. As per the flamegraph report, RS spent 31% > of the time in postScannerFilterRow, however the coprocessors > (AccessController, VisibilityController & ConstraintProcessor) does nothing > in that hook. > HBASE-14489 added the logic to avoid the call to postScannerFilterRow when > not needed which is not working as expected in HBase 2.x. AccessController, > VisibilityController & ConstraintProcessor override the postScannerFilterRow > with dummy (same as RegionObserver) implementation, so > RegionCoprocessorHost.hasCustomPostScannerFilterRow will be TRUE and call the > hook for all configured CPs while processing each row . Suppose we have > configured 5 region CPs and there are 1 M rows in table, then there will be 1 > * 5 M dummy call to postScannerFilterRow during whole table scan. > We need to remove postScannerFilterRow hook from these CPs as these are not > doing anything. > Another problem is in RegionCoprocessorHost.hasCustomPostScannerFilterRow > init logic, currently it is always TRUE even though we remove > postScannerFilterRow hook from AccessController, VisibilityController & > ConstraintProcessor, because we are finding postScannerFilterRow until it > is found (we look in configured CP's super class also) or clazz is NULL. > https://github.com/apache/hbase/blob/035c192eb665469ce0c071db86c78f4a873c123b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L301 > Supper class of Object (super class of AccessController) will be NULL, so > RegionCoprocessorHost.hasCustomPostScannerFilterRow will be set as TRUE > https://github.com/apache/hbase/blob/035c192eb665469ce0c071db86c78f4a873c123b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L279 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] apurtell commented on pull request #2755: HBASE-25376 [create-release] Fix double .asc
apurtell commented on pull request #2755: URL: https://github.com/apache/hbase/pull/2755#issuecomment-741970436 Ah and that's how my manual invocation messed up too, I did "mvn clean install deploy" (sigh) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
ndimiduk commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-741959562 > Anyway, if you think this is useful then let's try it. It's been useful for me. I don't know if it's useful to other release managers. I don't want to add unused code to our repository, so if there isn't another release manager who thinks this will be helpful, let's not merge it. > Just one question, after launching the VM, we will call a release script diectly, or we still need to use docker based release script instead the VM? I run the docker-based release script inside of this VM. I mention this at the very end of the README.md file, but maybe this information should be stated earlier, in a kind of summary usage statement. I'll wait for other RM's to chime in. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25363) Improve performance of HFileLinkCleaner by using ReadWriteLock instead of synchronize
[
https://issues.apache.org/jira/browse/HBASE-25363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246729#comment-17246729
]
Hudson commented on HBASE-25363:
Results for branch branch-2
[build #125 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Improve performance of HFileLinkCleaner by using ReadWriteLock instead of
> synchronize
> -
>
> Key: HBASE-25363
> URL: https://issues.apache.org/jira/browse/HBASE-25363
> Project: HBase
> Issue Type: Improvement
> Components: master
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> HFileLinkCleaner uses synchronized method isFileDeletable, as a result, to
> cross the cleaner chain, all files will be processed serially. It may block
> the fork-join thread which scans out some archived files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25277) postScannerFilterRow impacts Scan performance a lot in HBase 2.x
[
https://issues.apache.org/jira/browse/HBASE-25277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246730#comment-17246730
]
Hudson commented on HBASE-25277:
Results for branch branch-2
[build #125 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> postScannerFilterRow impacts Scan performance a lot in HBase 2.x
>
>
> Key: HBASE-25277
> URL: https://issues.apache.org/jira/browse/HBASE-25277
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, scan
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Labels: perfomance, scanning
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: test_report.png
>
>
> In our test we observed Scan performance is degraded by more than 60% in
> HBase-2.x as compared to 1.3.x. As per the flamegraph report, RS spent 31%
> of the time in postScannerFilterRow, however the coprocessors
> (AccessController, VisibilityController & ConstraintProcessor) does nothing
> in that hook.
> HBASE-14489 added the logic to avoid the call to postScannerFilterRow when
> not needed which is not working as expected in HBase 2.x. AccessController,
> VisibilityController & ConstraintProcessor override the postScannerFilterRow
> with dummy (same as RegionObserver) implementation, so
> RegionCoprocessorHost.hasCustomPostScannerFilterRow will be TRUE and call the
> hook for all configured CPs while processing each row . Suppose we have
> configured 5 region CPs and there are 1 M rows in table, then there will be 1
> * 5 M dummy call to postScannerFilterRow during whole table scan.
> We need to remove postScannerFilterRow hook from these CPs as these are not
> doing anything.
> Another problem is in RegionCoprocessorHost.hasCustomPostScannerFilterRow
> init logic, currently it is always TRUE even though we remove
> postScannerFilterRow hook from AccessController, VisibilityController &
> ConstraintProcessor, because we are finding postScannerFilterRow until it
> is found (we look in configured CP's super class also) or clazz is NULL.
> https://github.com/apache/hbase/blob/035c192eb665469ce0c071db86c78f4a873c123b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L301
> Supper class of Object (super class of AccessController) will be NULL, so
> RegionCoprocessorHost.hasCustomPostScannerFilterRow will be set as TRUE
> https://github.com/apache/hbase/blob/035c192eb665469ce0c071db86c78f4a873c123b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L279
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25328) Add builder method to create Tags.
[
https://issues.apache.org/jira/browse/HBASE-25328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246728#comment-17246728
]
Hudson commented on HBASE-25328:
Results for branch branch-2
[build #125 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/125/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add builder method to create Tags.
> --
>
> Key: HBASE-25328
> URL: https://issues.apache.org/jira/browse/HBASE-25328
> Project: HBase
> Issue Type: Improvement
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1
>
>
> In PHOENIX-6213 phoenix project is using Cell Tag feature to add some
> metadata for delete mutations. We are adding Cell Tags in co-processor but we
> need some util methods available in +PrivateCellUtil+ class.
> Below are the methods we need in phoenix.
> 1. +PrivateCellUtil#createCell(Cell cell, List tags)+ method has an api
> which will accept an existing Cell and list of tags to create a new cell.
> But RawCellBuilder has a builder method which doesn't have any method which
> accepts a cell. I need to explicitly convert my input cell by extracting all
> fields and use the builder methods (like setRow, setseFamily, etc) and then
> use the build method.
>
> 2. +PrivateCellUtil.getTags(Cell cell)+ returns a list of existing tags which
> I want to use and add a new tag.
> But RawCell#getTags() returns Iterator which then I have to iterate
> over them and depending on whether they are byte buffer backed or array
> backed, I need to convert them to List since RawCellBuilder#setTags accepts
> List of Tags. We are already doing this conversion in PrivateCellUtil#getTags
> method.
> All these conversion utility methods needs to be duplicated in phoenix
> project also.
>
> Is it reasonable to make PrivateCellUtil method LimitedPrivate with
> HBaseInterfaceAudience as COPROC ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2757: HBASE-25379 Make intial pause time configurable for regionserver while sending reportRegionStateTransition
Apache-HBase commented on pull request #2757: URL: https://github.com/apache/hbase/pull/2757#issuecomment-741950171 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 46s | master passed | | +1 :green_heart: | compile | 1m 55s | master passed | | +1 :green_heart: | shadedjars | 9m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 27s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 19s | the patch passed | | +1 :green_heart: | compile | 1m 53s | the patch passed | | +1 :green_heart: | javac | 1m 53s | the patch passed | | +1 :green_heart: | shadedjars | 8m 43s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 3s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 136m 49s | hbase-server in the patch passed. | | | | 178m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2757/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2757 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 399c71031a12 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c853c99b20 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2757/1/testReport/ | | Max. process+thread count | 3933 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2757/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2757: HBASE-25379 Make intial pause time configurable for regionserver while sending reportRegionStateTransition
Apache-HBase commented on pull request #2757: URL: https://github.com/apache/hbase/pull/2757#issuecomment-741946178 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 42s | master passed | | +1 :green_heart: | compile | 1m 21s | master passed | | +1 :green_heart: | shadedjars | 6m 30s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | +1 :green_heart: | shadedjars | 6m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 32s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 141m 13s | hbase-server in the patch passed. | | | | 171m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2757/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2757 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux de3761ebe91f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c853c99b20 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2757/1/testReport/ | | Max. process+thread count | 4946 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2757/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25328) Add builder method to create Tags.
[
https://issues.apache.org/jira/browse/HBASE-25328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246699#comment-17246699
]
Hudson commented on HBASE-25328:
Results for branch branch-2.3
[build #122 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/122/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/122/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/122/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/122/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/122/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add builder method to create Tags.
> --
>
> Key: HBASE-25328
> URL: https://issues.apache.org/jira/browse/HBASE-25328
> Project: HBase
> Issue Type: Improvement
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1
>
>
> In PHOENIX-6213 phoenix project is using Cell Tag feature to add some
> metadata for delete mutations. We are adding Cell Tags in co-processor but we
> need some util methods available in +PrivateCellUtil+ class.
> Below are the methods we need in phoenix.
> 1. +PrivateCellUtil#createCell(Cell cell, List tags)+ method has an api
> which will accept an existing Cell and list of tags to create a new cell.
> But RawCellBuilder has a builder method which doesn't have any method which
> accepts a cell. I need to explicitly convert my input cell by extracting all
> fields and use the builder methods (like setRow, setseFamily, etc) and then
> use the build method.
>
> 2. +PrivateCellUtil.getTags(Cell cell)+ returns a list of existing tags which
> I want to use and add a new tag.
> But RawCell#getTags() returns Iterator which then I have to iterate
> over them and depending on whether they are byte buffer backed or array
> backed, I need to convert them to List since RawCellBuilder#setTags accepts
> List of Tags. We are already doing this conversion in PrivateCellUtil#getTags
> method.
> All these conversion utility methods needs to be duplicated in phoenix
> project also.
>
> Is it reasonable to make PrivateCellUtil method LimitedPrivate with
> HBaseInterfaceAudience as COPROC ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2756: HBASE-25378 Legacy comparator in Hfile trailer will fail to load
Apache-HBase commented on pull request #2756: URL: https://github.com/apache/hbase/pull/2756#issuecomment-741916198 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 22s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 43s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 7m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 39s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 7m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 194m 35s | hbase-server in the patch passed. | | | | 225m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2756 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e0334633f83d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fbe338de1a | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/testReport/ | | Max. process+thread count | 3216 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun merged pull request #2726: HBASE-25293 Followup jira to address the client handling issue when c…
huaxiangsun merged pull request #2726: URL: https://github.com/apache/hbase/pull/2726 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25378) Legacy comparator in Hfile trailer will fail to load
[ https://issues.apache.org/jira/browse/HBASE-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246683#comment-17246683 ] ramkrishna.s.vasudevan commented on HBASE-25378: Ya Pankaj. Sorry even I missed this part. how ever it is good to fix it in the getComparatorClass() to handle this change. > Legacy comparator in Hfile trailer will fail to load > > > Key: HBASE-25378 > URL: https://issues.apache.org/jira/browse/HBASE-25378 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.3.2 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the > deadlock issue. But this introduced compatibility issue, old hfile with > comparator class as > "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open > due to ClassNotFoundException. > Also we should also handle the case when comparator class is > "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was > case before HBASE-24968. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] huaxiangsun commented on pull request #2726: HBASE-25293 Followup jira to address the client handling issue when c…
huaxiangsun commented on pull request #2726: URL: https://github.com/apache/hbase/pull/2726#issuecomment-741912765 Thanks Stack for review, pushing the changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2726: HBASE-25293 Followup jira to address the client handling issue when c…
huaxiangsun commented on a change in pull request #2726:
URL: https://github.com/apache/hbase/pull/2726#discussion_r539488382
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
##
@@ -262,16 +261,16 @@ private void cleanupReplicaReplicaStaleCache() {
private int refreshCatalogReplicaCount() {
int newNumOfReplicas = this.getNumOfReplicas.getAsInt();
LOG.debug("Refreshed replica count {}", newNumOfReplicas);
-if (newNumOfReplicas == 1) {
- LOG.warn("Table {}'s region replica count is 1, maybe a misconfiguration
or failure to "
-+ "fetch the replica count", tableName);
+// If the returned number of replicas is -1, it is caused by failure to
fetch the
+// replica count. Do not update the numOfReplicas in this case.
+if (newNumOfReplicas ==
CatalogReplicaLoadBalanceSelector.UNINITIALIZED_NUM_OF_REPLICAS) {
+ LOG.error("Failed to fetch Table {}'s region replica count", tableName);
Review comment:
Currently, it is logged every 2 minutes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2755: HBASE-25376 [create-release] Fix double .asc
saintstack commented on pull request #2755: URL: https://github.com/apache/hbase/pull/2755#issuecomment-741904830 @apurtell One liner now. No need of install target when you call deploy. See JIRA for 'proof' change works. #2758 has the logging+README change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2755: HBASE-25376 [create-release] Fix double .asc
Apache-HBase commented on pull request #2755: URL: https://github.com/apache/hbase/pull/2755#issuecomment-741904298 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2755: HBASE-25376 [create-release] Fix double .asc
Apache-HBase commented on pull request #2755: URL: https://github.com/apache/hbase/pull/2755#issuecomment-741904191 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2755/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2755 | | Optional Tests | | | uname | Linux ab063fc240ed 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c853c99b20 | | Max. process+thread count | 45 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2755/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2748: HBASE-25373 Remove HTrace completely in code base and try to make use…
Apache-HBase commented on pull request #2748: URL: https://github.com/apache/hbase/pull/2748#issuecomment-741901367 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 43s | master passed | | +1 :green_heart: | checkstyle | 2m 11s | master passed | | +1 :green_heart: | spotbugs | 18m 19s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | -0 :warning: | checkstyle | 2m 12s | root: The patch generated 1 new + 562 unchanged - 23 fixed = 563 total (was 585) | | -0 :warning: | rubocop | 0m 4s | The patch generated 4 new + 3 unchanged - 8 fixed = 7 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 16s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 17m 25s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 19m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 2m 54s | The patch does not generate ASF License warnings. | | | | 81m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2748 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle xml rubocop | | uname | Linux de93306a89d1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c853c99b20 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/3/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/3/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 137 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2748/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2758: HBASE-25380 [create-release] Add timestamping to log output
Apache-HBase commented on pull request #2758: URL: https://github.com/apache/hbase/pull/2758#issuecomment-741901165 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 4s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2758/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2758 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 9ba0d174dbaf 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c853c99b20 | | Max. process+thread count | 51 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2758/1/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2758: HBASE-25380 [create-release] Add timestamping to log output
Apache-HBase commented on pull request #2758: URL: https://github.com/apache/hbase/pull/2758#issuecomment-741900775 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2758/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2758 | | Optional Tests | | | uname | Linux 2d765bc744df 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c853c99b20 | | Max. process+thread count | 47 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2758/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2758: HBASE-25380 [create-release] Add timestamping to log output
Apache-HBase commented on pull request #2758: URL: https://github.com/apache/hbase/pull/2758#issuecomment-741900659 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2758/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2758 | | Optional Tests | | | uname | Linux 9698d6e1403c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c853c99b20 | | Max. process+thread count | 49 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2758/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-25378) Legacy comparator in Hfile trailer will fail to load
[ https://issues.apache.org/jira/browse/HBASE-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246669#comment-17246669 ] Pankaj Kumar edited comment on HBASE-25378 at 12/9/20, 4:49 PM: Thanks [~anoop.hbase] for looking into this. Yeah I missed FixedFileTrailer.getHBase1CompatibleName(String), HBASE-24968 wont break the data compatibility. Anyway let me test and confim. was (Author: pankajkumar): Yeah I missed FixedFileTrailer.getHBase1CompatibleName(String), HBASE-24968 wont break the data compatibility. Anyway let me test and confim. > Legacy comparator in Hfile trailer will fail to load > > > Key: HBASE-25378 > URL: https://issues.apache.org/jira/browse/HBASE-25378 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.3.2 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the > deadlock issue. But this introduced compatibility issue, old hfile with > comparator class as > "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open > due to ClassNotFoundException. > Also we should also handle the case when comparator class is > "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was > case before HBASE-24968. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25378) Legacy comparator in Hfile trailer will fail to load
[ https://issues.apache.org/jira/browse/HBASE-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246669#comment-17246669 ] Pankaj Kumar commented on HBASE-25378: -- Yeah I missed FixedFileTrailer.getHBase1CompatibleName(String), HBASE-24968 wont break the data compatibility. Anyway let me test and confim. > Legacy comparator in Hfile trailer will fail to load > > > Key: HBASE-25378 > URL: https://issues.apache.org/jira/browse/HBASE-25378 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.3.2 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the > deadlock issue. But this introduced compatibility issue, old hfile with > comparator class as > "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open > due to ClassNotFoundException. > Also we should also handle the case when comparator class is > "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was > case before HBASE-24968. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack opened a new pull request #2758: HBASE-25380 [create-release] Add timestamping to log output
saintstack opened a new pull request #2758: URL: https://github.com/apache/hbase/pull/2758 Added logging of timestamp so we can tell where we are spending time. Added context to the README copied from head of entrance script. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25380) [create-release] Add timestamping to log output
Michael Stack created HBASE-25380: - Summary: [create-release] Add timestamping to log output Key: HBASE-25380 URL: https://issues.apache.org/jira/browse/HBASE-25380 Project: HBase Issue Type: Sub-task Components: create-release Reporter: Michael Stack Add a timestamp when we output a status message. Without it is more difficult figuring where the time is being spent building. Updated the README too adding some setup text copied from the head of the script. The README is better place for this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] pankaj72981 commented on a change in pull request #2756: HBASE-25378 Legacy comparator in Hfile trailer will fail to load
pankaj72981 commented on a change in pull request #2756:
URL: https://github.com/apache/hbase/pull/2756#discussion_r539468971
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java
##
@@ -612,6 +612,8 @@ private String getHBase1CompatibleName(final String
comparator) {
comparatorKlass = CellComparatorImpl.class;
} else if
(comparatorClassName.equals(KeyValue.META_COMPARATOR.getLegacyKeyComparatorName())
||
comparatorClassName.equals(KeyValue.META_COMPARATOR.getClass().getName())
+ ||
(comparatorClassName.equals("org.apache.hadoop.hbase.CellComparator$MetaCellComparator"))
+ ||
(comparatorClassName.equals("org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator"))
Review comment:
Before HBASE-24968, MetaCellComparator.class.getName() was
org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25376) [create-release] Fix double .asc
[ https://issues.apache.org/jira/browse/HBASE-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246658#comment-17246658 ] Michael Stack commented on HBASE-25376: --- !Screen Shot 2020-12-09 at 8.24.56 AM.png! This staged release was done with the script before the fix. Notice .asc.asc. This is after the fix... no .asc.asc !Screen Shot 2020-12-09 at 8.24.07 AM.png! > [create-release] Fix double .asc > > > Key: HBASE-25376 > URL: https://issues.apache.org/jira/browse/HBASE-25376 > Project: HBase > Issue Type: Bug > Components: create-release >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Attachments: Screen Shot 2020-12-09 at 8.24.07 AM.png, Screen Shot > 2020-12-09 at 8.24.56 AM.png > > > We are signing the signature file when we run the create-release which > results in *.jar.asc.asc files up in maven repository. It looks to be this > line in release-util.sh maven_deploy: > - declare -a mvn_goals=(clean install) > + declare -a mvn_goals=(clean) > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on pull request #2755: HBASE-25376 [create-release] Fix double .asc
saintstack commented on pull request #2755: URL: https://github.com/apache/hbase/pull/2755#issuecomment-741892699 bq. 99% of the changes are out of scope for the fix? np This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25376) [create-release] Fix double .asc
[ https://issues.apache.org/jira/browse/HBASE-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-25376: -- Attachment: Screen Shot 2020-12-09 at 8.24.56 AM.png > [create-release] Fix double .asc > > > Key: HBASE-25376 > URL: https://issues.apache.org/jira/browse/HBASE-25376 > Project: HBase > Issue Type: Bug > Components: create-release >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Attachments: Screen Shot 2020-12-09 at 8.24.07 AM.png, Screen Shot > 2020-12-09 at 8.24.56 AM.png > > > We are signing the signature file when we run the create-release which > results in *.jar.asc.asc files up in maven repository. It looks to be this > line in release-util.sh maven_deploy: > - declare -a mvn_goals=(clean install) > + declare -a mvn_goals=(clean) > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25376) [create-release] Fix double .asc
[ https://issues.apache.org/jira/browse/HBASE-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-25376: -- Attachment: Screen Shot 2020-12-09 at 8.24.07 AM.png > [create-release] Fix double .asc > > > Key: HBASE-25376 > URL: https://issues.apache.org/jira/browse/HBASE-25376 > Project: HBase > Issue Type: Bug > Components: create-release >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Attachments: Screen Shot 2020-12-09 at 8.24.07 AM.png, Screen Shot > 2020-12-09 at 8.24.56 AM.png > > > We are signing the signature file when we run the create-release which > results in *.jar.asc.asc files up in maven repository. It looks to be this > line in release-util.sh maven_deploy: > - declare -a mvn_goals=(clean install) > + declare -a mvn_goals=(clean) > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25378) Legacy comparator in Hfile trailer will fail to load
[
https://issues.apache.org/jira/browse/HBASE-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246648#comment-17246648
]
Anoop Sam John commented on HBASE-25378:
At first thought its a data compatibility break issue.
Seems this wont create a compatibility issue by not able to read old cluster
generated HFiles. Because when we write the comparator class name in FFT, we
still use 1.x based comparator names
See FFT#toProtobuf ()
{code}
.setComparatorClassName(getHBase1CompatibleName(comparatorClassName))
..
private String getHBase1CompatibleName(final String comparator) {
if (comparator.equals(CellComparatorImpl.class.getName())) {
return KeyValue.COMPARATOR.getClass().getName();
}
if (comparator.equals(MetaCellComparator.class.getName())) {
return KeyValue.META_COMPARATOR.getClass().getName();
}
return comparator;
}
{code}
Though u can confirm with functional tests once.
> Legacy comparator in Hfile trailer will fail to load
>
>
> Key: HBASE-25378
> URL: https://issues.apache.org/jira/browse/HBASE-25378
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.3.2
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4
>
>
> HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the
> deadlock issue. But this introduced compatibility issue, old hfile with
> comparator class as
> "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open
> due to ClassNotFoundException.
> Also we should also handle the case when comparator class is
> "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was
> case before HBASE-24968.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2756: HBASE-25378 Legacy comparator in Hfile trailer will fail to load
Apache-HBase commented on pull request #2756: URL: https://github.com/apache/hbase/pull/2756#issuecomment-741883535 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | master passed | | +1 :green_heart: | compile | 0m 57s | master passed | | +1 :green_heart: | shadedjars | 6m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 32s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 6m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 146m 21s | hbase-server in the patch passed. | | | | 172m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2756 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 93e254e1204b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fbe338de1a | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/testReport/ | | Max. process+thread count | 4557 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25378) Legacy comparator in Hfile trailer will fail to load
[ https://issues.apache.org/jira/browse/HBASE-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-25378: - Description: HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the deadlock issue. But this introduced compatibility issue, old hfile with comparator class as "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open due to ClassNotFoundException. Also we should also handle the case when comparator class is "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was case before HBASE-24968. was: HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the deadlock issue. But this introduced incompatibility issue, old hfile with comparator class as "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open due to ClassNotFoundException. Also we should also handle the case when comparator class is "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was case before HBASE-24968. > Legacy comparator in Hfile trailer will fail to load > > > Key: HBASE-25378 > URL: https://issues.apache.org/jira/browse/HBASE-25378 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.3.2 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the > deadlock issue. But this introduced compatibility issue, old hfile with > comparator class as > "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open > due to ClassNotFoundException. > Also we should also handle the case when comparator class is > "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was > case before HBASE-24968. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25378) Legacy comparator in Hfile trailer will fail to load
[ https://issues.apache.org/jira/browse/HBASE-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-25378: --- Fix Version/s: 2.3.4 2.4.0 > Legacy comparator in Hfile trailer will fail to load > > > Key: HBASE-25378 > URL: https://issues.apache.org/jira/browse/HBASE-25378 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.3.2 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > HBASE-24968 moved MetaCellComparator out from CellComparatorImpl to avoid the > deadlock issue. But this introduced incompatibility issue, old hfile with > comparator class as > "org.apache.hadoop.hbase.CellComparator$MetaCellComparator" will fail to open > due to ClassNotFoundException. > Also we should also handle the case when comparator class is > "org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator", which was > case before HBASE-24968. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2757: HBASE-25379 Make intial pause time configurable for regionserver while sending reportRegionStateTransition
Apache-HBase commented on pull request #2757: URL: https://github.com/apache/hbase/pull/2757#issuecomment-741860359 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 38s | master passed | | +1 :green_heart: | checkstyle | 1m 37s | master passed | | +1 :green_heart: | spotbugs | 2m 42s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | checkstyle | 1m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 25s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 42m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2757/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2757 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 3b357330c773 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c853c99b20 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2757/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2756: HBASE-25378 Legacy comparator in Hfile trailer will fail to load
virajjasani commented on a change in pull request #2756:
URL: https://github.com/apache/hbase/pull/2756#discussion_r539393476
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java
##
@@ -612,6 +612,8 @@ private String getHBase1CompatibleName(final String
comparator) {
comparatorKlass = CellComparatorImpl.class;
} else if
(comparatorClassName.equals(KeyValue.META_COMPARATOR.getLegacyKeyComparatorName())
||
comparatorClassName.equals(KeyValue.META_COMPARATOR.getClass().getName())
+ ||
(comparatorClassName.equals("org.apache.hadoop.hbase.CellComparator$MetaCellComparator"))
+ ||
(comparatorClassName.equals("org.apache.hadoop.hbase.CellComparatorImpl$MetaCellComparator"))
Review comment:
Is `CellComparator` not enough? I just checked previous commit where
only `CellComparator` was present and not `CellComparatorImpl`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-25379) Make intial pause time configurable for regionserver while sending reportRegionStateTransition
[ https://issues.apache.org/jira/browse/HBASE-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-25379: - Fix Version/s: 3.0.0-alpha-1 Status: Patch Available (was: Open) > Make intial pause time configurable for regionserver while sending > reportRegionStateTransition > -- > > Key: HBASE-25379 > URL: https://issues.apache.org/jira/browse/HBASE-25379 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Currently initPauseTime is hard coded as 1000, > > https://github.com/apache/hbase/blob/c853c99b209cb2dfefb8d15595d9a9a5469e24fa/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L2440 > But in case of > ServerNotRunningYetException/PleaseHoldException/CallQueueTooBigException > RegionServer may wait for a long time (In our case it was ~7 min). > Currently we calculate pause time based on the number of retries and > RETRY_BACKOFF. So in a failure case RS will wait max 200 sec and wait will be > always 200 sec if retries further goes beyond 13. > We should make it configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] pankaj72981 opened a new pull request #2757: HBASE-25379 Make intial pause time configurable for regionserver while sending reportRegionStateTransition
pankaj72981 opened a new pull request #2757: URL: https://github.com/apache/hbase/pull/2757 Added "hbase.rpc.shortoperation.pause.time" for initial pause for reportRegionStateTransition. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25379) Make intial pause time configurable for regionserver while sending reportRegionStateTransition
Pankaj Kumar created HBASE-25379: Summary: Make intial pause time configurable for regionserver while sending reportRegionStateTransition Key: HBASE-25379 URL: https://issues.apache.org/jira/browse/HBASE-25379 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Pankaj Kumar Assignee: Pankaj Kumar Currently initPauseTime is hard coded as 1000, https://github.com/apache/hbase/blob/c853c99b209cb2dfefb8d15595d9a9a5469e24fa/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L2440 But in case of ServerNotRunningYetException/PleaseHoldException/CallQueueTooBigException RegionServer may wait for a long time (In our case it was ~7 min). Currently we calculate pause time based on the number of retries and RETRY_BACKOFF. So in a failure case RS will wait max 200 sec and wait will be always 200 sec if retries further goes beyond 13. We should make it configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25373) Remove HTrace completely in code base and try to make use of OpenTelemetry
[ https://issues.apache.org/jira/browse/HBASE-25373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246555#comment-17246555 ] Duo Zhang commented on HBASE-25373: --- After learning the docs of open-telemetry-java and opentelemetry-java-instrumentation, I think for a modern style tracing framework for java, the typical way is to support tracing through a java agent, without touching the code, and all the related configs for tracing, will be passed in through command line when starting. Anyway, I think the basic rule here, is to minimize the modification on java code. For HBase, I do not know whether it is easy to implement an agent and push it into open telemetry. It is not designed for better tracing, so let's try to use the opentelemetry-api to manually add tracing support first. Later we could see if it is possible to use an agent to do the instrumentation automatically. And also, I think we should remove all the sampler related code, as it will pull in the dependency of sdk and exporter. Will update the PR soon. > Remove HTrace completely in code base and try to make use of OpenTelemetry > -- > > Key: HBASE-25373 > URL: https://issues.apache.org/jira/browse/HBASE-25373 > Project: HBase > Issue Type: Sub-task > Components: tracing >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2756: HBASE-25378 Legacy comparator in Hfile trailer will fail to load
Apache-HBase commented on pull request #2756: URL: https://github.com/apache/hbase/pull/2756#issuecomment-741794200 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 45s | master passed | | +1 :green_heart: | checkstyle | 1m 7s | master passed | | +1 :green_heart: | spotbugs | 2m 2s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 2 new + 4 unchanged - 0 fixed = 6 total (was 4) | | -0 :warning: | whitespace | 0m 0s | The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 17m 18s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 38m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2756 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 6956fda5f4a5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fbe338de1a | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2756/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
