[jira] [Comment Edited] (HBASE-25709) Close region may stuck when region is compacting and skipped most cells read
[
https://issues.apache.org/jira/browse/HBASE-25709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553539#comment-17553539
]
Xiaolin Ha edited comment on HBASE-25709 at 6/14/22 5:58 AM:
-
Thanks for the excellent digging, [~vjasani] .
The problem here is that, after returned by the heartbeat cells, the matcher
was unexpectedly reset,
{code:java}
// If no limits exists in the scope LimitScope.Between_Cells then we are sure
we are changing
// rows. Else it is possible we are still traversing the same row so we must
perform the row
// comparison.
if (!scannerContext.hasAnyLimit(LimitScope.BETWEEN_CELLS) ||
matcher.currentRow() == null) {
this.countPerRow = 0;
matcher.setToNewRow(cell);
} {code}
then all the deletes of the row is cleared, and the next same row cells will be
wrongly matched. This means the scanner context before and after the heartbeat
return is inconsistent, it should hold the same row contexts until new rows.
As in the test case you provided, when
StoreScanner.HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK=2 and mid-results are
[q4,q5], and the heartbeat count reached after [q8/delete], then the [q8/put]
will be wrongly added to results because matcher does not has [q8/delete].
I think for this issue, we can use the HRegion#checkInterrupt to avoid the
stuck when closing region. Just as codes here,
{code:java}
// when reaching the heartbeat cells, try to return from the loop.
if (kvsScanned % cellsPerHeartbeatCheck == 0) {
- return
scannerContext.setScannerState(NextState.MORE_VALUES).hasMoreValues();
+ this.store.getHRegion().checkInterrupt();
} {code}
But we can also fix the matcher issue, because I think the matcher reset should
only happens when scanner reaching new rows. And the early return when reached
heartbeat cells can make user scanners return as soon as possible when
encounters mass deletion. The fix codes are,
{code:java}
@@ -276,9 +278,12 @@ public abstract class ScanQueryMatcher implements
ShipperListener {
* @param currentRow
*/
public void setToNewRow(Cell currentRow) {
- this.currentRow = currentRow;
- columns.reset();
- reset();
+ if (this.currentRow == null
+ || this.rowComparator.compareRows(currentRow, this.currentRow) != 0) {
+ this.currentRow = currentRow;
+ columns.reset();
+ reset();
+ }
} {code}
I prefer the second solution, what do you think? [~vjasani] [~apurtell]
was (Author: xiaolin ha):
Thanks for the excellent digging, [~vjasani] .
The problem here is that, after returned by the heartbeat cells, the matcher
was unexpectedly reset,
{code:java}
// If no limits exists in the scope LimitScope.Between_Cells then we are sure
we are changing
// rows. Else it is possible we are still traversing the same row so we must
perform the row
// comparison.
if (!scannerContext.hasAnyLimit(LimitScope.BETWEEN_CELLS) ||
matcher.currentRow() == null) {
this.countPerRow = 0;
matcher.setToNewRow(cell);
} {code}
then all the deletes of the row is cleared, and the next same row cells will be
wrongly matched.
As in the test case you provided, when
StoreScanner.HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK=2 and mid-results are
[q4,q5], and the heartbeat count reached after [q8/delete], then the [q8/put]
will be wrongly added to results because matcher does not has [q8/delete].
I think for this issue, we can use the HRegion#checkInterrupt to avoid the
stuck when closing region. Just as codes here,
{code:java}
// when reaching the heartbeat cells, try to return from the loop.
if (kvsScanned % cellsPerHeartbeatCheck == 0) {
- return
scannerContext.setScannerState(NextState.MORE_VALUES).hasMoreValues();
+ this.store.getHRegion().checkInterrupt();
} {code}
But we can also fix the matcher issue, because I think the matcher reset show
only happens when scanner reached new rows. And the early return when reached
heartbeat cells can make user scanners return as soon as possible when
encounters mass deletion. The fix codes are,
{code:java}
@@ -276,9 +278,12 @@ public abstract class ScanQueryMatcher implements
ShipperListener {
* @param currentRow
*/
public void setToNewRow(Cell currentRow) {
- this.currentRow = currentRow;
- columns.reset();
- reset();
+ if (this.currentRow == null
+ || this.rowComparator.compareRows(currentRow, this.currentRow) != 0) {
+ this.currentRow = currentRow;
+ columns.reset();
+ reset();
+ }
} {code}
I prefer the second solution, what do you think? [~vjasani] [~apurtell]
> Close region may stuck when region is compacting and skipped most cells read
>
>
> Key: HBASE-25709
> URL: https://issues.apache.org/jira/browse/HBASE-25709
>
[GitHub] [hbase] Apache-HBase commented on pull request #4532: HBASE-27117 Update the method comments for RegionServerAccounting
Apache-HBase commented on PR #4532: URL: https://github.com/apache/hbase/pull/4532#issuecomment-1154731797 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 25s | master passed | | +1 :green_heart: | compile | 0m 43s | master passed | | +1 :green_heart: | shadedjars | 4m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 27s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 46s | the patch passed | | +1 :green_heart: | compile | 0m 45s | the patch passed | | +1 :green_heart: | javac | 0m 45s | the patch passed | | +1 :green_heart: | shadedjars | 4m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 193m 48s | hbase-server in the patch failed. | | | | 214m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4532 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7ffbf48c53c9 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / f526609331 | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/testReport/ | | Max. process+thread count | 2502 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4532: HBASE-27117 Update the method comments for RegionServerAccounting
Apache-HBase commented on PR #4532: URL: https://github.com/apache/hbase/pull/4532#issuecomment-1154730948 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 27s | master passed | | +1 :green_heart: | compile | 0m 34s | master passed | | +1 :green_heart: | shadedjars | 3m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 0m 35s | the patch passed | | +1 :green_heart: | javac | 0m 35s | the patch passed | | +1 :green_heart: | shadedjars | 3m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 197m 32s | hbase-server in the patch failed. | | | | 213m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4532 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f5e20c16d8e9 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / f526609331 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/testReport/ | | Max. process+thread count | 2462 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26466) Immutable timeseries usecase - Create new region rather than split existing one
[
https://issues.apache.org/jira/browse/HBASE-26466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553898#comment-17553898
]
Viraj Jasani commented on HBASE-26466:
--
Let me get back in a couple of days on this.
> Immutable timeseries usecase - Create new region rather than split existing
> one
> ---
>
> Key: HBASE-26466
> URL: https://issues.apache.org/jira/browse/HBASE-26466
> Project: HBase
> Issue Type: Brainstorming
>Reporter: Viraj Jasani
>Priority: Major
>
> For insertion of immutable data usecase (specifically time-series data),
> region split mechanism doesn't seem to provide better availability when
> ingestion rate is very high. When we ingest lot of data, the region split
> policy tries to split the given hot region based on the size (either size of
> all stores combined or size of any single store exceeding max file size
> configured) if we consider default {_}SteppingSplitPolicy{_}. The latest hot
> regions tend to receive all latest inserts. When the region is split, the
> first half of the region (say daughterA) stays on the same server whereas the
> second half (daughterB) region – likely to become another hot region because
> all new latest updates come to second half region in the sequential write
> fashion – is moved out to other servers in the cluster. Hence, once new
> daughter region is created, client traffic will be redirected to another
> server. Client requests will be piled up when region split is triggered till
> new daughters come alive and once done, client will have to request meta for
> updated daughter region and redirect traffic to new server.
> If we could have configurable region creation strategy that 1) keeps the
> split disabled for the given table, and 2) create new region dynamically with
> lexicographically higher start key on the same server and update it's own
> region boundary, the client will have to look up meta once and continue
> ingestion without any degraded SLA caused by region split transitions.
> Note: region split might also encounter some complications, requiring the
> procedure to be rolled back from some step, or continue with internal
> retries, eventually further delaying the ingestion from clients.
>
> There are some complications around updating live region's start and end keys
> as this key range is immutable. We could brainstorm ideas around making them
> optionally mutable and any issues around them. For instance, client might
> continue writing data to the region with updated end key but writes will fail
> for out of range keys and hence, they will lookup in meta for updated
> key-space range (new region created with end key: EMPTY_END_ROW) of the table.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26856) BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted
[
https://issues.apache.org/jira/browse/HBASE-26856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553893#comment-17553893
]
Hudson commented on HBASE-26856:
Results for branch branch-2.4
[build #371 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/371/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/371/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/371/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/371/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/371/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted
> --
>
> Key: HBASE-26856
> URL: https://issues.apache.org/jira/browse/HBASE-26856
> Project: HBase
> Issue Type: Bug
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13
>
>
> In our production cluster we observed the cell value is modified after
> successful scanner read. After analyzing we observed OnheapDecodedCell is not
> created properly.
> We create OnheapDecodedCell with complete valAndTagsBuffer underlying array.
> {code:java}
> return new OnheapDecodedCell(Bytes.copy(keyBuffer, 0, this.keyLength),
> currentKey.getRowLength(), currentKey.getFamilyOffset(),
> currentKey.getFamilyLength(),
> currentKey.getQualifierOffset(), currentKey.getQualifierLength(),
> currentKey.getTimestamp(), currentKey.getTypeByte(),
> valAndTagsBuffer.array(),
> valAndTagsBuffer.arrayOffset() + vOffset, this.valueLength,
> memstoreTS, tagsArray,
> tOffset, this.tagsLength);
> {code}
> Here we are passing valAndTagsBuffer.array() for value extraction.
> The underlying array will be modified if it is altered anywhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4532: HBASE-27117 Update the method comments for RegionServerAccounting
Apache-HBase commented on PR #4532: URL: https://github.com/apache/hbase/pull/4532#issuecomment-1154640783 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 16s | master passed | | +1 :green_heart: | compile | 2m 15s | master passed | | +1 :green_heart: | checkstyle | 0m 29s | master passed | | +1 :green_heart: | spotless | 0m 50s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 8s | the patch passed | | +1 :green_heart: | compile | 2m 16s | the patch passed | | +1 :green_heart: | javac | 2m 16s | the patch passed | | +1 :green_heart: | checkstyle | 0m 26s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 0s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotless | 0m 38s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 34m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4532 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 9e2cf534d901 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / f526609331 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4532/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] 2005hithlj commented on pull request #4493: HBASE-27091 Speed up the loading of table descriptor from filesystem
2005hithlj commented on PR #4493: URL: https://github.com/apache/hbase/pull/4493#issuecomment-1154636650 The UT 'TestMultiRespectsLimits' is ok in my environment. @carp84 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4531: [HBASE-27116]fix the ruby issue which cause disable all table failure
Apache-HBase commented on PR #4531: URL: https://github.com/apache/hbase/pull/4531#issuecomment-1154625153 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 53s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 32s | master passed | | +1 :green_heart: | javadoc | 0m 21s | master passed | | -0 :warning: | patch | 4m 1s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 5s | the patch passed | | +1 :green_heart: | javadoc | 0m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 8m 3s | hbase-shell in the patch passed. | | | | 17m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4531/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4531 | | JIRA Issue | HBASE-27116 | | Optional Tests | javac javadoc unit | | uname | Linux ed40905be62d 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / f526609331 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4531/1/testReport/ | | Max. process+thread count | 1536 (vs. ulimit of 3) | | modules | C: hbase-shell U: hbase-shell | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4531/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4531: [HBASE-27116]fix the ruby issue which cause disable all table failure
Apache-HBase commented on PR #4531: URL: https://github.com/apache/hbase/pull/4531#issuecomment-1154624280 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 54s | master passed | | +1 :green_heart: | javadoc | 0m 13s | master passed | | -0 :warning: | patch | 3m 15s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 41s | the patch passed | | +1 :green_heart: | javadoc | 0m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 6m 49s | hbase-shell in the patch passed. | | | | 15m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4531/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4531 | | JIRA Issue | HBASE-27116 | | Optional Tests | javac javadoc unit | | uname | Linux 98445ed1dd3f 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / f526609331 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4531/1/testReport/ | | Max. process+thread count | 1523 (vs. ulimit of 3) | | modules | C: hbase-shell U: hbase-shell | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4531/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4531: [HBASE-27116]fix the ruby issue which cause disable all table failure
Apache-HBase commented on PR #4531: URL: https://github.com/apache/hbase/pull/4531#issuecomment-1154620981 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 24s | master passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | -0 :warning: | patch | 0m 48s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 4s | the patch passed | | -0 :warning: | rubocop | 0m 12s | The patch generated 1 new + 31 unchanged - 0 fixed = 32 total (was 31) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | spotless | 0m 39s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 8m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4531/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4531 | | JIRA Issue | HBASE-27116 | | Optional Tests | dupname asflicense javac spotless rubocop | | uname | Linux 80933bca80be 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / f526609331 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | rubocop | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4531/1/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-shell U: hbase-shell | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4531/1/console | | versions | git=2.17.1 maven=3.6.3 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-27116) Non-interactive missing for the hbase-shell
[ https://issues.apache.org/jira/browse/HBASE-27116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553851#comment-17553851 ] Dong0829 edited comment on HBASE-27116 at 6/14/22 1:45 AM: --- After a add back the eval_io, its working well for the non-interactive https://github.com/apache/hbase/pull/4531 was (Author: li0829): https://github.com/apache/hbase/pull/4531 > Non-interactive missing for the hbase-shell > --- > > Key: HBASE-27116 > URL: https://issues.apache.org/jira/browse/HBASE-27116 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.12 >Reporter: Dong0829 >Priority: Major > > During test 2.4.12, we found a lot of our integration testing failure for the > non-interactive mode in hbase-shell, looks like in 2.4.12, we removed the > "shell.eval_io" for the non-interactive shell, this caused the input/output > handle behavior for shell is the same for interactive and non-interactive > which caused unexpected error, for example: > > ``` > echo -e "disable_all '.*'\ny" | hbase shell -n > =Old version 2.4.4= > t1 > Disable the above 1 tables (y/n)? > 1 tables successfully disabled > Took 0.7811 seconds > =New version 2.4.12= > hbase:001:0> disable_all '.*'t1 > test_tableDisable the above 2 tables (y/n)? > Took 0.5270 seconds > NoMethodError: undefined method `chomp' for nil:NilClass > command at > uri:classloader:/shell/commands/disable_all.rb:41 > command_safe at uri:classloader:/shell/commands.rb:49 > translate_hbase_exceptions at uri:classloader:/shell/commands.rb:122 > command_safe at uri:classloader:/shell/commands.rb:49 > > run at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 >at classpath:/jar-bootstrap.rb:223}} > ``` -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] tomscut opened a new pull request, #4532: HBASE-27117 Update the method comments for RegionServerAccounting
tomscut opened a new pull request, #4532: URL: https://github.com/apache/hbase/pull/4532 JIRA: HBASE-27117. After [HBASE-15787](https://issues.apache.org/jira/browse/HBASE-15787), the return value type of RegionServerAccounting#isAboveHighWaterMark and RegionServerAccounting#isAboveLowWaterMark are no longer boolean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27116) Non-interactive missing for the hbase-shell
[ https://issues.apache.org/jira/browse/HBASE-27116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong0829 updated HBASE-27116: - Status: Patch Available (was: Open) https://github.com/apache/hbase/pull/4531 > Non-interactive missing for the hbase-shell > --- > > Key: HBASE-27116 > URL: https://issues.apache.org/jira/browse/HBASE-27116 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.12 >Reporter: Dong0829 >Priority: Major > > During test 2.4.12, we found a lot of our integration testing failure for the > non-interactive mode in hbase-shell, looks like in 2.4.12, we removed the > "shell.eval_io" for the non-interactive shell, this caused the input/output > handle behavior for shell is the same for interactive and non-interactive > which caused unexpected error, for example: > > ``` > echo -e "disable_all '.*'\ny" | hbase shell -n > =Old version 2.4.4= > t1 > Disable the above 1 tables (y/n)? > 1 tables successfully disabled > Took 0.7811 seconds > =New version 2.4.12= > hbase:001:0> disable_all '.*'t1 > test_tableDisable the above 2 tables (y/n)? > Took 0.5270 seconds > NoMethodError: undefined method `chomp' for nil:NilClass > command at > uri:classloader:/shell/commands/disable_all.rb:41 > command_safe at uri:classloader:/shell/commands.rb:49 > translate_hbase_exceptions at uri:classloader:/shell/commands.rb:122 > command_safe at uri:classloader:/shell/commands.rb:49 > > run at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 >at classpath:/jar-bootstrap.rb:223}} > ``` -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27117) Update the method comments for RegionServerAccounting
Tao Li created HBASE-27117: -- Summary: Update the method comments for RegionServerAccounting Key: HBASE-27117 URL: https://issues.apache.org/jira/browse/HBASE-27117 Project: HBase Issue Type: Bug Reporter: Tao Li Assignee: Tao Li After HBASE-15787, the return value type of RegionServerAccounting#isAboveHighWaterMark and RegionServerAccounting#isAboveLowWaterMark are no longer boolean. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HBASE-27116) Non-interactive missing for the hbase-shell
[ https://issues.apache.org/jira/browse/HBASE-27116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong0829 updated HBASE-27116: - Description: During test 2.4.12, we found a lot of our integration testing failure for the non-interactive mode in hbase-shell, looks like in 2.4.12, we removed the "shell.eval_io" for the non-interactive shell, this caused the input/output handle behavior for shell is the same for interactive and non-interactive which caused unexpected error, for example: ``` echo -e "disable_all '.*'\ny" | hbase shell -n =Old version 2.4.4= t1 Disable the above 1 tables (y/n)? 1 tables successfully disabled Took 0.7811 seconds =New version 2.4.12= hbase:001:0> disable_all '.*'t1 test_tableDisable the above 2 tables (y/n)? Took 0.5270 seconds NoMethodError: undefined method `chomp' for nil:NilClass command at uri:classloader:/shell/commands/disable_all.rb:41 command_safe at uri:classloader:/shell/commands.rb:49 translate_hbase_exceptions at uri:classloader:/shell/commands.rb:122 command_safe at uri:classloader:/shell/commands.rb:49 run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 at classpath:/jar-bootstrap.rb:223}} ``` was: During test 2.4.12, we found a lot of our integration testing failure for the non-interactive mode in hbase-shell, looks like in 2.4.12, we removed the "shell.eval_io" for the non-interactive shell, this caused the input/output handle behavior for shell is the same which caused unexpected error, for example: ``` echo -e "disable_all '.*'\ny" | hbase shell -n =Old version 2.4.4= t1 Disable the above 1 tables (y/n)? 1 tables successfully disabled Took 0.7811 seconds =New version 2.4.12= hbase:001:0> disable_all '.*'t1 test_tableDisable the above 2 tables (y/n)? Took 0.5270 seconds NoMethodError: undefined method `chomp' for nil:NilClass command at uri:classloader:/shell/commands/disable_all.rb:41 command_safe at uri:classloader:/shell/commands.rb:49 translate_hbase_exceptions at uri:classloader:/shell/commands.rb:122 command_safe at uri:classloader:/shell/commands.rb:49 run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 at classpath:/jar-bootstrap.rb:223}} ``` > Non-interactive missing for the hbase-shell > --- > > Key: HBASE-27116 > URL: https://issues.apache.org/jira/browse/HBASE-27116 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.12 >Reporter: Dong0829 >Priority: Major > > During test 2.4.12, we found a lot of our integration testing failure for the > non-interactive mode in hbase-shell, looks like in 2.4.12, we removed the > "shell.eval_io" for the non-interactive shell, this caused the input/output > handle behavior for shell is the same for interactive and non-interactive > which caused unexpected error, for example: > > ``` > echo -e "disable_all '.*'\ny" | hbase shell -n > =Old version 2.4.4= > t1 > Disable the above 1 tables (y/n)? > 1 tables successfully disabled > Took 0.7811 seconds > =New version 2.4.12= > hbase:001:0> disable_all '.*'t1 > test_tableDisable the above 2 tables (y/n)? > Took 0.5270 seconds > NoMethodError: undefined method `chomp' for nil:NilClass > command at > uri:classloader:/shell/commands/disable_all.rb:41 > command_safe at uri:classloader:/shell/commands.rb:49 > translate_hbase_exceptions at uri:classloader:/shell/commands.rb:122 > command_safe at uri:classloader:/shell/commands.rb:49 > > run at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 >at classpath:/jar-bootstrap.rb:223}} > ``` -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HBASE-27116) Non-interactive missing for the hbase-shell
[ https://issues.apache.org/jira/browse/HBASE-27116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong0829 updated HBASE-27116: - Description: During test 2.4.12, we found a lot of our integration testing failure for the non-interactive mode in hbase-shell, looks like in 2.4.12, we removed the "shell.eval_io" for the non-interactive shell, this caused the input/output handle behavior for shell is the same which caused unexpected error, for example: ``` echo -e "disable_all '.*'\ny" | hbase shell -n =Old version 2.4.4= t1 Disable the above 1 tables (y/n)? 1 tables successfully disabled Took 0.7811 seconds =New version 2.4.12= hbase:001:0> disable_all '.*'t1 test_tableDisable the above 2 tables (y/n)? Took 0.5270 seconds NoMethodError: undefined method `chomp' for nil:NilClass command at uri:classloader:/shell/commands/disable_all.rb:41 command_safe at uri:classloader:/shell/commands.rb:49 translate_hbase_exceptions at uri:classloader:/shell/commands.rb:122 command_safe at uri:classloader:/shell/commands.rb:49 run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 at classpath:/jar-bootstrap.rb:223}} ``` was: During test 2.4.12, we found a lot of our integration testing failure for the non-interactive mode in hbase-shell, looks like in 2.4.12, we removed the "shell.eval_io" for the non-interactive shell, this caused the input/output handle behavior for shell is the same which caused unexpected error, for example: ``` echo -e "disable_all '.*'\ny" | hbase shell -n =Old version 2.4.4= t1 Disable the above 1 tables (y/n)?1 tables successfully disabled Took 0.7811 seconds =New version 2.4.12= hbase:001:0> disable_all '.*'t1 test_tableDisable the above 2 tables (y/n)? Took 0.5270 seconds NoMethodError: undefined method `chomp' for nil:NilClass command at uri:classloader:/shell/commands/disable_all.rb:41 command_safe at uri:classloader:/shell/commands.rb:49 translate_hbase_exceptions at uri:classloader:/shell/commands.rb:122 command_safe at uri:classloader:/shell/commands.rb:49 run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 at classpath:/jar-bootstrap.rb:223}} ``` > Non-interactive missing for the hbase-shell > --- > > Key: HBASE-27116 > URL: https://issues.apache.org/jira/browse/HBASE-27116 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.12 >Reporter: Dong0829 >Priority: Major > > During test 2.4.12, we found a lot of our integration testing failure for the > non-interactive mode in hbase-shell, looks like in 2.4.12, we removed the > "shell.eval_io" for the non-interactive shell, this caused the input/output > handle behavior for shell is the same which caused unexpected error, for > example: > > ``` > echo -e "disable_all '.*'\ny" | hbase shell -n > =Old version 2.4.4= > t1 > Disable the above 1 tables (y/n)? > 1 tables successfully disabled > Took 0.7811 seconds > =New version 2.4.12= > hbase:001:0> disable_all '.*'t1 > test_tableDisable the above 2 tables (y/n)? > Took 0.5270 seconds > NoMethodError: undefined method `chomp' for nil:NilClass > command at > uri:classloader:/shell/commands/disable_all.rb:41 > command_safe at uri:classloader:/shell/commands.rb:49 > translate_hbase_exceptions at uri:classloader:/shell/commands.rb:122 > command_safe at uri:classloader:/shell/commands.rb:49 > > run at > uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 >at classpath:/jar-bootstrap.rb:223}} > ``` -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] alexdongli0829 opened a new pull request, #4531: fix the ruby issue which cause disable all table failure
alexdongli0829 opened a new pull request, #4531: URL: https://github.com/apache/hbase/pull/4531 https://issues.apache.org/jira/browse/HBASE-27116 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27116) Non-interactive missing for the hbase-shell
Dong0829 created HBASE-27116: Summary: Non-interactive missing for the hbase-shell Key: HBASE-27116 URL: https://issues.apache.org/jira/browse/HBASE-27116 Project: HBase Issue Type: Bug Components: shell Affects Versions: 2.4.12 Reporter: Dong0829 During test 2.4.12, we found a lot of our integration testing failure for the non-interactive mode in hbase-shell, looks like in 2.4.12, we removed the "shell.eval_io" for the non-interactive shell, this caused the input/output handle behavior for shell is the same which caused unexpected error, for example: ``` echo -e "disable_all '.*'\ny" | hbase shell -n =Old version 2.4.4= t1 Disable the above 1 tables (y/n)?1 tables successfully disabled Took 0.7811 seconds =New version 2.4.12= hbase:001:0> disable_all '.*'t1 test_tableDisable the above 2 tables (y/n)? Took 0.5270 seconds NoMethodError: undefined method `chomp' for nil:NilClass command at uri:classloader:/shell/commands/disable_all.rb:41 command_safe at uri:classloader:/shell/commands.rb:49 translate_hbase_exceptions at uri:classloader:/shell/commands.rb:122 command_safe at uri:classloader:/shell/commands.rb:49 run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:427 at classpath:/jar-bootstrap.rb:223}} ``` -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-27028) Add a shell command for flushing master local region
[
https://issues.apache.org/jira/browse/HBASE-27028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553838#comment-17553838
]
Hudson commented on HBASE-27028:
Results for branch master
[build #611 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add a shell command for flushing master local region
> -
>
> Key: HBASE-27028
> URL: https://issues.apache.org/jira/browse/HBASE-27028
> Project: HBase
> Issue Type: New Feature
> Components: shell
>Affects Versions: 3.0.0-alpha-3
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
> Fix For: 3.0.0-alpha-3
>
>
> As the data in the ‘master:store’ table becomes more and more important, it
> is necessary to add a shell command to manually trigger the flush operation
> on this table.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27106) HBase site generation should use its own JRuby
[
https://issues.apache.org/jira/browse/HBASE-27106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553839#comment-17553839
]
Hudson commented on HBASE-27106:
Results for branch master
[build #611 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HBase site generation should use its own JRuby
> --
>
> Key: HBASE-27106
> URL: https://issues.apache.org/jira/browse/HBASE-27106
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0-alpha-3
>
>
> The maven plugin we use to generate the reference guide,
> {{asciidoctor-maven-plugin}}, does not yet work with JRuby 9.3.
> Due to a dependency error we currently get a mix of JRuby 9.2 and 9.3
> libraries when building the site goal, which fails without an actionable
> error message. Correcting the dependency to consistently get JRuby 9.3 also
> results in failure, but due to gem resolution.
> We originally started overriding the JRuby version to get Aarch64 support.
> The version that ships with the plugin has this support already.
> Falling back to JRuby 9.2 for the plugin will mean our reference guide
> building won't work on M1 systems, but I don't see how we can change that
> until the plugin updates for JRuby 9.3 changes.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-25709) Close region may stuck when region is compacting and skipped most cells read
[
https://issues.apache.org/jira/browse/HBASE-25709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553837#comment-17553837
]
Hudson commented on HBASE-25709:
Results for branch master
[build #611 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/611/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Close region may stuck when region is compacting and skipped most cells read
>
>
> Key: HBASE-25709
> URL: https://issues.apache.org/jira/browse/HBASE-25709
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 1.7.1, 3.0.0-alpha-2, 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
> Attachments: Master-UI-RIT.png, RS-region-state.png
>
>
> We found in our cluster about stop region stuck. The region is compacting,
> and its store files has many TTL expired cells. Close region state
> marker(HRegion#writestate.writesEnabled) is not checked in compaction,
> because most cells were skipped.
> !RS-region-state.png|width=698,height=310!
>
> !Master-UI-RIT.png|width=693,height=157!
>
> HBASE-23968 has encountered similar problem, but the solution in it is outer
> the method
> InternalScanner#next(List result, ScannerContext scannerContext), which
> will not return if there are many skipped cells, for current compaction
> scanner context. As a result, we need to return in time in the next method,
> and then check the stop marker.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4530: HBASE-27025 Change Hbase book's description for 74.7.3. Load Balancin…
Apache-HBase commented on PR #4530: URL: https://github.com/apache/hbase/pull/4530#issuecomment-1154441502 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 31s | master passed | | +0 :ok: | refguide | 2m 49s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 2m 6s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 13m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4530/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4530 | | Optional Tests | dupname asflicense refguide spotless | | uname | Linux 70acbe46080f 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / f526609331 | | refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4530/1/yetus-general-check/output/branch-site/book.html | | refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4530/1/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 70 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4530/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4530: HBASE-27025 Change Hbase book's description for 74.7.3. Load Balancin…
Apache-HBase commented on PR #4530: URL: https://github.com/apache/hbase/pull/4530#issuecomment-1154432039 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 2m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4530/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4530 | | Optional Tests | | | uname | Linux 3cb629d0370d 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / f526609331 | | Max. process+thread count | 40 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4530/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4530: HBASE-27025 Change Hbase book's description for 74.7.3. Load Balancin…
Apache-HBase commented on PR #4530: URL: https://github.com/apache/hbase/pull/4530#issuecomment-1154431645 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 2m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4530/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4530 | | Optional Tests | | | uname | Linux dedd53d8cca2 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / f526609331 | | Max. process+thread count | 29 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4530/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26366) Provide meaningful parent spans to ZK interactions
[
https://issues.apache.org/jira/browse/HBASE-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553777#comment-17553777
]
Hudson commented on HBASE-26366:
Results for branch branch-2.5
[build #142 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide meaningful parent spans to ZK interactions
> --
>
> Key: HBASE-26366
> URL: https://issues.apache.org/jira/browse/HBASE-26366
> Project: HBase
> Issue Type: Sub-task
> Components: tracing
>Affects Versions: 2.5.0, 3.0.0-alpha-2
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
> Attachments: image.png,
> regroup-spans-for-recoverablezookeeper.create.png
>
>
> Enable tracing with {{-Dotel.traces.sampler=always_on}} and launch the
> standalone master, check the trace store. We see lots of single-span traces,
> each a call to different {{RecoverableZooKeeper}} methods, like {{create}}
> and {{exists}}. Whatever master startup thread this is should define its own
> root span.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-25709) Close region may stuck when region is compacting and skipped most cells read
[
https://issues.apache.org/jira/browse/HBASE-25709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553776#comment-17553776
]
Hudson commented on HBASE-25709:
Results for branch branch-2.5
[build #142 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/142/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Close region may stuck when region is compacting and skipped most cells read
>
>
> Key: HBASE-25709
> URL: https://issues.apache.org/jira/browse/HBASE-25709
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 1.7.1, 3.0.0-alpha-2, 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
> Attachments: Master-UI-RIT.png, RS-region-state.png
>
>
> We found in our cluster about stop region stuck. The region is compacting,
> and its store files has many TTL expired cells. Close region state
> marker(HRegion#writestate.writesEnabled) is not checked in compaction,
> because most cells were skipped.
> !RS-region-state.png|width=698,height=310!
>
> !Master-UI-RIT.png|width=693,height=157!
>
> HBASE-23968 has encountered similar problem, but the solution in it is outer
> the method
> InternalScanner#next(List result, ScannerContext scannerContext), which
> will not return if there are many skipped cells, for current compaction
> scanner context. As a result, we need to return in time in the next method,
> and then check the stop marker.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] huaxiangsun opened a new pull request, #4530: HBASE-27025 Change Hbase book's description for 74.7.3. Load Balancin…
huaxiangsun opened a new pull request, #4530: URL: https://github.com/apache/hbase/pull/4530 …g META table load -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-20951) Ratis LogService backed WALs
[ https://issues.apache.org/jira/browse/HBASE-20951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved HBASE-20951. Resolution: Later > Ratis LogService backed WALs > > > Key: HBASE-20951 > URL: https://issues.apache.org/jira/browse/HBASE-20951 > Project: HBase > Issue Type: New Feature > Components: wal >Reporter: Josh Elser >Priority: Major > > Umbrella issue for the Ratis+WAL work: > Design doc: > [https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit#|https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit] > The (over-simplified) goal is to re-think the current WAL APIs we have now, > ensure that they are de-coupled from the notion of being backed by HDFS, swap > the current implementations over to the new API, and then wire up the Ratis > LogService to the new WAL API. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-20951) Ratis LogService backed WALs
[
https://issues.apache.org/jira/browse/HBASE-20951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553775#comment-17553775
]
Josh Elser commented on HBASE-20951:
{quote}I am thinking of resolving this and all subtasks as WontFix or Abandoned.
{quote}
Yeah, I think a "Later" is appropriate. Let me do this.
> Ratis LogService backed WALs
>
>
> Key: HBASE-20951
> URL: https://issues.apache.org/jira/browse/HBASE-20951
> Project: HBase
> Issue Type: New Feature
> Components: wal
>Reporter: Josh Elser
>Priority: Major
>
> Umbrella issue for the Ratis+WAL work:
> Design doc:
> [https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit#|https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit]
> The (over-simplified) goal is to re-think the current WAL APIs we have now,
> ensure that they are de-coupled from the notion of being backed by HDFS, swap
> the current implementations over to the new API, and then wire up the Ratis
> LogService to the new WAL API.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Closed] (HBASE-6716) Hadoopqa is hosed
[ https://issues.apache.org/jira/browse/HBASE-6716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6716. -- > Hadoopqa is hosed > - > > Key: HBASE-6716 > URL: https://issues.apache.org/jira/browse/HBASE-6716 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Attachments: rm_support_dir.txt > > > See this thread on list: > http://search-hadoop.com/m/PtDLC19vEd62/%2522Looks+like+HadoopQA+is+hosed%2522=Looks+like+HadoopQA+is+hosed+ > Lots of the hadoopqa builds are failing complaining about missing dir. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6695) [Replication] Data will lose if RegionServer down during transferqueue
[ https://issues.apache.org/jira/browse/HBASE-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6695. -- > [Replication] Data will lose if RegionServer down during transferqueue > -- > > Key: HBASE-6695 > URL: https://issues.apache.org/jira/browse/HBASE-6695 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.94.1 >Reporter: terry zhang >Priority: Critical > Attachments: HBASE-6695-4trunk.patch, HBASE-6695-4trunk_v2.patch, > HBASE-6695-4trunk_v3.patch, HBASE-6695.patch > > > When we ware testing Replication failover feature we found if we kill a > regionserver during it transferqueue ,we found only part of the hlog znode > copy to the right path because failover process is interrupted. > Log: > 2012-08-29 12:20:05,660 INFO > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager: > Moving dw92.kgb.sqa.cm4,60020,1346210789716's hlogs to my queue > 2012-08-29 12:20:05,765 DEBUG > org.apache.hadoop.hbase.replication.ReplicationZookeeper: Creating > dw92.kgb.sqa.cm4%2C60020%2C13462107 89716.1346213720708 with data 210508162 > 2012-08-29 12:20:05,850 DEBUG > org.apache.hadoop.hbase.replication.ReplicationZookeeper: Creating > dw92.kgb.sqa.cm4%2C60020%2C13462107 89716.1346213886800 with data > 2012-08-29 12:20:05,938 DEBUG > org.apache.hadoop.hbase.replication.ReplicationZookeeper: Creating > dw92.kgb.sqa.cm4%2C60020%2C1346210789716.1346213830559 with data > 2012-08-29 12:20:06,055 DEBUG > org.apache.hadoop.hbase.replication.ReplicationZookeeper: Creating > dw92.kgb.sqa.cm4%2C60020%2C1346210789716.1346213775146 with data > 2012-08-29 12:20:06,277 WARN > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Failed all from region=.ME > TA.,,1.1028785192, hostname=dw93.kgb.sqa.cm4, port=60020 > java.util.concurrent.ExecutionException: java.net.ConnectException: > Connection refused > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) > at java.util.concurrent.FutureTask.get(FutureTask.java:83) > at > .. > This server is down . > ZK node status: > [zk: 10.232.98.77:2181(CONNECTED) 6] ls > /hbase-test3-repl/replication/rs/dw92.kgb.sqa.cm4,60020,1346210789716 > [lock, 1, 1-dw89.kgb.sqa.cm4,60020,1346202436268] > > dw92 is down , but Node dw92.kgb.sqa.cm4,60020,1346210789716 can't be deleted -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6678) TestMergeTool picks unlikely ZK port that could conflict
[
https://issues.apache.org/jira/browse/HBASE-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell closed HBASE-6678.
--
> TestMergeTool picks unlikely ZK port that could conflict
>
>
> Key: HBASE-6678
> URL: https://issues.apache.org/jira/browse/HBASE-6678
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.95.2
>Reporter: Gregory Chanan
>Priority: Trivial
>
> In TestMergeTool, we pick an "unlikely" port for zookeeper to be on. Perhaps
> it would be better if this was a privileged port < 1024.
> {code}
> // Make it so we try and connect to a zk that is not there (else we might
> // find a zk ensemble put up by another concurrent test and this will
> // mess up this test. Choose unlikely port. Default test port is 21818.
> // Default zk port is 2181.
> this.conf.setInt(HConstants.ZOOKEEPER_CLIENT_PORT, 10001);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Closed] (HBASE-6687) [replication] Improve the handling of deserialization exceptions in replication
[ https://issues.apache.org/jira/browse/HBASE-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6687. -- > [replication] Improve the handling of deserialization exceptions in > replication > --- > > Key: HBASE-6687 > URL: https://issues.apache.org/jira/browse/HBASE-6687 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Chris Trezzo >Priority: Minor > > If a clusterkey for a replication peer cluster can not be deserialized, we > log a warning and continue. This should probably be a stop the world event, > because we can no longer connect to the peer cluster. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6690) TestZooKeeperTableArchiveClient.testMultipleTables is flapping
[ https://issues.apache.org/jira/browse/HBASE-6690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6690. -- Assignee: (was: Jesse Yates) > TestZooKeeperTableArchiveClient.testMultipleTables is flapping > -- > > Key: HBASE-6690 > URL: https://issues.apache.org/jira/browse/HBASE-6690 > Project: HBase > Issue Type: Test >Reporter: Chris Trezzo >Priority: Minor > > TestZooKeeperTableArchiveClient.testMultipleTables is a flapping test. It is > complaining that some archived HFiles were not deleted. > Test history: > https://builds.apache.org/job/HBase-TRUNK/3293/testReport/junit/org.apache.hadoop.hbase.backup.example/TestZooKeeperTableArchiveClient/testMultipleTables/history/ > Error message: > Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:\[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c\] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-6719) [replication] Data will lose if open a Hlog failed more than maxRetriesMultiplier
[
https://issues.apache.org/jira/browse/HBASE-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell resolved HBASE-6719.
Assignee: (was: terry zhang)
Resolution: Abandoned
> [replication] Data will lose if open a Hlog failed more than
> maxRetriesMultiplier
> -
>
> Key: HBASE-6719
> URL: https://issues.apache.org/jira/browse/HBASE-6719
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: terry zhang
>Priority: Critical
> Attachments: 6719.txt, hbase-6719.patch
>
>
> Please Take a look below code
> {code:title=ReplicationSource.java|borderStyle=solid}
> protected boolean openReader(int sleepMultiplier) {
> {
> ...
> catch (IOException ioe) {
> LOG.warn(peerClusterZnode + " Got: ", ioe);
> // TODO Need a better way to determinate if a file is really gone but
> // TODO without scanning all logs dir
> if (sleepMultiplier == this.maxRetriesMultiplier) {
> LOG.warn("Waited too long for this file, considering dumping");
> return !processEndOfFile(); // Open a file failed over
> maxRetriesMultiplier(default 10)
> }
> }
> return true;
> ...
> }
> protected boolean processEndOfFile() {
> if (this.queue.size() != 0) {// Skipped this Hlog . Data loss
> this.currentPath = null;
> this.position = 0;
> return true;
> } else if (this.queueRecovered) { // Terminate Failover Replication
> source thread ,data loss
> this.manager.closeRecoveredQueue(this);
> LOG.info("Finished recovering the queue");
> this.running = false;
> return true;
> }
> return false;
> }
> {code}
> Some Time HDFS will meet some problem but actually Hlog file is OK , So after
> HDFS back ,Some data will lose and can not find them back in slave cluster.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Closed] (HBASE-6719) [replication] Data will lose if open a Hlog failed more than maxRetriesMultiplier
[
https://issues.apache.org/jira/browse/HBASE-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell closed HBASE-6719.
--
> [replication] Data will lose if open a Hlog failed more than
> maxRetriesMultiplier
> -
>
> Key: HBASE-6719
> URL: https://issues.apache.org/jira/browse/HBASE-6719
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: terry zhang
>Priority: Critical
> Attachments: 6719.txt, hbase-6719.patch
>
>
> Please Take a look below code
> {code:title=ReplicationSource.java|borderStyle=solid}
> protected boolean openReader(int sleepMultiplier) {
> {
> ...
> catch (IOException ioe) {
> LOG.warn(peerClusterZnode + " Got: ", ioe);
> // TODO Need a better way to determinate if a file is really gone but
> // TODO without scanning all logs dir
> if (sleepMultiplier == this.maxRetriesMultiplier) {
> LOG.warn("Waited too long for this file, considering dumping");
> return !processEndOfFile(); // Open a file failed over
> maxRetriesMultiplier(default 10)
> }
> }
> return true;
> ...
> }
> protected boolean processEndOfFile() {
> if (this.queue.size() != 0) {// Skipped this Hlog . Data loss
> this.currentPath = null;
> this.position = 0;
> return true;
> } else if (this.queueRecovered) { // Terminate Failover Replication
> source thread ,data loss
> this.manager.closeRecoveredQueue(this);
> LOG.info("Finished recovering the queue");
> this.running = false;
> return true;
> }
> return false;
> }
> {code}
> Some Time HDFS will meet some problem but actually Hlog file is OK , So after
> HDFS back ,Some data will lose and can not find them back in slave cluster.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Closed] (HBASE-6699) Setting username in Connection in non-secure HBase
[ https://issues.apache.org/jira/browse/HBASE-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6699. -- Assignee: (was: Himanshu Vashishtha) > Setting username in Connection in non-secure HBase > -- > > Key: HBASE-6699 > URL: https://issues.apache.org/jira/browse/HBASE-6699 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC >Affects Versions: 0.92.0, 0.92.1, 0.94.0, 0.94.1 >Reporter: Himanshu Vashishtha >Priority: Major > Attachments: HBase-6699-v1.patch > > > We recently had a requirement where we need to log the information about > various users who were using non-secure HBase cluster. > The user level logging is supported as part of security, but in 0.92, 0.94 > security related code is separate. This jira is about adding that support in > non-secure code. > This feature is already there in trunk, after we merge the security related > code. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6704) StoreScanner KV usage improvement
[ https://issues.apache.org/jira/browse/HBASE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6704. -- > StoreScanner KV usage improvement > - > > Key: HBASE-6704 > URL: https://issues.apache.org/jira/browse/HBASE-6704 > Project: HBase > Issue Type: Improvement > Components: regionserver > Environment: 89-fb >Reporter: Ramkumar Vadali >Priority: Trivial > Attachments: > 0001-jira-HBASE-6704-Improve-KV-usage-in-StoreScanner-nex.patch > > > StoreScanner#next tries to ensure that the KVs are pulled from the heap > in-order. The prevKV is used for this and is compared with the latest KV > pulled from the heap. Since copyKV can be modified by ScanQueryMatcher, > prevKV should track the KV pulled from the heap, and not copyKV. This is a > one-line fix. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6703) Increment does not check resources
[ https://issues.apache.org/jira/browse/HBASE-6703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6703. -- > Increment does not check resources > -- > > Key: HBASE-6703 > URL: https://issues.apache.org/jira/browse/HBASE-6703 > Project: HBase > Issue Type: Bug > Components: regionserver > Environment: 89-fb >Reporter: Ramkumar Vadali >Priority: Major > > The increment code does not check for resource constraints. This could cause > problems if it is called frequently. The other modifying code paths like put, > delete etc call checkResources(). This is a one-line fix -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6705) Refactor duplicate CF parsing code in ThriftServerRunner
[ https://issues.apache.org/jira/browse/HBASE-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6705. -- > Refactor duplicate CF parsing code in ThriftServerRunner > - > > Key: HBASE-6705 > URL: https://issues.apache.org/jira/browse/HBASE-6705 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Gryzykhin >Priority: Minor > > We have a lot of repetitive code conditional on famAndQf.length <= 1. > Refactoring that into a function. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6709) Refactoring QOS levels convention
[ https://issues.apache.org/jira/browse/HBASE-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6709. -- Assignee: (was: Himanshu Vashishtha) > Refactoring QOS levels convention > -- > > Key: HBASE-6709 > URL: https://issues.apache.org/jira/browse/HBASE-6709 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, regionserver >Affects Versions: 0.94.0 >Reporter: Himanshu Vashishtha >Priority: Minor > > Current naming convention about QOS is confusing. For first, there is no high > or low priority, rather it works based on whether the operation is a > root/meta lookup or a pure client side one. > With addition of ReplicationHandlers, we introduced a new level of QOS. > I think we should standarized QOS levels that reflects how they actually > works. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6712) Implement checkAndIncrement
[ https://issues.apache.org/jira/browse/HBASE-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6712. -- > Implement checkAndIncrement > --- > > Key: HBASE-6712 > URL: https://issues.apache.org/jira/browse/HBASE-6712 > Project: HBase > Issue Type: New Feature > Components: Client >Affects Versions: 0.92.1 >Reporter: Stefan Baritchii >Priority: Major > > increment should throw an exception if a row does not exist. instead it > creates the row. checkAndIncrement may be also a solution to it but needs > development. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6696) Add CP hooks pre and post split transaction point-of-no-return
[
https://issues.apache.org/jira/browse/HBASE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell closed HBASE-6696.
--
> Add CP hooks pre and post split transaction point-of-no-return
> --
>
> Key: HBASE-6696
> URL: https://issues.apache.org/jira/browse/HBASE-6696
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors, regionserver
>Affects Versions: 0.94.2, 0.95.2
>Reporter: Andrew Kyle Purtell
>Priority: Major
>
> In the discussion "Improving Coprocessor postSplit/postOpen synchronization"
> on user@hbase, a user implementing a secondary indexing scheme writes in:
> {quote}
> The goal with splits is to create two new daughter regions with the
> corresponding splits of the secondary indexes and lock these regions such
> that Puts/Deletes that occur while postSplit is in progress will be queued
> up so we don't run into consistency issues. [...] As of right now, the HBase
> coprocessors do not easily support a way to achieve this level of consistency
> in that there is no way to distinguish a region being opened from a split or
> a regular open.
> {quote}
> Setting aside the particulars of the use case, the issue is the {{preSplit}}
> hook does not provide the identities of the daughter regions (they don't
> exist yet) and the {{postSplit}} hook, while providing the identities of the
> daughter regions, runs after the master has processed the split and the
> daughters are already opened or opening. A CP implementer has no interception
> point available where the daughters exist but are not yet open.
> This JIRA proposes to add two new CP hooks to just before and after the
> point-of-no-return (PONR) in the split transaction: {{preSplitPONR}} and
> {{postSplitPONR}}. Perhaps the naming can be improved. This will address the
> above use case but additionally support overloading of the split transaction
> itself. We also need to address observer notification if the split
> transaction fails. This is like HBASE-5827 but scoped to this specific
> consideration only.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (HBASE-6712) Implement checkAndIncrement
[ https://issues.apache.org/jira/browse/HBASE-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-6712. Assignee: (was: Ted Yu) Resolution: Later > Implement checkAndIncrement > --- > > Key: HBASE-6712 > URL: https://issues.apache.org/jira/browse/HBASE-6712 > Project: HBase > Issue Type: New Feature > Components: Client >Affects Versions: 0.92.1 >Reporter: Stefan Baritchii >Priority: Major > > increment should throw an exception if a row does not exist. instead it > creates the row. checkAndIncrement may be also a solution to it but needs > development. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-6696) Add CP hooks pre and post split transaction point-of-no-return
[
https://issues.apache.org/jira/browse/HBASE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell resolved HBASE-6696.
Resolution: Won't Fix
> Add CP hooks pre and post split transaction point-of-no-return
> --
>
> Key: HBASE-6696
> URL: https://issues.apache.org/jira/browse/HBASE-6696
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors, regionserver
>Affects Versions: 0.94.2, 0.95.2
>Reporter: Andrew Kyle Purtell
>Priority: Major
>
> In the discussion "Improving Coprocessor postSplit/postOpen synchronization"
> on user@hbase, a user implementing a secondary indexing scheme writes in:
> {quote}
> The goal with splits is to create two new daughter regions with the
> corresponding splits of the secondary indexes and lock these regions such
> that Puts/Deletes that occur while postSplit is in progress will be queued
> up so we don't run into consistency issues. [...] As of right now, the HBase
> coprocessors do not easily support a way to achieve this level of consistency
> in that there is no way to distinguish a region being opened from a split or
> a regular open.
> {quote}
> Setting aside the particulars of the use case, the issue is the {{preSplit}}
> hook does not provide the identities of the daughter regions (they don't
> exist yet) and the {{postSplit}} hook, while providing the identities of the
> daughter regions, runs after the master has processed the split and the
> daughters are already opened or opening. A CP implementer has no interception
> point available where the daughters exist but are not yet open.
> This JIRA proposes to add two new CP hooks to just before and after the
> point-of-no-return (PONR) in the split transaction: {{preSplitPONR}} and
> {{postSplitPONR}}. Perhaps the naming can be improved. This will address the
> above use case but additionally support overloading of the split transaction
> itself. We also need to address observer notification if the split
> transaction fails. This is like HBASE-5827 but scoped to this specific
> consideration only.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Closed] (HBASE-6680) Allow configurable/multiple split log threads per RS.
[ https://issues.apache.org/jira/browse/HBASE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6680. -- > Allow configurable/multiple split log threads per RS. > - > > Key: HBASE-6680 > URL: https://issues.apache.org/jira/browse/HBASE-6680 > Project: HBase > Issue Type: Improvement >Reporter: Amitanand Aiyer >Priority: Minor > Attachments: HBASE-6680-94.patch > > > For rack failure case, it seems like there are multiple batches of split logs > to be processed. Allow a configurable number of split log worker per RS, so > that we split more logs quickly. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6681) Propagate RegionOverloadedException to the Thrift client
[ https://issues.apache.org/jira/browse/HBASE-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6681. -- > Propagate RegionOverloadedException to the Thrift client > > > Key: HBASE-6681 > URL: https://issues.apache.org/jira/browse/HBASE-6681 > Project: HBase > Issue Type: New Feature >Reporter: Mikhail Gryzykhin >Priority: Minor > > To correctly propagate RegionOverloadedException (added in HBASE-6423) to the > Thrift client we need to make it a Thrift exception. One way to do that is to > inherit from IOError and add the desired client-side delay in milliseconds to > IOError. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-6681) Propagate RegionOverloadedException to the Thrift client
[ https://issues.apache.org/jira/browse/HBASE-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-6681. Resolution: Incomplete > Propagate RegionOverloadedException to the Thrift client > > > Key: HBASE-6681 > URL: https://issues.apache.org/jira/browse/HBASE-6681 > Project: HBase > Issue Type: New Feature >Reporter: Mikhail Gryzykhin >Priority: Minor > > To correctly propagate RegionOverloadedException (added in HBASE-6423) to the > Thrift client we need to make it a Thrift exception. One way to do that is to > inherit from IOError and add the desired client-side delay in milliseconds to > IOError. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6708) TEST org.apache.hadoop.hbase.replication.TestReplication.queueFailover flaps
[ https://issues.apache.org/jira/browse/HBASE-6708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6708. -- > TEST org.apache.hadoop.hbase.replication.TestReplication.queueFailover flaps > > > Key: HBASE-6708 > URL: https://issues.apache.org/jira/browse/HBASE-6708 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sameer Vaishampayan >Priority: Major > > https://builds.apache.org/job/HBase-TRUNK/3292/testReport/junit/org.apache.hadoop.hbase.replication/TestReplication/queueFailover/ > Error Message > test timed out after 30 milliseconds > Stacktrace > java.lang.Exception: test timed out after 30 milliseconds > at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1028) > at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:826) > at org.apache.hadoop.hbase.client.HTable.put(HTable.java:801) > at > org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1042) > at > org.apache.hadoop.hbase.replication.TestReplication.queueFailover(TestReplication.java:734) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-6680) Allow configurable/multiple split log threads per RS.
[ https://issues.apache.org/jira/browse/HBASE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-6680. Assignee: (was: Amitanand Aiyer) Resolution: Abandoned > Allow configurable/multiple split log threads per RS. > - > > Key: HBASE-6680 > URL: https://issues.apache.org/jira/browse/HBASE-6680 > Project: HBase > Issue Type: Improvement >Reporter: Amitanand Aiyer >Priority: Minor > Attachments: HBASE-6680-94.patch > > > For rack failure case, it seems like there are multiple batches of split logs > to be processed. Allow a configurable number of split log worker per RS, so > that we split more logs quickly. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6675) TestTableInputFormatScan takes too much time: 652.393s
[ https://issues.apache.org/jira/browse/HBASE-6675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6675. -- > TestTableInputFormatScan takes too much time: 652.393s > -- > > Key: HBASE-6675 > URL: https://issues.apache.org/jira/browse/HBASE-6675 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Priority: Minor > > On trunk, as of today: > Running org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 652.393 sec > Target beeing less than 3 minutes. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6674) Check behavior of current surefire trunk on Hadoop QA
[ https://issues.apache.org/jira/browse/HBASE-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6674. -- > Check behavior of current surefire trunk on Hadoop QA > - > > Key: HBASE-6674 > URL: https://issues.apache.org/jira/browse/HBASE-6674 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon >Priority: Trivial > Attachments: 5processes.patch, 5processes.patch, 5processes.patch, > 6674.patch, 6674.v2.patch, 6674.v2.patch, 6674.v2.patch, 6674.v2.patch > > > Not to be committed. > Surefire 2.13 is in progress. Let's check that it works for us before it's > released. Locally it's acceptable. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6667) TestCatalogJanitor occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-6667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell closed HBASE-6667.
--
Assignee: (was: Jesse Yates)
> TestCatalogJanitor occasionally fails
> -
>
> Key: HBASE-6667
> URL: https://issues.apache.org/jira/browse/HBASE-6667
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Major
> Attachments: java_6667-v0.txt, testCatalogJanitor-output.txt
>
>
> Here is the OS:
> Linux sea0 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:02:55 UTC 2011
> x86_64 x86_64 x86_64 GNU/Linux
> {code}
> testArchiveOldRegion(org.apache.hadoop.hbase.master.TestCatalogJanitor) Time
> elapsed: 0.007 sec <<< FAILURE!
> java.lang.AssertionError: Not the same number of current files
> Expected (2): Gotten (0):
> Not Found:
> _store0
> _store1
> Extra:
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertNull(Assert.java:551)
> at
> org.apache.hadoop.hbase.util.HFileArchiveTestingUtil.assertArchiveEqualToOriginal(HFileArchiveTestingUtil.java:132)
> at
> org.apache.hadoop.hbase.util.HFileArchiveTestingUtil.assertArchiveEqualToOriginal(HFileArchiveTestingUtil.java:95)
> at
> org.apache.hadoop.hbase.master.TestCatalogJanitor.testArchiveOldRegion(TestCatalogJanitor.java:623)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Closed] (HBASE-6664) Filter compound comparators nor comparator subclasses are compared properly by test code
[ https://issues.apache.org/jira/browse/HBASE-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6664. -- > Filter compound comparators nor comparator subclasses are compared properly > by test code > > > Key: HBASE-6664 > URL: https://issues.apache.org/jira/browse/HBASE-6664 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Michael Stack >Priority: Major > Labels: delete > > Gregory over in https://reviews.apache.org/r/6670/ explains the issue. See > the comment here at > https://reviews.apache.org/r/6670/diff/1/?file=142078#file142078line88 Its > about when tests call areSerializedFieldsEqual to figure if objects are > equal. When its comparators, and the comparator has been subclassed or is > made of multiple sub comparators, then we'll not compare right. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6672) Split reads/writes from HBaseClient into smaller chunks, to avoid allocating large directMemoryBuffers
[ https://issues.apache.org/jira/browse/HBASE-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6672. -- > Split reads/writes from HBaseClient into smaller chunks, to avoid allocating > large directMemoryBuffers > -- > > Key: HBASE-6672 > URL: https://issues.apache.org/jira/browse/HBASE-6672 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.89-fb >Reporter: Amitanand Aiyer >Priority: Minor > > HBaseServer seems to already have this fix. HBaseClient does not. Seen some > issue with reading a large KV. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6666) Allow RegionObserver to access other regions on the same server.
[ https://issues.apache.org/jira/browse/HBASE-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-. -- > Allow RegionObserver to access other regions on the same server. > > > Key: HBASE- > URL: https://issues.apache.org/jira/browse/HBASE- > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Priority: Major > > This is an idea I had some time back. It would be nice if a RegionObserver > (or Endpoint) could get access to all other regions on the same RegionServer > to efficiently make updates to those regions as well (instead of going > through the standard HTable path). > Together with a smart region placement strategy this can lead to much better > performance for some coprocessor tasks. > Maybe it could be abstracted in a special HTable implementation. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6668) disable in shell may make confused to user
[ https://issues.apache.org/jira/browse/HBASE-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6668. -- Assignee: (was: Zhou wenjian) > disable in shell may make confused to user > --- > > Key: HBASE-6668 > URL: https://issues.apache.org/jira/browse/HBASE-6668 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 0.94.1 >Reporter: Zhou wenjian >Priority: Major > > hbase(main):002:0> disable 'logTable' > 0 row(s) in 2.0910 seconds > hbase(main):003:0> disable 'logTable' > 0 row(s) in 0.0260 seconds > and we can found table are disabled in log when disable first appears > but when i disable it again the client just return seemed to be sucessful and > I can not find any log described it in the log. > look into the admin.rb, find below > > #-- > # Disables a table > def disable(table_name) > tableExists(table_name) > return if disabled?(table_name) > @admin.disableTable(table_name) > end > that would confuse us when we found it disabled already but returns nothing -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6670) Untangle mixture of protobuf and Writable reference / usage
[ https://issues.apache.org/jira/browse/HBASE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6670. -- > Untangle mixture of protobuf and Writable reference / usage > --- > > Key: HBASE-6670 > URL: https://issues.apache.org/jira/browse/HBASE-6670 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Priority: Critical > > Currently HbaseObjectWritable uses ProtobufUtil to perform serialization of > Scan objects, ProtobufUtil.toParameter() calls > HbaseObjectWritable.writeObject(). > We should untangle such mixture and ultimately remove HbaseObjectWritable -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6661) Mark single parameter HTable constructors as deprecated in 0.90 and 0.92
[ https://issues.apache.org/jira/browse/HBASE-6661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6661. -- > Mark single parameter HTable constructors as deprecated in 0.90 and 0.92 > > > Key: HBASE-6661 > URL: https://issues.apache.org/jira/browse/HBASE-6661 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Daniel Cryans >Assignee: Ted Yu >Priority: Critical > Fix For: 0.90.7, 0.92.2 > > Attachments: 6661.92 > > > HBASE-4746 removed 2 constructors from HTable for 0.94.0 but we never marked > them as deprecated. It's not too late to fix it in 0.90 and 0.92 point > releases. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6660) Meta assignment to a region server caused continuous NPE loop during postOpenDeployTasks
[
https://issues.apache.org/jira/browse/HBASE-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell closed HBASE-6660.
--
> Meta assignment to a region server caused continuous NPE loop during
> postOpenDeployTasks
>
>
> Key: HBASE-6660
> URL: https://issues.apache.org/jira/browse/HBASE-6660
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.2
> Environment: CentOS release 5.7
>Reporter: Shrijeet Paliwal
>Priority: Major
>
> Recently I upgraded three data centers to our own checkout of 0.92.2, last
> commit :
> {noformat}
> commit 5accb6a1be4776630126ac21d07adb652b74df95
> Author: Zhihong Yu
> Date: Mon Aug 20 18:19:45 2012 +
> HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted
> before parent entries, shouldn't compare HRegionInfo's (Enis)
> {noformat}
> Two upgrades went fine, upgrade to one data center failed. Failed in the
> sense that ROOT and META assignment took forever. Panic struck I restarted
> master and all region servers. I may have deleted zookeeper node
> /hbase/root-region-server as well, dont ask me why :-(
> After this I managed to get ROOT assigned. But META assignment got stuck
> again.
> The log is here :
> https://raw.github.com/gist/3455435/adebd118b47aa3d715201010aa09e5eb8930033c/npe_rs_0.92.2.log
> Notice how region server was stuck in a loop of NPE (grep
> processBatchCallback). There is one more NPE related to zookeeper
> constructor.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (HBASE-6673) Clear up the invalid ResultScanner in the ThriftServerRunner
[ https://issues.apache.org/jira/browse/HBASE-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-6673. Resolution: Invalid > Clear up the invalid ResultScanner in the ThriftServerRunner > > > Key: HBASE-6673 > URL: https://issues.apache.org/jira/browse/HBASE-6673 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Priority: Major > > Clear up the invalid ResultScanner in the ThriftServerRunner -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-6676) Add build information to README.txt
[ https://issues.apache.org/jira/browse/HBASE-6676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-6676. Resolution: Won't Fix > Add build information to README.txt > --- > > Key: HBASE-6676 > URL: https://issues.apache.org/jira/browse/HBASE-6676 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Jesse Yates >Priority: Major > Labels: documentation, maven > > It would be really nice if we include build information about the current > build in the README.txt. Pulling up older versions of HBase means guessing at > the right build phase that needs to be run to build an install, since maven > can be like a crystal ball at times. To ensure DRY maybe we can roll this > into a maven build task? -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6657) Merge Filter and FilterBase
[ https://issues.apache.org/jira/browse/HBASE-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6657. -- Assignee: (was: Gregory Chanan) > Merge Filter and FilterBase > --- > > Key: HBASE-6657 > URL: https://issues.apache.org/jira/browse/HBASE-6657 > Project: HBase > Issue Type: Bug > Components: Filters >Reporter: Gregory Chanan >Priority: Minor > > After HBASE-6477, Filter is an abstract class, as is FilterBase. It probably > doesn't make much sense to keep both. > See Review Request for more info: > https://reviews.apache.org/r/6670/ -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6673) Clear up the invalid ResultScanner in the ThriftServerRunner
[ https://issues.apache.org/jira/browse/HBASE-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6673. -- > Clear up the invalid ResultScanner in the ThriftServerRunner > > > Key: HBASE-6673 > URL: https://issues.apache.org/jira/browse/HBASE-6673 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Priority: Major > > Clear up the invalid ResultScanner in the ThriftServerRunner -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6676) Add build information to README.txt
[ https://issues.apache.org/jira/browse/HBASE-6676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6676. -- > Add build information to README.txt > --- > > Key: HBASE-6676 > URL: https://issues.apache.org/jira/browse/HBASE-6676 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Jesse Yates >Priority: Major > Labels: documentation, maven > > It would be really nice if we include build information about the current > build in the README.txt. Pulling up older versions of HBase means guessing at > the right build phase that needs to be run to build an install, since maven > can be like a crystal ball at times. To ensure DRY maybe we can roll this > into a maven build task? -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6656) Cannot call a Coprocessor Endpoint from a RegionObserver
[
https://issues.apache.org/jira/browse/HBASE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell closed HBASE-6656.
--
> Cannot call a Coprocessor Endpoint from a RegionObserver
>
>
> Key: HBASE-6656
> URL: https://issues.apache.org/jira/browse/HBASE-6656
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 0.92.1
> Environment: CentOS5
>Reporter: Mauricio Morales
>Priority: Major
>
> I'm trying to call a Coprocessor Endpoint from within the preGet handler of a
> RegionObserver, and it's throwing Class Loader issues.
> The exact same Coprocessor Endpoint works perfectly from remote Java client,
> however, fails from within the same Region Server.
> For our particular test environment, only 1 Region Server is available, so I
> guess it's a "local" call that fails, and perhaps a remote RegionServer
> wouldn't fail, but that doesn't justify the issue :).
> The Code within the RegionObserver is roughly (way reduced) as follows:
> ---
> @Override
> public void preGet(ObserverContext e, Get
> get, List results)
> throws IOException {
> Map> results;
> // scan: for all regions
> try {
>
> Batch.Call> batchCall = new
> Batch.Call>() {
> public Set
> call(PlatformStatsIndexEndpointProtocol instance) throws IOException{
> return instance.getKeyTokenByPrefix(index,
> match, additionalMatches);
> }
> };
> results =
> indexTable.coprocessorExec(PlatformStatsIndexEndpointProtocol.class, null,
> null, batchCall);
> } catch (Throwable e1) {
> e1.printStackTrace();
> throw new IOException(e1);
> }
>
> Set finalResultSet = new HashSet();
> for (Map.Entry> e : results.entrySet()) {
> finalResultSet.addAll(e.getValue());
> }
> }
> ---
> The Code for the Coprocessor Endpoint is irrelevant, as it never gets
> executed.
> This is the Exception I get on the Client side (Server side logged exception
> below).
> ---
> Thu Aug 23 17:37:45 CST 2012,
> org.apache.hadoop.hbase.client.HTable$5@26659db7,java.io.IOException:
> java.io.IOException: java.io.IOException: java.lang.IllegalArgumentException:
> interface com.company.hbase.platformstats.PlatformStatsIndexEndpointProtocol
> is not visible from class loader
> at
> com.company.hbase.platformstats.IndexQueryRegionObserver.preGet(IndexQueryRegionObserver.java:100)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:553)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3737)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3639)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
> at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
> Caused by: java.io.IOException: java.lang.IllegalArgumentException: interface
> com.company.hbase.platformstats.PlatformStatsIndexEndpointProtocol is not
> visible from class loader
> at
> com.company.hbase.platformstats.PlatformStatsIndexer.getKeyTokens(PlatformStatsIndexer.java:390)
> at
> com.company.hbase.platformstats.PlatformStatsIndexer.getKeyTokensBySubstring(PlatformStatsIndexer.java:348)
> at
> com.company.hbase.platformstats.IndexQueryRegionObserver.searchTokens(IndexQueryRegionObserver.java:148)
> at
> com.company.hbase.platformstats.IndexQueryRegionObserver.preGet(IndexQueryRegionObserver.java:97)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: interface
> com.company.hbase.platformstats.PlatformStatsIndexEndpointProtocol is not
> visible from class loader
> at java.lang.reflect.Proxy.getProxyClass(Proxy.java:353)
> at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:581)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$4.call(HConnectionManager.java:1451)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
>
[jira] [Closed] (HBASE-6653) Minor type and Incorrect link at WIKI index page
[ https://issues.apache.org/jira/browse/HBASE-6653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6653. -- > Minor type and Incorrect link at WIKI index page > > > Key: HBASE-6653 > URL: https://issues.apache.org/jira/browse/HBASE-6653 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.95.2 >Reporter: Liang Xie >Priority: Trivial > Labels: documentation > > At "http://wiki.apache.org/hadoop/Hbase;, > 1) "HBase dist-lists" should be "HBase mail-lists" > 2) the link for "Rolling Restart of HBase" should be changed from > "http://hbase.apache.org/book.html#decommission; to > "http://hbase.apache.org/book.html#rolling; -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6652) [replication]replicationQueueSizeCapacity and replicationQueueNbCapacity default value is too big, Slave regionserver maybe outmemory after master start replication
[ https://issues.apache.org/jira/browse/HBASE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6652. -- > [replication]replicationQueueSizeCapacity and replicationQueueNbCapacity > default value is too big, Slave regionserver maybe outmemory after master > start replication > > > Key: HBASE-6652 > URL: https://issues.apache.org/jira/browse/HBASE-6652 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.94.1 >Reporter: terry zhang >Priority: Major > > now our replication replicationQueueSizeCapacity is set to 64M and > replicationQueueNbCapacity is set to 25000. So when a master cluster with > many regionserver replicate to a small cluster 。 Slave rpc queue will full > and out of memory . > java.util.concurrent.ExecutionException: java.io.IOException: Call queue is > full, is ipc.server.max.callqueue.size too small? > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) > at java.util.concurrent.FutureTask.get(FutureTask.java:83) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java: > 1524) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1376) > at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:700) > at > org.apache.hadoop.hbase.client.HTablePool$PooledHTable.batch(HTablePool.java:361) > at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:172) > at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(ReplicationSink.java:129) > at > org.apache.hadoop.hbase.replication.regionserver.Replication.replicateLogEntries(Replication.java:139) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.replicateLogEntries(HRegionServer.java:4018) > at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:361) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1414) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6650) Import .META. table exported from 0.94
[ https://issues.apache.org/jira/browse/HBASE-6650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6650. -- > Import .META. table exported from 0.94 > -- > > Key: HBASE-6650 > URL: https://issues.apache.org/jira/browse/HBASE-6650 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Priority: Major > > HBASE-6052 converts .META. and ROOT table content to protobuf. > This JIRA allows .META. table exported from 0.94 (see HBASE-3271) to be > imported into live cluster. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-6645) Quarantine invalid reference hfiles.
[ https://issues.apache.org/jira/browse/HBASE-6645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-6645. Resolution: Implemented > Quarantine invalid reference hfiles. > > > Key: HBASE-6645 > URL: https://issues.apache.org/jira/browse/HBASE-6645 > Project: HBase > Issue Type: New Feature >Reporter: Jonathan Hsieh >Priority: Major > > Similar to the quarantining feature introduced to hbck in HBASE-6586, we've > encountered cases of bad reference files. We should add a feature to > quarantine reference files if their parents are not present or valid. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6636) [0.92 UNIT TESTS] TestMasterZKSessionRecovery.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry failed in 0.92 #513 and #496
[ https://issues.apache.org/jira/browse/HBASE-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6636. -- Assignee: (was: Michael Stack) > [0.92 UNIT TESTS] > TestMasterZKSessionRecovery.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry > failed in 0.92 #513 and #496 > --- > > Key: HBASE-6636 > URL: https://issues.apache.org/jira/browse/HBASE-6636 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.1 >Reporter: Michael Stack >Priority: Critical > Fix For: 0.92.3 > > > The test class TestMasterZKSessionRecovery has been removed in trunk. Its > master tests were moved elsewhere or removed because useless (See nkeywal > reasoning over in HBASE-5572 "KeeperException.SessionExpiredException > management could be improved in Master"; it was actually removed by > HBASE-5549 "Master can fail if ZooKeeper session expires"). > TestMasterZKSessionRecovery in 0.92 and 0.94 has an extra test that was not > in trunk, the sporadically failing > testRegionAssignmentAfterMasterRecoveryDueToZKExpiry. This was added by > "HBASE-6046 > Master retry on ZK session expiry causes inconsistent region assignments". -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6651) Improve thread safety of HTablePool
[ https://issues.apache.org/jira/browse/HBASE-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6651. -- Assignee: (was: Hiroshi Ikeda) > Improve thread safety of HTablePool > --- > > Key: HBASE-6651 > URL: https://issues.apache.org/jira/browse/HBASE-6651 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.1 >Reporter: Hiroshi Ikeda >Priority: Major > Attachments: HBASE-6651-V10.patch, HBASE-6651-V11.patch, > HBASE-6651-V12.patch, HBASE-6651-V13.patch, HBASE-6651-V14-0.96.patch, > HBASE-6651-V14-0.98.patch, HBASE-6651-V14-trunk.patch, > HBASE-6651-V15-0.96.patch, HBASE-6651-V15-0.98.patch, > HBASE-6651-V15-trunk.patch, HBASE-6651-V16-0.96.patch, > HBASE-6651-V16-0.98.patch, HBASE-6651-V16-trunk.patch, HBASE-6651-V2.patch, > HBASE-6651-V3.patch, HBASE-6651-V4.patch, HBASE-6651-V5.patch, > HBASE-6651-V6.patch, HBASE-6651-V7.patch, HBASE-6651-V8.patch, > HBASE-6651-V9.patch, HBASE-6651.patch, sample.zip, sample.zip, > sharedmap_for_hbaseclient.zip > > > There are some operations in HTablePool accessing PoolMap in multiple places > without any explicit synchronization. > For example HTablePool.closeTablePool() calls PoolMap.values(), and calls > PoolMap.remove(). If other threads add new instances to the pool in the > middle of the calls, the newly added instances might be dropped. > (HTablePool.closeTablePool() also has another problem that calling it by > multiple threads causes accessing HTable by multiple threads.) > Moreover, PoolMap is not thread safe for the same reason. > For example PoolMap.put() calles ConcurrentMap.get() and calles > ConcurrentMap.put(). If other threads add a new instance to the concurent map > in the middle of the calls, the new instance might be dropped. > And also implementations of Pool have the same problems. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6640) [0.89-fb] Allow multiple regions to be opened simultaneously
[ https://issues.apache.org/jira/browse/HBASE-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6640. -- > [0.89-fb] Allow multiple regions to be opened simultaneously > > > Key: HBASE-6640 > URL: https://issues.apache.org/jira/browse/HBASE-6640 > Project: HBase > Issue Type: Improvement >Reporter: Amitanand Aiyer >Assignee: Amitanand Aiyer >Priority: Critical > Fix For: 0.89-fb > > Attachments: > 0001-HBASE-6640-0.89-fb-Allow-multiple-regions-to-be-open.patch > > > Allow regions to be opened in parallel. This should reduce the time it takes > to replay and reopen regions in case of a unclean restart. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6646) Upgrade to 0.96 section in the book
[ https://issues.apache.org/jira/browse/HBASE-6646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6646. -- > Upgrade to 0.96 section in the book > --- > > Key: HBASE-6646 > URL: https://issues.apache.org/jira/browse/HBASE-6646 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.95.2 >Reporter: Enis Soztutar >Priority: Blocker > > We should have an upgrade section in the book for 0.96. Raising this as > blocker. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-6652) [replication]replicationQueueSizeCapacity and replicationQueueNbCapacity default value is too big, Slave regionserver maybe outmemory after master start replication
[ https://issues.apache.org/jira/browse/HBASE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-6652. Assignee: (was: terry zhang) Resolution: Incomplete > [replication]replicationQueueSizeCapacity and replicationQueueNbCapacity > default value is too big, Slave regionserver maybe outmemory after master > start replication > > > Key: HBASE-6652 > URL: https://issues.apache.org/jira/browse/HBASE-6652 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.94.1 >Reporter: terry zhang >Priority: Major > > now our replication replicationQueueSizeCapacity is set to 64M and > replicationQueueNbCapacity is set to 25000. So when a master cluster with > many regionserver replicate to a small cluster 。 Slave rpc queue will full > and out of memory . > java.util.concurrent.ExecutionException: java.io.IOException: Call queue is > full, is ipc.server.max.callqueue.size too small? > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) > at java.util.concurrent.FutureTask.get(FutureTask.java:83) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java: > 1524) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1376) > at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:700) > at > org.apache.hadoop.hbase.client.HTablePool$PooledHTable.batch(HTablePool.java:361) > at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:172) > at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(ReplicationSink.java:129) > at > org.apache.hadoop.hbase.replication.regionserver.Replication.replicateLogEntries(Replication.java:139) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.replicateLogEntries(HRegionServer.java:4018) > at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:361) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1414) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6645) Quarantine invalid reference hfiles.
[ https://issues.apache.org/jira/browse/HBASE-6645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6645. -- > Quarantine invalid reference hfiles. > > > Key: HBASE-6645 > URL: https://issues.apache.org/jira/browse/HBASE-6645 > Project: HBase > Issue Type: New Feature >Reporter: Jonathan Hsieh >Priority: Major > > Similar to the quarantining feature introduced to hbck in HBASE-6586, we've > encountered cases of bad reference files. We should add a feature to > quarantine reference files if their parents are not present or valid. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6559) HDFS size does not decrease when deleting a column family
[ https://issues.apache.org/jira/browse/HBASE-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6559. -- > HDFS size does not decrease when deleting a column family > -- > > Key: HBASE-6559 > URL: https://issues.apache.org/jira/browse/HBASE-6559 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.94.1 >Reporter: J Mohamed Zahoor >Priority: Minor > > When deleting a column family of a table and doing major compaction, the disk > space occupied by the column family is still the same. A "fs -dus > /hbase///" shows a large value > suggesting that the column family is not deleted in the file system. The > reported version was 0.94.1 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6639) Class.newInstance() can throw any checked exceptions and must be encapsulated with catching Exception
[ https://issues.apache.org/jira/browse/HBASE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6639. -- > Class.newInstance() can throw any checked exceptions and must be encapsulated > with catching Exception > - > > Key: HBASE-6639 > URL: https://issues.apache.org/jira/browse/HBASE-6639 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.1 >Reporter: Hiroshi Ikeda >Priority: Minor > Attachments: HBASE-6639-V2.patch, HBASE-6639-V3.patch, > HBASE-6639.patch > > > There are some logics to call Class.newInstance() without catching Exception, > for example, in the method CoprocessorHost.loadInstance(). > Class.newInstance() is declared to throw InstantiationException and > IllegalAccessException but indeed the method can throw any checked exceptions > without declaration. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6635) Refactor HFile version selection and exception handling.
[ https://issues.apache.org/jira/browse/HBASE-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6635. -- > Refactor HFile version selection and exception handling. > > > Key: HBASE-6635 > URL: https://issues.apache.org/jira/browse/HBASE-6635 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Priority: Major > Attachments: hfile.png > > > Trunk and 0.94's HFile code has some fairly convoluted code for bypassing > checksums and has mixed usage of runtime and io exceptions when error > conditions arise. This jira would clean up the code to have better > encapsulation and be more explicit about what kinds of exceptions are thrown > and what they mean. (This was partially spurred by comments in reviews of > HBASE-6586). -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6558) Avoid ssh to localhost in single node setup.
[ https://issues.apache.org/jira/browse/HBASE-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6558. -- > Avoid ssh to localhost in single node setup. > - > > Key: HBASE-6558 > URL: https://issues.apache.org/jira/browse/HBASE-6558 > Project: HBase > Issue Type: Improvement > Components: scripts > Environment: mac osx mountain lion, hbase 89-fb >Reporter: Ramkumar Vadali >Priority: Trivial > Original Estimate: 24h > Remaining Estimate: 24h > > The use of ssh in scripts like zookeepers.sh and regionservers.sh for a > single node setup is not necessary. We can execute the command directly. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6563) s.isMajorCompaction() throws npe will cause current major Compaction checking abort
[ https://issues.apache.org/jira/browse/HBASE-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6563. -- Assignee: (was: Zhou wenjian) > s.isMajorCompaction() throws npe will cause current major Compaction checking > abort > --- > > Key: HBASE-6563 > URL: https://issues.apache.org/jira/browse/HBASE-6563 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Zhou wenjian >Priority: Major > Attachments: HBASE-6563-trunk-v2.patch, HBASE-6563-trunk-v3.patch, > HBASE-6563-trunk.patch > > > 2012-05-05 00:49:43,265 ERROR > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: > Caught exception > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:938) > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:917) > at > org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3250) > at > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1222) > at org.apache.hadoop.hbase.Chore.run(Chore.java:66) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6562) Fake KVs are sometimes passed to filters
[ https://issues.apache.org/jira/browse/HBASE-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6562. -- > Fake KVs are sometimes passed to filters > > > Key: HBASE-6562 > URL: https://issues.apache.org/jira/browse/HBASE-6562 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Priority: Minor > Attachments: 6562-0.94-v1.txt, 6562-0.96-v1.txt, 6562-v2.txt, > 6562-v3.txt, 6562-v4.txt, 6562-v5.txt, 6562.txt, minimalTest.java > > > In internal tests at Salesforce we found that fake row keys sometimes are > passed to filters (Filter.filterRowKey(...) specifically). > The KVs are eventually filtered by the StoreScanner/ScanQueryMatcher, but the > row key is passed to filterRowKey in RegionScannImpl *before* that happens. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6560) Deleting column family does not reclaim space in HDFS
[ https://issues.apache.org/jira/browse/HBASE-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6560. -- > Deleting column family does not reclaim space in HDFS > - > > Key: HBASE-6560 > URL: https://issues.apache.org/jira/browse/HBASE-6560 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.94.1 >Reporter: J Mohamed Zahoor >Priority: Minor > > When deleting a column family of a table and doing major compaction, the disk > space occupied by the column family is still the same. A "fs -dus > /hbase///" shows a large value > suggesting that the column family is not deleted in the file system. The > reported version was 0.94.1 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-6635) Refactor HFile version selection and exception handling.
[ https://issues.apache.org/jira/browse/HBASE-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-6635. Resolution: Won't Fix > Refactor HFile version selection and exception handling. > > > Key: HBASE-6635 > URL: https://issues.apache.org/jira/browse/HBASE-6635 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Priority: Major > Attachments: hfile.png > > > Trunk and 0.94's HFile code has some fairly convoluted code for bypassing > checksums and has mixed usage of runtime and io exceptions when error > conditions arise. This jira would clean up the code to have better > encapsulation and be more explicit about what kinds of exceptions are thrown > and what they mean. (This was partially spurred by comments in reviews of > HBASE-6586). -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6625) If we have hundreds of thousands of regions getChildren will encouter zk exception
[ https://issues.apache.org/jira/browse/HBASE-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6625. -- Assignee: (was: Zhou wenjian) > If we have hundreds of thousands of regions getChildren will encouter zk > exception > --- > > Key: HBASE-6625 > URL: https://issues.apache.org/jira/browse/HBASE-6625 > Project: HBase > Issue Type: Bug >Reporter: Zhou wenjian >Priority: Major > > 2012-05-13 19:37:37,528 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager$ExistsUnassignedAsyncCallback: > rs=CreateNewTableWith10Regions,\x05\xB3\x06 > g\xE8r\xBB]\x09\xCF,1336724029944.079cb2f8a375e66fa089291b82f2a03f. > state=OFFLINE, ts=1336909053108 > 2012-05-13 19:37:37,528 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager$CreateUnassignedAsyncCallback: > rs=CreateNewTableWith10Regions,\x08s\x84\x8 > 8$7\xB1\xC4\xFCg,1336724030660.76c07780231942231013c7feb5e5eb14. > state=OFFLINE, ts=1336909055089, server=dw76.kgb.sqa.cm4,60020,1336908983944 > 2012-05-13 19:37:37,528 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager$CreateUnassignedAsyncCallback: > rs=CreateNewTableWith10Regions,\x08s\x89\xC > B\x9B\xF0\xE4\xCA\x97\xB0,1336724030660.fa38b9d8367387a64a327087cb43b3e0. > state=OFFLINE, ts=1336909055089, server=dw76.kgb.sqa.cm4,60020,1336908983944 > 2012-05-13 19:37:37,528 INFO > org.apache.hadoop.hbase.master.AssignmentManager: > dw76.kgb.sqa.cm4,60020,1336908983944 unassigned znodes=58464 of total=120002 > 2012-05-13 19:37:37,758 WARN org.apache.zookeeper.ClientCnxn: Session > 0x13745fc2c8d0001 for server dw51.kgb.sqa.cm4/10.232.98.51:2180, unexpected > error, clos > ing socket connection and attempting reconnect > java.io.IOException: Packet len4320092 is out of range! > at > org.apache.zookeeper.ClientCnxn$SendThread.readLength(ClientCnxn.java:710) > at > org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:869) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1130) > 2012-05-13 19:37:37,860 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil: > master:6-0x13745fc2c8d0001 Unable to list children of znode > /hbase-new4/unassigned > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss for /hbase-new4/unassigned > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:90) > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:42) > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243) > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchForNewChildren(ZKUtil.java:302) > > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndGetNewChildren(ZKUtil.java:413) > > at > org.apache.hadoop.hbase.master.AssignmentManager.nodeChildrenChanged(AssignmentManager.java:759) > > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:314) > > at > org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) > at > org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) > 2012-05-13 19:37:37,861 ERROR > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher: > master:6-0x13745fc2c8d0001 Received unexpected KeeperException, re-thro > wing exception > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss for /hbase-new4/unassigned > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:90) > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:42) > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243) > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchForNewChildren(ZKUtil.java:302) > > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndGetNewChildren(ZKUtil.java:413) > > at > org.apache.hadoop.hbase.master.AssignmentManager.nodeChildrenChanged(AssignmentManager.java:759) > > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:314) > > at > org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) > at > org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) > 2012-05-13 19:37:37,861 FATAL org.apache.hadoop.hbase.master.HMaster: > Unexpected ZK exception reading unassigned children > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss for /hbase-new4/unassigned > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:90) > at >
[jira] [Closed] (HBASE-6572) Tiered HFile storage
[ https://issues.apache.org/jira/browse/HBASE-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6572. -- > Tiered HFile storage > > > Key: HBASE-6572 > URL: https://issues.apache.org/jira/browse/HBASE-6572 > Project: HBase > Issue Type: Brainstorming >Reporter: Andrew Kyle Purtell >Priority: Major > > Consider how we might enable tiered HFile storage. If HDFS has the > capability, we could create certain files on solid state devices where they > might be frequently accessed, especially for random reads; and others (and by > default) on spinning media as before. We could support the move of frequently > read HFiles from spinning media to solid state. We already have CF statistics > for this, would only need to add requisite admin interface; could even > consider an autotiering option. > Dhruba Borthakur did some early work in this area and wrote up his findings: > http://hadoopblog.blogspot.com/2012/05/hadoop-and-solid-state-drives.html . > It is important to note the findings but I suggest most of the > recommendations are out of scope of this JIRA. This JIRA seeks to find an > initial use case that produces a reasonable benefit, and serves as a testbed > for further improvements. If I may paraphrase Dhruba's findings (any > misstatements and errors are mine): First, the DFSClient code paths introduce > significant latency, so the HDFS client (and presumably the DataNode, as the > next bottleneck) will need significant work to knock that down. Need to > investigate optimized (perhaps read-only) DFS clients, server side read and > caching strategies. Second, RegionServers are heavily threaded and this > imposes a lot of monitor contention and context switching cost. Need to > investigate reducing the number of threads in a RegionServer, nonblocking IO > and RPC. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6557) Avoid ssh to localhost in single node setup.
[ https://issues.apache.org/jira/browse/HBASE-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6557. -- > Avoid ssh to localhost in single node setup. > - > > Key: HBASE-6557 > URL: https://issues.apache.org/jira/browse/HBASE-6557 > Project: HBase > Issue Type: Improvement > Components: scripts > Environment: mac osx mountain lion, hbase 89-fb >Reporter: Ramkumar Vadali >Priority: Trivial > > The use of ssh in scripts like zookeepers.sh and regionservers.sh for a > single node setup is not necessary. We can execute the command directly. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6624) [Replication]currentNbOperations should set to 0 after update the shippedOpsRate
[ https://issues.apache.org/jira/browse/HBASE-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6624. -- Assignee: (was: terry zhang) > [Replication]currentNbOperations should set to 0 after update the > shippedOpsRate > > > Key: HBASE-6624 > URL: https://issues.apache.org/jira/browse/HBASE-6624 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.94.0 >Reporter: terry zhang >Priority: Major > Attachments: jira-6624.patch > > > now currentNbOperations will not reset to 0 and increase after replication > start. Now this value is used for calculate shippedOpsRate. if it is not > reset to 0 shippedOpsRate is not correct -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6566) support different block replication for hlog
[ https://issues.apache.org/jira/browse/HBASE-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6566. -- > support different block replication for hlog > > > Key: HBASE-6566 > URL: https://issues.apache.org/jira/browse/HBASE-6566 > Project: HBase > Issue Type: Bug >Reporter: Lijin Bin >Priority: Major > > current, hfile and hlog use the same block replication, some times we can set > different block replication, for example in some cluster that not every > important, we may set the block replication to 2 for hlog to improve > performance. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6629) [0.89-fb] Fast fail client operations if the regionserver is repeatedly unreachable.
[ https://issues.apache.org/jira/browse/HBASE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6629. -- > [0.89-fb] Fast fail client operations if the regionserver is repeatedly > unreachable. > > > Key: HBASE-6629 > URL: https://issues.apache.org/jira/browse/HBASE-6629 > Project: HBase > Issue Type: New Feature >Reporter: Amitanand Aiyer >Assignee: Amitanand Aiyer >Priority: Minor > Fix For: 0.89-fb > > > We have seen occassional RSW reboots in the production cluster. > On the client end, the reduction in the operation throughput is a lot more > than the %-age of nodes disconnected. This is because operations to the > disconnected machines are holding up resources that could otherwise be used > for successful operation. > This change enables the client to detect when there are repeated failures to > a regionserver, and fast fail operations so we do not hold up resources. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6627) TestMultiVersions.testGetRowVersions is flaky
[ https://issues.apache.org/jira/browse/HBASE-6627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6627. -- Assignee: (was: Nicolas Liochon) > TestMultiVersions.testGetRowVersions is flaky > - > > Key: HBASE-6627 > URL: https://issues.apache.org/jira/browse/HBASE-6627 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.2 > Environment: hadoop-qa mainly, seems to happen tests in parallel; > difficult to reproduce on a single test. >Reporter: Nicolas Liochon >Priority: Major > Attachments: 6627.v1.patch > > > org.apache.hadoop.hbase.TestMultiVersions.testGetRowVersions > Shutting down > Stacktrace > java.io.IOException: Shutting down > at > org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:229) > at > org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:92) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:688) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:661) > at > org.apache.hadoop.hbase.TestMultiVersions.testGetRowVersions(TestMultiVersions.java:143) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:18) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6628) Add HBASE-6059 to 0.94 branch
[ https://issues.apache.org/jira/browse/HBASE-6628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6628. -- > Add HBASE-6059 to 0.94 branch > - > > Key: HBASE-6628 > URL: https://issues.apache.org/jira/browse/HBASE-6628 > Project: HBase > Issue Type: Task >Reporter: Michael Stack >Priority: Major > > Look at adding HBASE-6059 to 0.94. Its in trunk. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6556) Avoid ssh to localhost in startup scripts
[ https://issues.apache.org/jira/browse/HBASE-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6556. -- > Avoid ssh to localhost in startup scripts > - > > Key: HBASE-6556 > URL: https://issues.apache.org/jira/browse/HBASE-6556 > Project: HBase > Issue Type: Improvement > Components: scripts > Environment: Mac OSX Mountain Lion, HBase 89-fb >Reporter: Ramkumar Vadali >Priority: Trivial > > The use of ssh in scripts like zookeepers.sh and regionservers.sh for a > single node setup is not necessary. We can execute the command directly. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6555) Avoid ssh to localhost in startup scripts
[ https://issues.apache.org/jira/browse/HBASE-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6555. -- > Avoid ssh to localhost in startup scripts > - > > Key: HBASE-6555 > URL: https://issues.apache.org/jira/browse/HBASE-6555 > Project: HBase > Issue Type: Improvement > Components: scripts > Environment: Mac OSX Mountain Lion, HBase 89-fb >Reporter: Ramkumar Vadali >Priority: Trivial > > The use of ssh in scripts like zookeepers.sh and regionservers.sh for a > single node setup is not necessary. We can execute the command directly. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-6555) Avoid ssh to localhost in startup scripts
[ https://issues.apache.org/jira/browse/HBASE-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-6555. Assignee: (was: Sean Busbey) Resolution: Won't Fix > Avoid ssh to localhost in startup scripts > - > > Key: HBASE-6555 > URL: https://issues.apache.org/jira/browse/HBASE-6555 > Project: HBase > Issue Type: Improvement > Components: scripts > Environment: Mac OSX Mountain Lion, HBase 89-fb >Reporter: Ramkumar Vadali >Priority: Trivial > > The use of ssh in scripts like zookeepers.sh and regionservers.sh for a > single node setup is not necessary. We can execute the command directly. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6551) Refactoring ReplicationSink to make it more responsive of cluster health
[ https://issues.apache.org/jira/browse/HBASE-6551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6551. -- Assignee: (was: Himanshu Vashishtha) > Refactoring ReplicationSink to make it more responsive of cluster health > > > Key: HBASE-6551 > URL: https://issues.apache.org/jira/browse/HBASE-6551 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Himanshu Vashishtha >Priority: Major > > ReplicationSink replicates the WALEdits in the local cluster. It uses native > HBase client to insert the mutations. Sometime, it takes a while to process > it (may be due to region splitting, gc pause, etc) and it undergoes the > retrial phase. > It has two repercussions: > a) The regionserver handler which is serving the request (till now, a > priority handler) is blocked for this period. > b) The caller may get timed out and it will retry it anyway, but the handler > serving the ReplicationSink requests is still working. > Refactoring ReplicationSink to have the following features: > a) Making it more configurable (have its own number of retrial limit, > connection timeout, etc) > b) Add a fail fast behavior so that it bails out in case caller is timedout, > or any exception in processing the mutation batch. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6549) Refactoring ReplicationSink to make it more responsive of cluster health
[ https://issues.apache.org/jira/browse/HBASE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6549. -- Assignee: (was: Himanshu Vashishtha) > Refactoring ReplicationSink to make it more responsive of cluster health > > > Key: HBASE-6549 > URL: https://issues.apache.org/jira/browse/HBASE-6549 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Himanshu Vashishtha >Priority: Major > > ReplicationSink replicates the WALEdits in the local cluster. It uses native > HBase client to insert the mutations. Sometime, it takes a while to process > it (may be due to region splitting, gc pause, etc) and it undergoes the > retrial phase. > It has two repercussions: > a) The regionserver handler which is serving the request (till now, a > priority handler) is blocked for this period. > b) The caller may get timed out and it will retry it anyway, but the handler > serving the ReplicationSink requests is still working. > Refactoring ReplicationSink to have the following features: > a) Making it more configurable (have its own number of retrial limit, > connection timeout, etc) > b) Add a fail fast behavior so that it bails out in case caller is timedout, > or any exception in processing the mutation batch. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6548) Refactoring ReplicationSink to make it more responsive of cluster health
[ https://issues.apache.org/jira/browse/HBASE-6548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6548. -- Assignee: (was: Himanshu Vashishtha) > Refactoring ReplicationSink to make it more responsive of cluster health > > > Key: HBASE-6548 > URL: https://issues.apache.org/jira/browse/HBASE-6548 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Himanshu Vashishtha >Priority: Major > > ReplicationSink replicates the WALEdits in the local cluster. It uses native > HBase client to insert the mutations. Sometime, it takes a while to process > it (region splitting, gc pause, etc) and it undergoes the retrial phase. It > has two repercussions: > a) The regionserver handler which is serving the request is blocked for this > period. > b) The caller may get timed out and it will retry it anyway, but the handler > serving the ReplicationSink requests is still working. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6545) Refactoring ReplicationSink to make it more responsive of cluster health
[ https://issues.apache.org/jira/browse/HBASE-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6545. -- Assignee: (was: Himanshu Vashishtha) > Refactoring ReplicationSink to make it more responsive of cluster health > > > Key: HBASE-6545 > URL: https://issues.apache.org/jira/browse/HBASE-6545 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Himanshu Vashishtha >Priority: Major > > ReplicationSink replicates the WALEdits in the local cluster. It uses native > HBase client to insert the mutations. Sometime, it takes a while to process > it (region splitting, gc pause, etc) and it undergoes the retrial phase. It > has two repercussions: > a) The regionserver handler which is serving the request is blocked for this > period. > b) The caller may get timed out and it will retry it anyway, but the handler > serving the ReplicationSink requests is still working. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (HBASE-6546) Refactoring ReplicationSink to make it more responsive of cluster health
[ https://issues.apache.org/jira/browse/HBASE-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell closed HBASE-6546. -- Assignee: (was: Himanshu Vashishtha) > Refactoring ReplicationSink to make it more responsive of cluster health > > > Key: HBASE-6546 > URL: https://issues.apache.org/jira/browse/HBASE-6546 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Himanshu Vashishtha >Priority: Major > > ReplicationSink replicates the WALEdits in the local cluster. It uses native > HBase client to insert the mutations. Sometime, it takes a while to process > it (region splitting, gc pause, etc) and it undergoes the retrial phase. It > has two repercussions: > a) The regionserver handler which is serving the request is blocked for this > period. > b) The caller may get timed out and it will retry it anyway, but the handler > serving the ReplicationSink requests is still working. -- This message was sent by Atlassian Jira (v8.20.7#820007)
