[GitHub] [hbase] virajjasani commented on issue #1074: HBASE-23350 Make compaction files cacheonWrite configurable based on threshold
virajjasani commented on issue #1074: HBASE-23350 Make compaction files cacheonWrite configurable based on threshold URL: https://github.com/apache/hbase/pull/1074#issuecomment-579622879 @abhinabasarkar Please rebase your branch once This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani edited a comment on issue #754: HBASE-22978 : Online slow response log
virajjasani edited a comment on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-579605235 > I think you misunderstood me here... > > "With the patch we would run at RS entrance only and for any slow running query it will go to this ring buffer running at RS. Just that this patch doesn't include storing slow logs in HDFS, which is also a good idea and probably up for a subtask given the priority is storing slow logs in ring buffer first. Storing them in HDFS would be just an option as part of another Jira(sub-task)." > > What I was saying is that your ringbuffer only keeps dfsclient ops. I was wondering if we could put actual requests into a system like this. If they are taking a long time, we could inspect the ringbuffer to see which type of queries are taking a while. It would be coarser grained than the dfs accesses only. Was just wondering if you think we could do this in a follow-on based on work h ere. Oh okk, my bad. Looks good for the follow-up task for sure, unless we are already putting these requests in ring buffer. One question: Does actual request not come to `RpcServer.logResponse()` already? If not, then definitely +1 for follow-up task. I thought all requests already do land in RpcServer. No? Edit: @saintstack you are suggesting to run ringbuffer at RS entrance and consume request soon as it arrives at RS and before triggering DFSClient ops? If so, we need to determine which one is slow for RS to send response to user/HMaster and accordingly log entry in ring buffer right? +1 for the idea. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani edited a comment on issue #754: HBASE-22978 : Online slow response log
virajjasani edited a comment on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-579605235 > I think you misunderstood me here... > > "With the patch we would run at RS entrance only and for any slow running query it will go to this ring buffer running at RS. Just that this patch doesn't include storing slow logs in HDFS, which is also a good idea and probably up for a subtask given the priority is storing slow logs in ring buffer first. Storing them in HDFS would be just an option as part of another Jira(sub-task)." > > What I was saying is that your ringbuffer only keeps dfsclient ops. I was wondering if we could put actual requests into a system like this. If they are taking a long time, we could inspect the ringbuffer to see which type of queries are taking a while. It would be coarser grained than the dfs accesses only. Was just wondering if you think we could do this in a follow-on based on work h ere. Oh okk, my bad. Looks good for the follow-up task for sure, unless we are already putting these requests in ring buffer. One question: Does actual request not come to `RpcServer.logResponse()` already? If not, then definitely +1 for follow-up task. I thought all requests already do land in RpcServer. No? Edit: @saintstack you are suggesting to run ringbuffer at RS entrance and consume request soon as it arrives at RS and before triggering DFSClient ops? If so, we need to determine which one is slow for RS to send response to HBaseClient/HMaster and accordingly log entry in ring buffer right? +1 for the idea. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1074: HBASE-23350 Make compaction files cacheonWrite configurable based on threshold
Apache-HBase commented on issue #1074: HBASE-23350 Make compaction files cacheonWrite configurable based on threshold URL: https://github.com/apache/hbase/pull/1074#issuecomment-579607860 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | patch | 0m 4s | https://github.com/apache/hbase/pull/1074 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/1074 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1074/4/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23752) Fix a couple more test failures from nightly run
[ https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025624#comment-17025624 ] Bharath Vissapragada commented on HBASE-23752: -- [~stack]/[~andrew.purt...@gmail.com] Mind taking a look at this? This is the last (hopefully) patch that fixes all the test failures from the full nightly run. I verified that the two failed tests are flakes, ran them locally. > Fix a couple more test failures from nightly run > > > Key: HBASE-23752 > URL: https://issues.apache.org/jira/browse/HBASE-23752 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.3.0, 1.6.0, HBASE-18095 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > > After merging the patches HBASE-23731 and HBASE-23647, nightly run found the > following issues. > TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR > runs missed it somehow. > TestCustomSaslAuthenticationProvider - This was written pre master registry. > Has to be updated. It was never run as a part of the test suite. > TestSnapshotScannerHDFSAclController: Cluster restart did not reset the > cached connection state > > [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it > accordingly. Will submit a PR shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on issue #754: HBASE-22978 : Online slow response log
virajjasani commented on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-579605235 > I think you misunderstood me here... > > "With the patch we would run at RS entrance only and for any slow running query it will go to this ring buffer running at RS. Just that this patch doesn't include storing slow logs in HDFS, which is also a good idea and probably up for a subtask given the priority is storing slow logs in ring buffer first. Storing them in HDFS would be just an option as part of another Jira(sub-task)." > > What I was saying is that your ringbuffer only keeps dfsclient ops. I was wondering if we could put actual requests into a system like this. If they are taking a long time, we could inspect the ringbuffer to see which type of queries are taking a while. It would be coarser grained than the dfs accesses only. Was just wondering if you think we could do this in a follow-on based on work h ere. Oh okk, my bad. Looks good for the follow-up task for sure, unless we are already putting these requests in ring buffer. One question: Does actual request not come to `RpcServer.logResponse()` already? If not, then definitely +1 for follow-up task. I thought all requests already do land in RpcServer. No? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025621#comment-17025621 ] Bharath Vissapragada commented on HBASE-18095: -- Thanks [~stack]. > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Kyle Purtell >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch > > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on issue #754: HBASE-22978 : Online slow response log
saintstack commented on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-579603450 I think you misunderstood me here... "With the patch we would run at RS entrance only and for any slow running query it will go to this ring buffer running at RS. Just that this patch doesn't include storing slow logs in HDFS, which is also a good idea and probably up for a subtask given the priority is storing slow logs in ring buffer first. Storing them in HDFS would be just an option as part of another Jira(sub-task)." What I was saying is that your ringbuffer only keeps dfsclient ops. I was wondering if we could put actual requests into a system like this. If they are taking a long time, we could inspect the ringbuffer to see which type of queries are taking a while. It would be coarser grained than the dfs accesses only. Was just wondering if you think we could do this in a follow-on based on work h ere. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23279) Switch default block encoding to ROW_INDEX_V1
[ https://issues.apache.org/jira/browse/HBASE-23279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025610#comment-17025610 ] Michael Stack commented on HBASE-23279: --- Seems like a problem that needs some work. We'd need a metric to see when index causes us to overspill our current block. We'd need to keep a running account of the 'tax' each key accumulates on add -- based off the encoder in place -- that we'd factor along w/ Cell size figuring when to cut the current block. > Switch default block encoding to ROW_INDEX_V1 > - > > Key: HBASE-23279 > URL: https://issues.apache.org/jira/browse/HBASE-23279 > Project: HBase > Issue Type: Wish >Affects Versions: 3.0.0, 2.3.0 >Reporter: Lars Hofhansl >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-23279.master.000.patch, > HBASE-23279.master.001.patch, HBASE-23279.master.002.patch, > HBASE-23279.master.003.patch, HBASE-23279.master.004.patch, > HBASE-23279.master.005.patch > > > Currently we set both block encoding and compression to NONE. > ROW_INDEX_V1 has many advantages and (almost) no disadvantages (the hfiles > are slightly larger about 3% or so). I think that would a better default than > NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23760) (2.1) Helper method to configure secure DFS cluster for tests
[
https://issues.apache.org/jira/browse/HBASE-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025605#comment-17025605
]
HBase QA commented on HBASE-23760:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
28s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
15s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} branch-2.1 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
20s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
59s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s{color} | {color:green} hbase-server: The patch generated 0 new + 4
unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} The patch passed checkstyle in hbase-thrift {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch passed checkstyle in hbase-endpoint
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
21m 51s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.7 2.8.5 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}198m
43s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
7s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
4s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} |
[GitHub] [hbase] virajjasani opened a new pull request #1105: HBASE-23749 : TestHFileWriterV3 for all encoders
virajjasani opened a new pull request #1105: HBASE-23749 : TestHFileWriterV3 for all encoders URL: https://github.com/apache/hbase/pull/1105 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23752) Fix a couple more test failures from nightly run
[
https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025603#comment-17025603
]
HBase QA commented on HBASE-23752:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-18095/client-locate-meta-no-zookeeper Compile
Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
4s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
51s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
49s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 54s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}171m 24s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}236m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/1102 |
| JIRA Issue | HBASE-23752 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2fb08e78648f 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1102/out/precommit/personality/provided.sh
|
| git revision |
[GitHub] [hbase] Apache-HBase commented on issue #1102: HBASE-23752: Fix remaining test failures from nightly runs
Apache-HBase commented on issue #1102: HBASE-23752: Fix remaining test failures from nightly runs URL: https://github.com/apache/hbase/pull/1102#issuecomment-579600378 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 4s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | compile | 0m 59s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | checkstyle | 1m 14s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | shadedjars | 5m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +0 :ok: | spotbugs | 4m 51s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 49s | HBASE-18095/client-locate-meta-no-zookeeper passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 43s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | checkstyle | 1m 12s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 54s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | | +1 :green_heart: | findbugs | 5m 3s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 171m 24s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 236m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1102 | | JIRA Issue | HBASE-23752 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2fb08e78648f 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1102/out/precommit/personality/provided.sh | | git revision | HBASE-18095/client-locate-meta-no-zookeeper / acf2a45cb0 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/2/testReport/ | | Max. process+thread count | 4849 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025599#comment-17025599 ] Michael Stack commented on HBASE-18095: --- [~bharathv] Pushed again. I think you need the recent errorprone fixes HBASE-23753 on your branch too. > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Kyle Purtell >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch > > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23753) Update of errorprone generated failures
[ https://issues.apache.org/jira/browse/HBASE-23753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-23753. --- Hadoop Flags: Reviewed Assignee: Michael Stack Resolution: Fixed Pushed to branch-2 and master. Thanks for review [~bharathv] > Update of errorprone generated failures > --- > > Key: HBASE-23753 > URL: https://issues.apache.org/jira/browse/HBASE-23753 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Parent issue updated hbase-thirdparty which updated errorprone. The nightly > compile started failing with 'compile'/errorprone complaints. All look good. > Let me fix. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on issue #1103: HBASE-23753 Update of errorprone generated failures
saintstack commented on issue #1103: HBASE-23753 Update of errorprone generated failures URL: https://github.com/apache/hbase/pull/1103#issuecomment-579568299 Merged on cli w/ @bharathv suggestions. Forgot to add signed-off-by for you... @bharathv ... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1103: HBASE-23753 Update of errorprone generated failures
Apache-HBase commented on issue #1103: HBASE-23753 Update of errorprone generated failures URL: https://github.com/apache/hbase/pull/1103#issuecomment-579562696 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 57s | branch-2 passed | | +1 :green_heart: | compile | 1m 45s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 9s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 20s | branch-2 passed | | +0 :ok: | spotbugs | 4m 59s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 6m 25s | the patch passed | | +1 :green_heart: | compile | 1m 49s | the patch passed | | +1 :green_heart: | javac | 1m 49s | the patch passed | | -1 :x: | checkstyle | 1m 38s | hbase-server: The patch generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 21m 12s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 15s | the patch passed | | +1 :green_heart: | findbugs | 5m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 4m 17s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 156m 46s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 49s | The patch does not generate ASF License warnings. | | | | 241m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1103/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1103 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6a29a7fe5386 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1103/out/precommit/personality/provided.sh | | git revision | branch-2 / 36cdcad43f | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1103/1/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1103/1/testReport/ | | Max. process+thread count | 5290 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1103/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025538#comment-17025538
]
Hudson commented on HBASE-18095:
Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #54 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/54/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/54//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/54//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/54//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/54//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Provide an option for clients to find the server hosting META that does not
> involve the ZooKeeper client
>
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the
> regionserver hosting the meta table region. Site configuration provides the
> client a list of ZK quorum peers and the client uses an embedded ZK client to
> query meta location. Timeouts and retry behavior of this embedded ZK client
> are managed orthogonally to HBase layer settings and in some cases the ZK
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage
> or network partition.
> We should consider new configuration settings that provide a list of
> well-known master and backup master locations, and with this information the
> client can contact any of the master processes directly. Any master in either
> active or passive state will track meta location and respond to requests for
> it with its cached last known location. If this location is stale, the client
> can ask again with a flag set that requests the master refresh its location
> cache and return the up-to-date location. Every client interaction with the
> cluster thus uses only HBase RPC as transport, with appropriate settings
> applied to the connection. The configuration toggle that enables this
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and
> contact the ZK service directly at the beginning of the connection lifecycle.
> This has several benefits. ZK service need not be exposed to clients, and
> their potential abuse, yet no benefit ZK provides the HBase server cluster is
> compromised. Normalizing HBase client and ZK client timeout settings and
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer
> necessary.
> And, from [~ghelmling]: There is an additional complication here for
> token-based authentication. When a delegation token is used for SASL
> authentication, the client uses the cluster ID obtained from Zookeeper to
> select the token identifier to use. So there would also need to be some
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23045) currentPath may be stitched in a loop in replication source code.
[ https://issues.apache.org/jira/browse/HBASE-23045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-23045: - Attachment: HBASE-23045.branch-1.000.patch > currentPath may be stitched in a loop in replication source code. > -- > > Key: HBASE-23045 > URL: https://issues.apache.org/jira/browse/HBASE-23045 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.2.6.1 >Reporter: kangkang.guo >Assignee: kangkang.guo >Priority: Critical > Fix For: 1.2.6.1, 1.6.0 > > Attachments: HBASE-23045.branch-1.000.patch, > HBASE-23045.branch-1.000.patch, HBASE-23045.branch-1.2.0001.patch > > > When the openReader encounters a FileNotFoundException, we may go to all > possible directories to find the current hlog. When found, the path may be > wrong, and it is looped together. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1102: HBASE-23752: Fix remaining test failures from nightly runs
Apache-HBase commented on issue #1102: HBASE-23752: Fix remaining test failures from nightly runs URL: https://github.com/apache/hbase/pull/1102#issuecomment-579552336 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 51s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | compile | 1m 16s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | checkstyle | 1m 31s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | shadedjars | 6m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +0 :ok: | spotbugs | 5m 36s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 33s | HBASE-18095/client-locate-meta-no-zookeeper passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 24s | the patch passed | | +1 :green_heart: | compile | 1m 22s | the patch passed | | +1 :green_heart: | javac | 1m 22s | the patch passed | | +1 :green_heart: | checkstyle | 1m 26s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 6m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 25m 12s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 48s | the patch passed | | +1 :green_heart: | findbugs | 5m 24s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 176m 40s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 261m 16s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.assignment.TestCloseRegionWhileRSCrash | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1102 | | JIRA Issue | HBASE-23752 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 126028041dfa 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1102/out/precommit/personality/provided.sh | | git revision | HBASE-18095/client-locate-meta-no-zookeeper / 21be409963 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/1/testReport/ | | Max. process+thread count | 4937 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23752) Fix a couple more test failures from nightly run
[
https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025521#comment-17025521
]
HBase QA commented on HBASE-23752:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-18095/client-locate-meta-no-zookeeper Compile
Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 5m
36s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
33s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 12s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}176m 40s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}261m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.assignment.TestCloseRegionWhileRSCrash |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1102/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/1102 |
| JIRA Issue | HBASE-23752 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 126028041dfa 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[GitHub] [hbase] Apache-HBase commented on issue #1104: HBASE-23763 Add 'new on release line report' for git/jira audit tool
Apache-HBase commented on issue #1104: HBASE-23763 Add 'new on release line report' for git/jira audit tool URL: https://github.com/apache/hbase/pull/1104#issuecomment-579548162 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 38s | Maven dependency ordering for branch | | -1 :x: | pylint | 0m 1s | Error running pylint. Please check pylint stderr files. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | -1 :x: | pylint | 0m 1s | Error running pylint. Please check pylint stderr files. | | +1 :green_heart: | pylint | 0m 1s | There were no new pylint issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 3m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1104/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1104 | | Optional Tests | dupname asflicense markdownlint pylint | | uname | Linux 22f1d46f43fd 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1104/out/precommit/personality/provided.sh | | git revision | master / 92a4a1d7b7 | | pylint | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1104/1/artifact/out/branch-pylint-stderr.txt | | pylint | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1104/1/artifact/out/patch-pylint-stderr.txt | | Max. process+thread count | 47 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1104/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) pylint=1.6.5 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work started] (HBASE-23763) Add 'new on release line report' for git/jira audit tool
[
https://issues.apache.org/jira/browse/HBASE-23763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-23763 started by Nick Dimiduk.
> Add 'new on release line report' for git/jira audit tool
>
>
> Key: HBASE-23763
> URL: https://issues.apache.org/jira/browse/HBASE-23763
> Project: HBase
> Issue Type: Task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0
>
>
> Codify building the summary of what's new on a release line branch (i.e.,
> {{branch-2}}), but not yet released on earlier release branches of that line.
> Builds a cvs report that looks like
> [https://home.apache.org/~ndimiduk/new_for_branch-2.csv|https://home.apache.org/~ndimiduk/new_for_branch-2.csv]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #1104: HBASE-23763 Add 'new on release line report' for git/jira audit tool
ndimiduk opened a new pull request #1104: HBASE-23763 Add 'new on release line report' for git/jira audit tool URL: https://github.com/apache/hbase/pull/1104 Codify building the summary of what's new on a release line branch (i.e., `branch-2`), but not yet released on earlier release branches of that line. Builds a cvs report that looks like https://home.apache.org/~ndimiduk/new_for_branch-2.csv Also clean up arguments by grouping them into related categories, and allow selecting which actions to take (i.e., build db from git, build db from jira, generate report). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23304) Implement RPCs needed for master based registry
[ https://issues.apache.org/jira/browse/HBASE-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-23304: - Fix Version/s: (was: HBASE-18095) 2.3.0 3.0.0 > Implement RPCs needed for master based registry > --- > > Key: HBASE-23304 > URL: https://issues.apache.org/jira/browse/HBASE-23304 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 3.0.0 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > We need to implement RPCs on masters needed by client to fetch information > like clusterID, active master server name, meta locations etc. These RPCs are > used by clients during connection init. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23257) Track ClusterID in stand by masters
[ https://issues.apache.org/jira/browse/HBASE-23257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-23257: - Fix Version/s: 2.3.0 > Track ClusterID in stand by masters > --- > > Key: HBASE-23257 > URL: https://issues.apache.org/jira/browse/HBASE-23257 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 3.0.0 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Currently, only active master tracks the cluster ID. As a part of removing > client dependency on ZK (HBASE-18095), it was noted that having stand by > masters serve ClusterID will help load balance the client requests instead of > hot-spotting the active master. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23281) Track meta region changes on masters
[ https://issues.apache.org/jira/browse/HBASE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-23281: - Fix Version/s: (was: HBASE-18095) 2.3.0 3.0.0 > Track meta region changes on masters > > > Key: HBASE-23281 > URL: https://issues.apache.org/jira/browse/HBASE-23281 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 3.0.0 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > This subtask is for tracking meta region changes on masters. This was already > done as a part of prototype patch for HBASE-18095. The goal is to create a > separate PR with small scope of changes so that it is easier to review. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23275) Track active master server name in ActiveMasterManager
[ https://issues.apache.org/jira/browse/HBASE-23275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-23275: - Fix Version/s: (was: HBASE-18095) 2.3.0 3.0.0 > Track active master server name in ActiveMasterManager > -- > > Key: HBASE-23275 > URL: https://issues.apache.org/jira/browse/HBASE-23275 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 3.0.0 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > We already track whether the cluster has an active master, it is just another > RPC to the zookeeper to fetch the active master's hostname. Tracking it helps > load balance client requests to fetch the active master information. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23763) Add 'new on release line report' for git/jira audit tool
[
https://issues.apache.org/jira/browse/HBASE-23763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-23763:
-
Description:
Codify building the summary of what's new on a release line branch (i.e.,
{{branch-2}}), but not yet released on earlier release branches of that line.
Builds a cvs report that looks like
[https://home.apache.org/~ndimiduk/new_for_branch-2.csv|https://home.apache.org/~ndimiduk/new_for_branch-2.csv]
was:
Codify building the summary of what's new on a release line branch (i.e.,
{{branch-2}}), but not yet released on earlier release branches of that line.
Builds a cvs report that looks like
http://home.apache.org/~ndimiduk/new_for_branch-2.csv
> Add 'new on release line report' for git/jira audit tool
>
>
> Key: HBASE-23763
> URL: https://issues.apache.org/jira/browse/HBASE-23763
> Project: HBase
> Issue Type: Task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0
>
>
> Codify building the summary of what's new on a release line branch (i.e.,
> {{branch-2}}), but not yet released on earlier release branches of that line.
> Builds a cvs report that looks like
> [https://home.apache.org/~ndimiduk/new_for_branch-2.csv|https://home.apache.org/~ndimiduk/new_for_branch-2.csv]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23763) Add 'new on release line report' for git/jira audit tool
Nick Dimiduk created HBASE-23763:
Summary: Add 'new on release line report' for git/jira audit tool
Key: HBASE-23763
URL: https://issues.apache.org/jira/browse/HBASE-23763
Project: HBase
Issue Type: Task
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 3.0.0
Codify building the summary of what's new on a release line branch (i.e.,
{{branch-2}}), but not yet released on earlier release branches of that line.
Builds a cvs report that looks like
http://home.apache.org/~ndimiduk/new_for_branch-2.csv
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23762) [OpenTracing] Add documentation
Wei-Chiu Chuang created HBASE-23762: --- Summary: [OpenTracing] Add documentation Key: HBASE-23762 URL: https://issues.apache.org/jira/browse/HBASE-23762 Project: HBase Issue Type: Sub-task Components: tracing Affects Versions: 3.0.0 Reporter: Wei-Chiu Chuang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025515#comment-17025515 ] Michael Stack commented on HBASE-18095: --- Rebased [~bharathv] > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Kyle Purtell >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch > > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23761) The new cache entry can overflow the maxSize in CachedEntryQueue
Swaroopa Kadam created HBASE-23761: -- Summary: The new cache entry can overflow the maxSize in CachedEntryQueue Key: HBASE-23761 URL: https://issues.apache.org/jira/browse/HBASE-23761 Project: HBase Issue Type: Bug Affects Versions: 2.1.8 Reporter: Swaroopa Kadam Assignee: Swaroopa Kadam When we check whether the new entry should be added into CachedEntryQueue#queue, we need to be sure that after adding the entry to the queue, the cache size does not overflow maxSize. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] bharathv commented on issue #1097: HBASE-23281: Track meta region locations in masters (#830)
bharathv commented on issue #1097: HBASE-23281: Track meta region locations in masters (#830) URL: https://github.com/apache/hbase/pull/1097#issuecomment-579542239 Will be merged as a part of #1098 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv closed pull request #1097: HBASE-23281: Track meta region locations in masters (#830)
bharathv closed pull request #1097: HBASE-23281: Track meta region locations in masters (#830) URL: https://github.com/apache/hbase/pull/1097 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv closed pull request #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812)
bharathv closed pull request #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812) URL: https://github.com/apache/hbase/pull/1096 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv closed pull request #1095: HBASE-23257: Track clusterID in stand by masters (#798)
bharathv closed pull request #1095: HBASE-23257: Track clusterID in stand by masters (#798) URL: https://github.com/apache/hbase/pull/1095 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23760) (2.1) Helper method to configure secure DFS cluster for tests
[ https://issues.apache.org/jira/browse/HBASE-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-23760: --- Status: Patch Available (was: Open) > (2.1) Helper method to configure secure DFS cluster for tests > - > > Key: HBASE-23760 > URL: https://issues.apache.org/jira/browse/HBASE-23760 > Project: HBase > Issue Type: Task > Components: security, test >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.9 > > Attachments: HBASE-23760.001.branch-2.1.patch > > > Let's get [~weichiu]'s HBASE-20950 onto branch-2.1. It will make the backport > for HBASE-17115 that much easier. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] bharathv commented on issue #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812)
bharathv commented on issue #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812) URL: https://github.com/apache/hbase/pull/1096#issuecomment-579542165 Will be merged as a part of #1098 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] apurtell merged pull request #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904)
apurtell merged pull request #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904) URL: https://github.com/apache/hbase/pull/1098 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv commented on issue #1095: HBASE-23257: Track clusterID in stand by masters (#798)
bharathv commented on issue #1095: HBASE-23257: Track clusterID in stand by masters (#798) URL: https://github.com/apache/hbase/pull/1095#issuecomment-579542077 Will be merged as a part of #1098 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22120) Replace HTrace with OpenTracing
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025513#comment-17025513 ] Wei-Chiu Chuang commented on HBASE-22120: - Started a thread to discuss a feature branch for this. > Replace HTrace with OpenTracing > --- > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: New Feature > Components: tracing >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Assignee: Wei-Chiu Chuang >Priority: Major > > h2. Deprecate HTrace usage in HBase > * HBase 1.x (branch-1) > * Declare HTrace (htrace 3.x) deprecated in the user doc. > * HBase 2.x (branch-2) > * Declare HTrace deprecated in the user doc. Furthermore, state that it is > known not working. > * Either fix the trace context propagation bug in HBase 2.x, or backport > OpenTracing support from the master branch. I am inclined to the latter. > * HBase 3.x (master branch) > * Remove HTrace entirely. > * Add OpenTracing APIs. Potentially backport to HBase 2.4. > * Replace OpenTracing API with OpenTelemetry when the latter stabilizes. > h1. Milestones > # Doc -- deprecation notice > # Replace existing HTrace code with OpenTracing code in the master branch > (3.x) > # Java (a [poc|https://github.com/jojochuang/hbase/tree/HBASE-22120] is > currently under way) > # HBase shell and scripts (Ruby, shell script) > # Doc > # Add new trace instrumentation code for new features not instrumented by > the existing HTace code. > # Propagate the traces to other systems such as HDFS and MapReduce. > # Support other OpenTracing tracers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on issue #754: HBASE-22978 : Online slow response log
virajjasani commented on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-579541178 @ndimiduk @saintstack Would you be able to spend some time for review today?(I am available today) If not, I can work on this after 2/3 days. Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1095: HBASE-23257: Track clusterID in stand by masters (#798)
bharathv commented on a change in pull request #1095: HBASE-23257: Track
clusterID in stand by masters (#798)
URL: https://github.com/apache/hbase/pull/1095#discussion_r372135621
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/CachedClusterId.java
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.ClusterId;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+
+/**
+ * Caches the cluster ID of the cluster. For standby masters, this is used to
serve the client
+ * RPCs that fetch the cluster ID. ClusterID is only created by an active
master if one does not
+ * already exist. Standby masters just read the information from the file
system. This class is
+ * thread-safe.
+ *
+ * TODO: Make it a singleton without affecting concurrent junit tests.
+ */
+@InterfaceAudience.Private
+public class CachedClusterId {
+
+ public static final Logger LOG =
LoggerFactory.getLogger(CachedClusterId.class);
+ private static final int MAX_FETCH_TIMEOUT_MS = 1;
+
+ private Path rootDir;
+ private FileSystem fs;
+
+ // When true, indicates that a FileSystem fetch of ClusterID is in progress.
This is used to
+ // avoid multiple fetches from FS and let only one thread fetch the
information.
+ AtomicBoolean fetchInProgress = new AtomicBoolean(false);
+
+ // When true, it means that the cluster ID has been fetched successfully
from fs.
+ private AtomicBoolean isClusterIdSet = new AtomicBoolean(false);
+ // Immutable once set and read multiple times.
+ private ClusterId clusterId;
+
+ // cache stats for testing.
+ private AtomicInteger cacheMisses = new AtomicInteger(0);
Review comment:
Agree... wish Java had C++ like macros that we could use to inject DEBUG
only code :D.
This code also exists in the master, I'll have to cleanup from there. Can I
do it as a follow up?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812)
apurtell commented on a change in pull request #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812) URL: https://github.com/apache/hbase/pull/1096#discussion_r372135673 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java ## @@ -17,25 +17,24 @@ * limitations under the License. */ package org.apache.hadoop.hbase.master; - import java.io.IOException; +import java.util.Optional; import java.util.concurrent.atomic.AtomicBoolean; - -import org.apache.hadoop.hbase.zookeeper.MasterAddressTracker; -import org.apache.hadoop.hbase.zookeeper.ZKUtil; -import org.apache.hadoop.hbase.zookeeper.ZKWatcher; -import org.apache.hadoop.hbase.zookeeper.ZNodePaths; -import org.apache.yetus.audience.InterfaceAudience; import org.apache.hadoop.hbase.Server; import org.apache.hadoop.hbase.ServerName; import org.apache.hadoop.hbase.ZNodeClearer; import org.apache.hadoop.hbase.exceptions.DeserializationException; import org.apache.hadoop.hbase.monitoring.MonitoredTask; -import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil; +import org.apache.hadoop.hbase.zookeeper.MasterAddressTracker; import org.apache.hadoop.hbase.zookeeper.ZKListener; +import org.apache.hadoop.hbase.zookeeper.ZKUtil; +import org.apache.hadoop.hbase.zookeeper.ZKWatcher; +import org.apache.hadoop.hbase.zookeeper.ZNodePaths; +import org.apache.yetus.audience.InterfaceAudience; import org.apache.zookeeper.KeeperException; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil; Review comment: I guess it doesn't, never mind... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812)
apurtell commented on a change in pull request #1096: HBASE-23275: Track active
master's address in ActiveMasterManager (#812)
URL: https://github.com/apache/hbase/pull/1096#discussion_r372135345
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/CachedClusterId.java
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.ClusterId;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+
+/**
+ * Caches the cluster ID of the cluster. For standby masters, this is used to
serve the client
+ * RPCs that fetch the cluster ID. ClusterID is only created by an active
master if one does not
+ * already exist. Standby masters just read the information from the file
system. This class is
+ * thread-safe.
+ *
+ * TODO: Make it a singleton without affecting concurrent junit tests.
+ */
+@InterfaceAudience.Private
+public class CachedClusterId {
+
+ public static final Logger LOG =
LoggerFactory.getLogger(CachedClusterId.class);
+ private static final int MAX_FETCH_TIMEOUT_MS = 1;
+
+ private Path rootDir;
+ private FileSystem fs;
+
+ // When true, indicates that a FileSystem fetch of ClusterID is in progress.
This is used to
+ // avoid multiple fetches from FS and let only one thread fetch the
information.
+ AtomicBoolean fetchInProgress = new AtomicBoolean(false);
+
+ // When true, it means that the cluster ID has been fetched successfully
from fs.
+ private AtomicBoolean isClusterIdSet = new AtomicBoolean(false);
+ // Immutable once set and read multiple times.
+ private ClusterId clusterId;
+
+ // cache stats for testing.
+ private AtomicInteger cacheMisses = new AtomicInteger(0);
+
+ public CachedClusterId(Configuration conf) throws IOException {
+rootDir = FSUtils.getRootDir(conf);
+fs = rootDir.getFileSystem(conf);
+ }
+
+ /**
+ * Succeeds only once, when setting to a non-null value. Overwrites are not
allowed.
+ */
+ private void setClusterId(ClusterId id) {
+if (id == null || isClusterIdSet.get()) {
+ return;
+}
+clusterId = id;
+isClusterIdSet.set(true);
+ }
+
+ /**
+ * Returns a cached copy of the cluster ID. null if the cache is not
populated.
+ */
+ private String getClusterId() {
+if (!isClusterIdSet.get()) {
+ return null;
+}
+// It is ok to read without a lock since clusterId is immutable once set.
+return clusterId.toString();
+ }
+
+ /**
+ * Attempts to fetch the cluster ID from the file system. If no attempt is
already in progress,
+ * synchronously fetches the cluster ID and sets it. If an attempt is
already in progress,
+ * returns right away and the caller is expected to wait for the fetch to
finish.
+ * @return true if the attempt is done, false if another thread is already
fetching it.
+ */
+ private boolean attemptFetch() {
+if (fetchInProgress.compareAndSet(false, true)) {
+ // A fetch is not in progress, so try fetching the cluster ID
synchronously and then notify
+ // the waiting threads.
+ try {
+cacheMisses.incrementAndGet();
+setClusterId(FSUtils.getClusterId(fs, rootDir));
+ } catch (IOException e) {
+LOG.warn("Error fetching cluster ID", e);
+ } finally {
+Preconditions.checkState(fetchInProgress.compareAndSet(true, false));
+synchronized (fetchInProgress) {
+ fetchInProgress.notifyAll();
+}
+ }
+ return true;
+}
+return false;
+ }
+
+ private void waitForFetchToFinish() throws InterruptedException {
+synchronized (fetchInProgress) {
+ while
[GitHub] [hbase] apurtell commented on a change in pull request #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812)
apurtell commented on a change in pull request #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812) URL: https://github.com/apache/hbase/pull/1096#discussion_r372134729 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java ## @@ -17,25 +17,24 @@ * limitations under the License. */ package org.apache.hadoop.hbase.master; - import java.io.IOException; +import java.util.Optional; import java.util.concurrent.atomic.AtomicBoolean; - -import org.apache.hadoop.hbase.zookeeper.MasterAddressTracker; -import org.apache.hadoop.hbase.zookeeper.ZKUtil; -import org.apache.hadoop.hbase.zookeeper.ZKWatcher; -import org.apache.hadoop.hbase.zookeeper.ZNodePaths; -import org.apache.yetus.audience.InterfaceAudience; import org.apache.hadoop.hbase.Server; import org.apache.hadoop.hbase.ServerName; import org.apache.hadoop.hbase.ZNodeClearer; import org.apache.hadoop.hbase.exceptions.DeserializationException; import org.apache.hadoop.hbase.monitoring.MonitoredTask; -import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil; +import org.apache.hadoop.hbase.zookeeper.MasterAddressTracker; import org.apache.hadoop.hbase.zookeeper.ZKListener; +import org.apache.hadoop.hbase.zookeeper.ZKUtil; +import org.apache.hadoop.hbase.zookeeper.ZKWatcher; +import org.apache.hadoop.hbase.zookeeper.ZNodePaths; +import org.apache.yetus.audience.InterfaceAudience; import org.apache.zookeeper.KeeperException; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil; Review comment: Does this show up as a checkstyle nit? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812)
apurtell commented on a change in pull request #1096: HBASE-23275: Track active
master's address in ActiveMasterManager (#812)
URL: https://github.com/apache/hbase/pull/1096#discussion_r372135013
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/CachedClusterId.java
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.ClusterId;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+
+/**
+ * Caches the cluster ID of the cluster. For standby masters, this is used to
serve the client
+ * RPCs that fetch the cluster ID. ClusterID is only created by an active
master if one does not
+ * already exist. Standby masters just read the information from the file
system. This class is
+ * thread-safe.
+ *
+ * TODO: Make it a singleton without affecting concurrent junit tests.
+ */
+@InterfaceAudience.Private
+public class CachedClusterId {
+
+ public static final Logger LOG =
LoggerFactory.getLogger(CachedClusterId.class);
+ private static final int MAX_FETCH_TIMEOUT_MS = 1;
+
+ private Path rootDir;
+ private FileSystem fs;
+
+ // When true, indicates that a FileSystem fetch of ClusterID is in progress.
This is used to
+ // avoid multiple fetches from FS and let only one thread fetch the
information.
+ AtomicBoolean fetchInProgress = new AtomicBoolean(false);
+
+ // When true, it means that the cluster ID has been fetched successfully
from fs.
+ private AtomicBoolean isClusterIdSet = new AtomicBoolean(false);
+ // Immutable once set and read multiple times.
+ private ClusterId clusterId;
+
+ // cache stats for testing.
+ private AtomicInteger cacheMisses = new AtomicInteger(0);
Review comment:
Same comment as per the master version of this change: A bit wasteful just
for a unit test? Nit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1103: HBASE-23753 Update of errorprone generated failures
saintstack commented on a change in pull request #1103: HBASE-23753 Update of
errorprone generated failures
URL: https://github.com/apache/hbase/pull/1103#discussion_r372134839
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
##
@@ -152,12 +153,11 @@ public long getSeqid() {
* priority beyond FIFO, they should override {@link #getPriority()}.
*/
@Override
- public int compareTo(Runnable o) {
-EventHandler eh = (EventHandler)o;
-if(getPriority() != eh.getPriority()) {
- return (getPriority() < eh.getPriority()) ? -1 : 1;
+ public int compareTo(EventHandler o) {
+if(getPriority() != o.getPriority()) {
Review comment:
Will do
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1103: HBASE-23753 Update of errorprone generated failures
saintstack commented on a change in pull request #1103: HBASE-23753 Update of
errorprone generated failures
URL: https://github.com/apache/hbase/pull/1103#discussion_r372135010
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSinkManager.java
##
@@ -65,12 +90,8 @@ public void testChooseSinks() {
for (int i = 0; i < totalServers; i++) {
serverNames.add(mock(ServerName.class));
}
-
-when(replicationEndpoint.getRegionServers())
- .thenReturn(serverNames);
-
+this.replicationEndpoint.setRegionServers(serverNames);
Review comment:
Will change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1103: HBASE-23753 Update of errorprone generated failures
saintstack commented on a change in pull request #1103: HBASE-23753 Update of errorprone generated failures URL: https://github.com/apache/hbase/pull/1103#discussion_r372134724 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java ## @@ -18,6 +18,7 @@ */ package org.apache.hadoop.hbase.executor; +import java.awt.*; Review comment: Mistake. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22775) Enhance logging for peer related operations
[ https://issues.apache.org/jira/browse/HBASE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025508#comment-17025508 ] Andrew Kyle Purtell commented on HBASE-22775: - +1 > Enhance logging for peer related operations > --- > > Key: HBASE-22775 > URL: https://issues.apache.org/jira/browse/HBASE-22775 > Project: HBase > Issue Type: Improvement >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Minor > Attachments: HBASE-22775.master.001.patch, > HBASE-22775.master.002.patch > > > Now we don't have good logging regarding peer operations, for example addPeer > does not log itself: > [https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationPeerStorage.java#L102] > This Jira is aiming to enhancing this area -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23751) Move core to hbase-thirdparty 3.2.0
[
https://issues.apache.org/jira/browse/HBASE-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025506#comment-17025506
]
Hudson commented on HBASE-23751:
Results for branch branch-2
[build #2436 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2436/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2436//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2436//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2436//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move core to hbase-thirdparty 3.2.0
> ---
>
> Key: HBASE-23751
> URL: https://issues.apache.org/jira/browse/HBASE-23751
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23751-Move-core-to-hbase-thirdparty-3.2.0.patch
>
>
> Move core to use just-released 3.2.0 hbase-thirdparty. The change has
> already been through build courtesy of
> https://github.com/apache/hbase/pull/1086
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] apurtell commented on a change in pull request #1095: HBASE-23257: Track clusterID in stand by masters (#798)
apurtell commented on a change in pull request #1095: HBASE-23257: Track
clusterID in stand by masters (#798)
URL: https://github.com/apache/hbase/pull/1095#discussion_r372131884
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/CachedClusterId.java
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.ClusterId;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+
+/**
+ * Caches the cluster ID of the cluster. For standby masters, this is used to
serve the client
+ * RPCs that fetch the cluster ID. ClusterID is only created by an active
master if one does not
+ * already exist. Standby masters just read the information from the file
system. This class is
+ * thread-safe.
+ *
+ * TODO: Make it a singleton without affecting concurrent junit tests.
+ */
+@InterfaceAudience.Private
+public class CachedClusterId {
+
+ public static final Logger LOG =
LoggerFactory.getLogger(CachedClusterId.class);
+ private static final int MAX_FETCH_TIMEOUT_MS = 1;
+
+ private Path rootDir;
+ private FileSystem fs;
+
+ // When true, indicates that a FileSystem fetch of ClusterID is in progress.
This is used to
+ // avoid multiple fetches from FS and let only one thread fetch the
information.
+ AtomicBoolean fetchInProgress = new AtomicBoolean(false);
+
+ // When true, it means that the cluster ID has been fetched successfully
from fs.
+ private AtomicBoolean isClusterIdSet = new AtomicBoolean(false);
+ // Immutable once set and read multiple times.
+ private ClusterId clusterId;
+
+ // cache stats for testing.
+ private AtomicInteger cacheMisses = new AtomicInteger(0);
Review comment:
Nit: A bit wasteful to have this just for a test?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1103: HBASE-23753 Update of errorprone generated failures
bharathv commented on a change in pull request #1103: HBASE-23753 Update of
errorprone generated failures
URL: https://github.com/apache/hbase/pull/1103#discussion_r372133413
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSinkManager.java
##
@@ -65,12 +90,8 @@ public void testChooseSinks() {
for (int i = 0; i < totalServers; i++) {
serverNames.add(mock(ServerName.class));
}
-
-when(replicationEndpoint.getRegionServers())
- .thenReturn(serverNames);
-
+this.replicationEndpoint.setRegionServers(serverNames);
Review comment:
nit: I guess 'this' is redundant in multiple places.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1103: HBASE-23753 Update of errorprone generated failures
bharathv commented on a change in pull request #1103: HBASE-23753 Update of
errorprone generated failures
URL: https://github.com/apache/hbase/pull/1103#discussion_r372132709
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
##
@@ -152,12 +153,11 @@ public long getSeqid() {
* priority beyond FIFO, they should override {@link #getPriority()}.
*/
@Override
- public int compareTo(Runnable o) {
-EventHandler eh = (EventHandler)o;
-if(getPriority() != eh.getPriority()) {
- return (getPriority() < eh.getPriority()) ? -1 : 1;
+ public int compareTo(EventHandler o) {
+if(getPriority() != o.getPriority()) {
Review comment:
nit: include a null check in the beginning for o?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1103: HBASE-23753 Update of errorprone generated failures
bharathv commented on a change in pull request #1103: HBASE-23753 Update of errorprone generated failures URL: https://github.com/apache/hbase/pull/1103#discussion_r372131545 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java ## @@ -18,6 +18,7 @@ */ package org.apache.hadoop.hbase.executor; +import java.awt.*; Review comment: nit: looks incorrect? Might also run into checkstyle issue with start import. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23760) (2.1) Helper method to configure secure DFS cluster for tests
[ https://issues.apache.org/jira/browse/HBASE-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-23760: --- Attachment: HBASE-23760.001.branch-2.1.patch > (2.1) Helper method to configure secure DFS cluster for tests > - > > Key: HBASE-23760 > URL: https://issues.apache.org/jira/browse/HBASE-23760 > Project: HBase > Issue Type: Task > Components: security, test >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.9 > > Attachments: HBASE-23760.001.branch-2.1.patch > > > Let's get [~weichiu]'s HBASE-20950 onto branch-2.1. It will make the backport > for HBASE-17115 that much easier. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23760) (2.1) Helper method to configure secure DFS cluster for tests
[ https://issues.apache.org/jira/browse/HBASE-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-23760: --- Attachment: HBASE-23760.001.patch > (2.1) Helper method to configure secure DFS cluster for tests > - > > Key: HBASE-23760 > URL: https://issues.apache.org/jira/browse/HBASE-23760 > Project: HBase > Issue Type: Task > Components: security, test >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.9 > > > Let's get [~weichiu]'s HBASE-20950 onto branch-2.1. It will make the backport > for HBASE-17115 that much easier. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23760) (2.1) Helper method to configure secure DFS cluster for tests
[ https://issues.apache.org/jira/browse/HBASE-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-23760: --- Attachment: (was: HBASE-23760.001.patch) > (2.1) Helper method to configure secure DFS cluster for tests > - > > Key: HBASE-23760 > URL: https://issues.apache.org/jira/browse/HBASE-23760 > Project: HBase > Issue Type: Task > Components: security, test >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.9 > > > Let's get [~weichiu]'s HBASE-20950 onto branch-2.1. It will make the backport > for HBASE-17115 that much easier. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23752) Fix a couple more test failures from nightly run
[ https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-23752: - Description: After merging the patches HBASE-23731 and HBASE-23647, nightly run found the following issues. TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR runs missed it somehow. TestCustomSaslAuthenticationProvider - This was written pre master registry. Has to be updated. It was never run as a part of the test suite. TestSnapshotScannerHDFSAclController: Cluster restart did not reset the cached connection state [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it accordingly. Will submit a PR shortly. was: After merging the patches HBASE-23731 and HBASE-23647, nightly run found a couple of issues, TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR runs missed it somehow. TestCustomSaslAuthenticationProvider - This was written pre master registry. Has to be updated. It was never run as a part of the test suite. TestSnapshotScannerHDFSAclController: Cluster restart did not reset the cached connection state [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it accordingly. Will submit a PR shortly. > Fix a couple more test failures from nightly run > > > Key: HBASE-23752 > URL: https://issues.apache.org/jira/browse/HBASE-23752 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.3.0, 1.6.0, HBASE-18095 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > > After merging the patches HBASE-23731 and HBASE-23647, nightly run found the > following issues. > TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR > runs missed it somehow. > TestCustomSaslAuthenticationProvider - This was written pre master registry. > Has to be updated. It was never run as a part of the test suite. > TestSnapshotScannerHDFSAclController: Cluster restart did not reset the > cached connection state > > [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it > accordingly. Will submit a PR shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23752) Fix a couple more test failures from nightly run
[ https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-23752: - Description: After merging the patches HBASE-23731 and HBASE-23647, nightly run found a couple of issues, TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR runs missed it somehow. TestCustomSaslAuthenticationProvider - This was written pre master registry. Has to be updated. It was never run as a part of the test suite. TestSnapshotScannerHDFSAclController: Cluster restart did not reset the cached connection state [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it accordingly. Will submit a PR shortly. was: After merging the patches HBASE-23731 and HBASE-23647, nightly run found a couple of issues, TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR runs missed it somehow. TestCustomSaslAuthenticationProvider - This was written pre master registry. Has to be updated. It was never run as a part of the test suite. [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it accordingly. Will submit a PR shortly. > Fix a couple more test failures from nightly run > > > Key: HBASE-23752 > URL: https://issues.apache.org/jira/browse/HBASE-23752 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.3.0, 1.6.0, HBASE-18095 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > > After merging the patches HBASE-23731 and HBASE-23647, nightly run found a > couple of issues, > TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR > runs missed it somehow. > TestCustomSaslAuthenticationProvider - This was written pre master registry. > Has to be updated. It was never run as a part of the test suite. > TestSnapshotScannerHDFSAclController: Cluster restart did not reset the > cached connection state > > [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it > accordingly. Will submit a PR shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23752) Fix a couple more test failures from nightly run
[ https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-23752: - Status: Patch Available (was: Open) > Fix a couple more test failures from nightly run > > > Key: HBASE-23752 > URL: https://issues.apache.org/jira/browse/HBASE-23752 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.3.0, 1.6.0, HBASE-18095 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > > After merging the patches HBASE-23731 and HBASE-23647, nightly run found a > couple of issues, > TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR > runs missed it somehow. > TestCustomSaslAuthenticationProvider - This was written pre master registry. > Has to be updated. It was never run as a part of the test suite. > > [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it > accordingly. Will submit a PR shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23760) (2.1) Helper method to configure secure DFS cluster for tests
Josh Elser created HBASE-23760: -- Summary: (2.1) Helper method to configure secure DFS cluster for tests Key: HBASE-23760 URL: https://issues.apache.org/jira/browse/HBASE-23760 Project: HBase Issue Type: Task Components: security, test Reporter: Josh Elser Assignee: Josh Elser Fix For: 2.1.9 Let's get [~weichiu]'s HBASE-20950 onto branch-2.1. It will make the backport for HBASE-17115 that much easier. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23602) TTL Before Which No Data is Purged
[ https://issues.apache.org/jira/browse/HBASE-23602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated HBASE-23602: Fix Version/s: (was: 2.3.0) 2.4.0 > TTL Before Which No Data is Purged > -- > > Key: HBASE-23602 > URL: https://issues.apache.org/jira/browse/HBASE-23602 > Project: HBase > Issue Type: New Feature >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 3.0.0, 1.6.0, 2.4.0 > > > HBase currently offers operators a choice. They can set > KEEP_DELETED_CELLS=true and VERSIONS to max value, plus no TTL, and they will > always have a complete history of all changes (but high storage costs and > penalties to read performance). Or they can have KEEP_DELETED_CELLS=false and > VERSIONS/TTL set to some reasonable values, but that means that major > compactions can destroy the ability to do a consistent snapshot read of any > prior time. (This limits the usefulness and correctness of, for example, > Phoenix's SCN lookback feature.) > I propose having a new TTL property to give a minimum age that an expired or > deleted Cell would have to achieve before it could be purged. (I see that > HBASE-10118 already does something similar for the delete markers > themselves.) > This would allow operators to have a consistent history for some finite > amount of recent time while still purging out the "long tail" of obsolete / > deleted versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23602) TTL Before Which No Data is Purged
[ https://issues.apache.org/jira/browse/HBASE-23602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025500#comment-17025500 ] Geoffrey Jacoby commented on HBASE-23602: - [~ndimiduk] - Thanks for checking. This will probably slip to 2.4, since I haven't had a chance to get started on it yet. I'll adjust the fix version. > TTL Before Which No Data is Purged > -- > > Key: HBASE-23602 > URL: https://issues.apache.org/jira/browse/HBASE-23602 > Project: HBase > Issue Type: New Feature >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > > HBase currently offers operators a choice. They can set > KEEP_DELETED_CELLS=true and VERSIONS to max value, plus no TTL, and they will > always have a complete history of all changes (but high storage costs and > penalties to read performance). Or they can have KEEP_DELETED_CELLS=false and > VERSIONS/TTL set to some reasonable values, but that means that major > compactions can destroy the ability to do a consistent snapshot read of any > prior time. (This limits the usefulness and correctness of, for example, > Phoenix's SCN lookback feature.) > I propose having a new TTL property to give a minimum age that an expired or > deleted Cell would have to achieve before it could be purged. (I see that > HBASE-10118 already does something similar for the delete markers > themselves.) > This would allow operators to have a consistent history for some finite > amount of recent time while still purging out the "long tail" of obsolete / > deleted versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23756) [OpenTracing] Add OpenTracing dependency and helper methods
[ https://issues.apache.org/jira/browse/HBASE-23756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HBASE-23756: --- Assignee: Wei-Chiu Chuang > [OpenTracing] Add OpenTracing dependency and helper methods > --- > > Key: HBASE-23756 > URL: https://issues.apache.org/jira/browse/HBASE-23756 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22120) Replace HTrace with OpenTracing
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-22120: Description: h2. Deprecate HTrace usage in HBase * HBase 1.x (branch-1) * Declare HTrace (htrace 3.x) deprecated in the user doc. * HBase 2.x (branch-2) * Declare HTrace deprecated in the user doc. Furthermore, state that it is known not working. * Either fix the trace context propagation bug in HBase 2.x, or backport OpenTracing support from the master branch. I am inclined to the latter. * HBase 3.x (master branch) * Remove HTrace entirely. * Add OpenTracing APIs. Potentially backport to HBase 2.4. * Replace OpenTracing API with OpenTelemetry when the latter stabilizes. h1. Milestones # Doc -- deprecation notice # Replace existing HTrace code with OpenTracing code in the master branch (3.x) # Java (a [poc|https://github.com/jojochuang/hbase/tree/HBASE-22120] is currently under way) # HBase shell and scripts (Ruby, shell script) # Doc # Add new trace instrumentation code for new features not instrumented by the existing HTace code. # Propagate the traces to other systems such as HDFS and MapReduce. # Support other OpenTracing tracers. was:Suggested in HBASE-22115 > Replace HTrace with OpenTracing > --- > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: New Feature > Components: tracing >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Assignee: Wei-Chiu Chuang >Priority: Major > > h2. Deprecate HTrace usage in HBase > * HBase 1.x (branch-1) > * Declare HTrace (htrace 3.x) deprecated in the user doc. > * HBase 2.x (branch-2) > * Declare HTrace deprecated in the user doc. Furthermore, state that it is > known not working. > * Either fix the trace context propagation bug in HBase 2.x, or backport > OpenTracing support from the master branch. I am inclined to the latter. > * HBase 3.x (master branch) > * Remove HTrace entirely. > * Add OpenTracing APIs. Potentially backport to HBase 2.4. > * Replace OpenTracing API with OpenTelemetry when the latter stabilizes. > h1. Milestones > # Doc -- deprecation notice > # Replace existing HTrace code with OpenTracing code in the master branch > (3.x) > # Java (a [poc|https://github.com/jojochuang/hbase/tree/HBASE-22120] is > currently under way) > # HBase shell and scripts (Ruby, shell script) > # Doc > # Add new trace instrumentation code for new features not instrumented by > the existing HTace code. > # Propagate the traces to other systems such as HDFS and MapReduce. > # Support other OpenTracing tracers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23759) [OpenTracing] Migrate hbase shell and scripts from HTrace to OpenTracing
[ https://issues.apache.org/jira/browse/HBASE-23759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HBASE-23759: --- Assignee: Wei-Chiu Chuang > [OpenTracing] Migrate hbase shell and scripts from HTrace to OpenTracing > > > Key: HBASE-23759 > URL: https://issues.apache.org/jira/browse/HBASE-23759 > Project: HBase > Issue Type: Sub-task >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23757) [OpenTracing] Migrate from HTrace to OpenTracing (Java code)
[ https://issues.apache.org/jira/browse/HBASE-23757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HBASE-23757: --- Assignee: Wei-Chiu Chuang > [OpenTracing] Migrate from HTrace to OpenTracing (Java code) > > > Key: HBASE-23757 > URL: https://issues.apache.org/jira/browse/HBASE-23757 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23758) [OpenTracing] Propagate trace context to server correctly
[ https://issues.apache.org/jira/browse/HBASE-23758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HBASE-23758: --- Assignee: Wei-Chiu Chuang > [OpenTracing] Propagate trace context to server correctly > - > > Key: HBASE-23758 > URL: https://issues.apache.org/jira/browse/HBASE-23758 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > HBASE-18601 breaks trace propagation. Instead of fixing it for HTrace, we > should fix it for OpenTracing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22120) Replace HTrace with OpenTracing
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-22120: Component/s: tracing > Replace HTrace with OpenTracing > --- > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: New Feature > Components: tracing >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Assignee: Wei-Chiu Chuang >Priority: Major > > Suggested in HBASE-22115 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23759) [OpenTracing] Migrate hbase shell and scripts from HTrace to OpenTracing
Wei-Chiu Chuang created HBASE-23759: --- Summary: [OpenTracing] Migrate hbase shell and scripts from HTrace to OpenTracing Key: HBASE-23759 URL: https://issues.apache.org/jira/browse/HBASE-23759 Project: HBase Issue Type: Sub-task Reporter: Wei-Chiu Chuang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23754) [DOC] HBase Web UI runs on different port
[
https://issues.apache.org/jira/browse/HBASE-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025496#comment-17025496
]
Sean Busbey commented on HBASE-23754:
-
The default port for the web ui of the master is still 16010, so I'm curious on
what configs you have set up.
> [DOC] HBase Web UI runs on different port
> -
>
> Key: HBASE-23754
> URL: https://issues.apache.org/jira/browse/HBASE-23754
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Yiğit Oner
>Priority: Trivial
>
> Just started to work through the ref guide and started up the stand-alone
> setup. The guide mentions to look up [http://localhost:16010/] for the HBase
> Web UI, but there is no process listening to that port.
>
> After checking netstat and logs I've discovered, that the UI runs on port
> 43105
>
> {code}
> 2020-01-28 22:17:43,658 INFO [main] http.HttpServer: Jetty bound to port 43105
> 2020-01-28 22:17:43,658 INFO [main] mortbay.log: jetty-6.1.26
> 2020-01-28 22:17:43,697 INFO [main] mortbay.log: Started
> SelectChannelConnector@0.0.0.0:4310
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23754) [DOC] HBase Web UI runs on different port
[
https://issues.apache.org/jira/browse/HBASE-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025492#comment-17025492
]
Sean Busbey commented on HBASE-23754:
-
That doesn't sound right. Is the "use random ports" feature on maybe
> [DOC] HBase Web UI runs on different port
> -
>
> Key: HBASE-23754
> URL: https://issues.apache.org/jira/browse/HBASE-23754
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Yiğit Oner
>Priority: Trivial
>
> Just started to work through the ref guide and started up the stand-alone
> setup. The guide mentions to look up [http://localhost:16010/] for the HBase
> Web UI, but there is no process listening to that port.
>
> After checking netstat and logs I've discovered, that the UI runs on port
> 43105
>
> {code}
> 2020-01-28 22:17:43,658 INFO [main] http.HttpServer: Jetty bound to port 43105
> 2020-01-28 22:17:43,658 INFO [main] mortbay.log: jetty-6.1.26
> 2020-01-28 22:17:43,697 INFO [main] mortbay.log: Started
> SelectChannelConnector@0.0.0.0:4310
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #754: HBASE-22978 : Online slow response log
Apache-HBase commented on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-579516300 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 16s | master passed | | +1 :green_heart: | compile | 3m 5s | master passed | | +1 :green_heart: | checkstyle | 2m 22s | master passed | | +0 :ok: | refguide | 4m 42s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 4m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 7s | master passed | | +0 :ok: | spotbugs | 1m 20s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 23m 21s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 59s | the patch passed | | +1 :green_heart: | compile | 3m 4s | the patch passed | | +1 :green_heart: | cc | 3m 4s | the patch passed | | +1 :green_heart: | javac | 3m 4s | the patch passed | | -1 :x: | checkstyle | 2m 20s | root: The patch generated 1 new + 448 unchanged - 2 fixed = 449 total (was 450) | | +1 :green_heart: | rubocop | 0m 8s | There were no new rubocop issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 4m 46s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 4m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 2s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 8m 18s | the patch passed | | +1 :green_heart: | javadoc | 5m 16s | the patch passed | | +1 :green_heart: | findbugs | 25m 43s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 251m 26s | root in the patch failed. | | +1 :green_heart: | asflicense | 5m 3s | The patch does not generate ASF License warnings. | | | | 389m 56s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestAdminShell | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/754 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml cc hbaseprotoc prototool rubocop | | uname | Linux a30d514f1d69 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-754/out/precommit/personality/provided.sh | | git revision | master / e836f5684f | | Default Java | 1.8.0_181 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/8/artifact/out/branch-site/book.html | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/8/artifact/out/diff-checkstyle-root.txt | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/8/artifact/out/patch-site/book.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/8/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/8/testReport/ | | Max. process+thread count | 5787 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-server hbase-thrift hbase-shell . U: . | |
[jira] [Created] (HBASE-23758) [OpenTracing] Propagate trace context to server correctly
Wei-Chiu Chuang created HBASE-23758: --- Summary: [OpenTracing] Propagate trace context to server correctly Key: HBASE-23758 URL: https://issues.apache.org/jira/browse/HBASE-23758 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0-beta-1 Reporter: Wei-Chiu Chuang HBASE-18601 breaks trace propagation. Instead of fixing it for HTrace, we should fix it for OpenTracing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025486#comment-17025486 ] Bharath Vissapragada commented on HBASE-18095: -- [~stack] Sure. I'll put up a discuss thread on dev once the branch is stable. I'm working on fixing on the last set of test failures exposed in nightly (via HBASE-23752). > Does the branch need updating (seems old when I compare to master) Yes please. Can you please rebase it when you get a chance? > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Kyle Purtell >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch > > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23757) [OpenTracing] Migrate from HTrace to OpenTracing (Java code)
Wei-Chiu Chuang created HBASE-23757: --- Summary: [OpenTracing] Migrate from HTrace to OpenTracing (Java code) Key: HBASE-23757 URL: https://issues.apache.org/jira/browse/HBASE-23757 Project: HBase Issue Type: Sub-task Affects Versions: 3.0.0 Reporter: Wei-Chiu Chuang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23756) [OpenTracing] Add OpenTracing dependency and helper methods
Wei-Chiu Chuang created HBASE-23756: --- Summary: [OpenTracing] Add OpenTracing dependency and helper methods Key: HBASE-23756 URL: https://issues.apache.org/jira/browse/HBASE-23756 Project: HBase Issue Type: Sub-task Affects Versions: 3.0.0 Reporter: Wei-Chiu Chuang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23755) [OpenTracing] Declare HTrace is unusable in the user doc
[ https://issues.apache.org/jira/browse/HBASE-23755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-23755: Parent: HBASE-22120 Issue Type: Sub-task (was: Improvement) > [OpenTracing] Declare HTrace is unusable in the user doc > > > Key: HBASE-23755 > URL: https://issues.apache.org/jira/browse/HBASE-23755 > Project: HBase > Issue Type: Sub-task >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > The trace doesn't work at all in HBase 2.0 and above after HBASE-18601 (the > trace doesn't get picked up at the server side). We should make a note in the > user doc stating it is > # unusable > # deprecated in HBase 2.x because HTrace is in Attic. > # removed from HBase 3.0 and above. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23755) [OpenTracing] Declare HTrace is unusable in the user doc
Wei-Chiu Chuang created HBASE-23755: --- Summary: [OpenTracing] Declare HTrace is unusable in the user doc Key: HBASE-23755 URL: https://issues.apache.org/jira/browse/HBASE-23755 Project: HBase Issue Type: Improvement Reporter: Wei-Chiu Chuang Assignee: Wei-Chiu Chuang Fix For: 3.0.0, 2.3.0 The trace doesn't work at all in HBase 2.0 and above after HBASE-18601 (the trace doesn't get picked up at the server side). We should make a note in the user doc stating it is # unusable # deprecated in HBase 2.x because HTrace is in Attic. # removed from HBase 3.0 and above. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025478#comment-17025478 ] Michael Stack commented on HBASE-18095: --- Does the branch need updating (seems old when I compare to master) > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Kyle Purtell >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch > > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22120) Replace HTrace with OpenTracing
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-22120: Issue Type: New Feature (was: Bug) > Replace HTrace with OpenTracing > --- > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Assignee: Wei-Chiu Chuang >Priority: Major > > Suggested in HBASE-22115 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025471#comment-17025471 ] Michael Stack commented on HBASE-18095: --- I'd be in favor of master merge after you think it stable out on the branch. Merge discussion should be on dev list FYI. Let me see if can do a quick review of the branch... Hopefully will help. > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Kyle Purtell >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch > > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack opened a new pull request #1103: HBASE-23753 Update of errorprone generated failures
saintstack opened a new pull request #1103: HBASE-23753 Update of errorprone generated failures URL: https://github.com/apache/hbase/pull/1103 hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java Complains about mismatch in types when Compare. Implement Compare in base Interface. hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java Complains pbs never return null. hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSinkManager.java Needed redo because errorprone complains can't mock Service from guava. hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicasWithRestartScenarios.java hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController.java Unrelated...adding one-liner debug statements chasing other test failures. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23754) [DOC] HBase Web UI runs on different port
[
https://issues.apache.org/jira/browse/HBASE-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiğit Oner updated HBASE-23754:
---
Description:
Just started to work through the ref guide and started up the stand-alone
setup. The guide mentions to look up [http://localhost:16010/] for the HBase
Web UI, but there is no process listening to that port.
After checking netstat and logs I've discovered, that the UI runs on port 43105
{code}
2020-01-28 22:17:43,658 INFO [main] http.HttpServer: Jetty bound to port 43105
2020-01-28 22:17:43,658 INFO [main] mortbay.log: jetty-6.1.26
2020-01-28 22:17:43,697 INFO [main] mortbay.log: Started
SelectChannelConnector@0.0.0.0:4310
{code}
was:
Just started to work through the ref guide and started up the stand-alone
setup. The guide mentions to look up [http://localhost:16010/] for the HBase
Web UI, but there is no process listening to that port.
After checking netstat and logs I've discovered, that the UI runs on port 43105
{code:java}
2020-01-28 22:17:43,658 INFO [main] http.HttpServer: Jetty bound to port 43105
2020-01-28 22:17:43,658 INFO [main] mortbay.log: jetty-6.1.26
2020-01-28 22:17:43,697 INFO [main] mortbay.log: Started
SelectChannelConnector@0.0.0.0:4310{
code}
> [DOC] HBase Web UI runs on different port
> -
>
> Key: HBASE-23754
> URL: https://issues.apache.org/jira/browse/HBASE-23754
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Yiğit Oner
>Priority: Trivial
>
> Just started to work through the ref guide and started up the stand-alone
> setup. The guide mentions to look up [http://localhost:16010/] for the HBase
> Web UI, but there is no process listening to that port.
>
> After checking netstat and logs I've discovered, that the UI runs on port
> 43105
>
> {code}
> 2020-01-28 22:17:43,658 INFO [main] http.HttpServer: Jetty bound to port 43105
> 2020-01-28 22:17:43,658 INFO [main] mortbay.log: jetty-6.1.26
> 2020-01-28 22:17:43,697 INFO [main] mortbay.log: Started
> SelectChannelConnector@0.0.0.0:4310
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23754) [DOC] HBase Web UI runs on different port
Yiğit Oner created HBASE-23754:
--
Summary: [DOC] HBase Web UI runs on different port
Key: HBASE-23754
URL: https://issues.apache.org/jira/browse/HBASE-23754
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Yiğit Oner
Just started to work through the ref guide and started up the stand-alone
setup. The guide mentions to look up [http://localhost:16010/] for the HBase
Web UI, but there is no process listening to that port.
After checking netstat and logs I've discovered, that the UI runs on port 43105
{code:java}
2020-01-28 22:17:43,658 INFO [main] http.HttpServer: Jetty bound to port 43105
2020-01-28 22:17:43,658 INFO [main] mortbay.log: jetty-6.1.26
2020-01-28 22:17:43,697 INFO [main] mortbay.log: Started
SelectChannelConnector@0.0.0.0:4310{
code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23748) Include HBASE-21284 to branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-23748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025457#comment-17025457 ] Sakthi commented on HBASE-23748: [~apurtell] / [~liuml07] / [~elserj] can I get any of your +1s here, please? > Include HBASE-21284 to branch-2.2 > - > > Key: HBASE-23748 > URL: https://issues.apache.org/jira/browse/HBASE-23748 > Project: HBase > Issue Type: Sub-task >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Fix For: 2.2.3 > > Attachments: hbase-23748.branch-2.2.001.patch > > > HBASE-21284 was ought to be present in 2.2. But by the time the commit was > done, the branch had been cut already. Hence this Jira to track it's > inclusion. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23753) Update of errorprone generated failures
Michael Stack created HBASE-23753: - Summary: Update of errorprone generated failures Key: HBASE-23753 URL: https://issues.apache.org/jira/browse/HBASE-23753 Project: HBase Issue Type: Sub-task Reporter: Michael Stack Fix For: 3.0.0, 2.3.0 Parent issue updated hbase-thirdparty which updated errorprone. The nightly compile started failing with 'compile'/errorprone complaints. All look good. Let me fix. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23180) Add a hbck2 testing tool
[ https://issues.apache.org/jira/browse/HBASE-23180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-23180: --- Release Note: This adds a new tool that spins up hbase on hadoop minicluster and mimicks actions of hbck2 to verify it's functionalities. The tool has been designed for easy addition of more compound actions (i.e. collection of actions) to verify hbck2. deleteRegionInMeta.sh is one such example of a compound action script. Summary: Add a hbck2 testing tool (was: Create a nightly build to verify hbck2) > Add a hbck2 testing tool > > > Key: HBASE-23180 > URL: https://issues.apache.org/jira/browse/HBASE-23180 > Project: HBase > Issue Type: Task >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Labels: hbck2 > Fix For: hbase-operator-tools-1.1.0 > > > Quoting myself from the discussion thread from the dev mailing list "*How do > we test hbck2?*" - > "Planning to start working on a nightly build that can spin up a > mini-cluster, load some data into it, do some actions to bring the cluster > into an undesirable state that hbck2 can fix and then invoke the hbck2 to see > if things work well. > > Plan is to start small with one of the hbck2 commands and remaining ones can > be added incrementally. As of now I would like to start with making sure the > job uses one of the hbase versions (probably 2.1.x/2.2.x), we can discuss > about the need to run the job against all the present hbase versions/taking > in a bunch of hbase versions as input and running against them/or just a > single version. > > The job script would be located in our operator-tools repo." -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23180) Add a hbck2 testing tool
[ https://issues.apache.org/jira/browse/HBASE-23180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi resolved HBASE-23180. Resolution: Fixed > Add a hbck2 testing tool > > > Key: HBASE-23180 > URL: https://issues.apache.org/jira/browse/HBASE-23180 > Project: HBase > Issue Type: Task >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Labels: hbck2 > Fix For: hbase-operator-tools-1.1.0 > > > Quoting myself from the discussion thread from the dev mailing list "*How do > we test hbck2?*" - > "Planning to start working on a nightly build that can spin up a > mini-cluster, load some data into it, do some actions to bring the cluster > into an undesirable state that hbck2 can fix and then invoke the hbck2 to see > if things work well. > > Plan is to start small with one of the hbck2 commands and remaining ones can > be added incrementally. As of now I would like to start with making sure the > job uses one of the hbase versions (probably 2.1.x/2.2.x), we can discuss > about the need to run the job against all the present hbase versions/taking > in a bunch of hbase versions as input and running against them/or just a > single version. > > The job script would be located in our operator-tools repo." -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase-operator-tools] jatsakthi merged pull request #47: HBASE-23180 hbck2 testing tool
jatsakthi merged pull request #47: HBASE-23180 hbck2 testing tool URL: https://github.com/apache/hbase-operator-tools/pull/47 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-20952) Re-visit the WAL API
[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025449#comment-17025449
]
Zach York commented on HBASE-20952:
---
[~elserj] Is there any further work planned on this? I'm going to be touching
the WAL and want to try to refactor some of the current structures. I think a
lot of what I was going to do was orthogonal to this, but should this be kept
in mind?
> Re-visit the WAL API
>
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Josh Elser
>Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase-operator-tools] jatsakthi commented on issue #47: HBASE-23180 hbck2 testing tool
jatsakthi commented on issue #47: HBASE-23180 hbck2 testing tool URL: https://github.com/apache/hbase-operator-tools/pull/47#issuecomment-579468990 Thanks for your reviews, Peter & Sean. Let me get this in. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] busbey commented on issue #47: HBASE-23180 hbck2 testing tool
busbey commented on issue #47: HBASE-23180 hbck2 testing tool URL: https://github.com/apache/hbase-operator-tools/pull/47#issuecomment-579466139 Looks great! Thanks for getting this done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv opened a new pull request #1102: HBASE-23752: Fix remaining test failures from nightly runs
bharathv opened a new pull request #1102: HBASE-23752: Fix remaining test failures from nightly runs URL: https://github.com/apache/hbase/pull/1102 TestFromClientSideWithCoprocessor: Initialization bug causing parameterized runs to fail. TestCustomSaslAuthenticationProvider: Test config had to be fixed because it was written pre-master registry implementation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23752) Fix a couple more test failures from nightly run
[ https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-23752: - Summary: Fix a couple more test failures from nightly run (was: Fix a couple more tests from nightly run) > Fix a couple more test failures from nightly run > > > Key: HBASE-23752 > URL: https://issues.apache.org/jira/browse/HBASE-23752 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.3.0, 1.6.0, HBASE-18095 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > > After merging the patches HBASE-23731 and HBASE-23647, nightly run found a > couple of issues, > TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR > runs missed it somehow. > TestCustomSaslAuthenticationProvider - This was written pre master registry. > Has to be updated. It was never run as a part of the test suite. > > [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it > accordingly. Will submit a PR shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23752) Fix a couple more tests from nightly run
[ https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-23752: - Affects Version/s: HBASE-18095 > Fix a couple more tests from nightly run > > > Key: HBASE-23752 > URL: https://issues.apache.org/jira/browse/HBASE-23752 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.3.0, 1.6.0, HBASE-18095 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > > After merging the patches HBASE-23731 and HBASE-23647, nightly run found a > couple of issues, > TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR > runs missed it somehow. > TestCustomSaslAuthenticationProvider - This was written pre master registry. > Has to be updated. It was never run as a part of the test suite. > > [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it > accordingly. Will submit a PR shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23752) Fix a couple more tests from nightly run
Bharath Vissapragada created HBASE-23752: Summary: Fix a couple more tests from nightly run Key: HBASE-23752 URL: https://issues.apache.org/jira/browse/HBASE-23752 Project: HBase Issue Type: Bug Components: test Affects Versions: 3.0.0, 2.3.0, 1.6.0 Reporter: Bharath Vissapragada Assignee: Bharath Vissapragada After merging the patches HBASE-23731 and HBASE-23647, nightly run found a couple of issues, TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR runs missed it somehow. TestCustomSaslAuthenticationProvider - This was written pre master registry. Has to be updated. It was never run as a part of the test suite. [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it accordingly. Will submit a PR shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
