[GitHub] [hbase] JeongDaeKim commented on issue #749: HBASE-23205 Correctly update the position of WALs currently being replicated
JeongDaeKim commented on issue #749: HBASE-23205 Correctly update the position of WALs currently being replicated URL: https://github.com/apache/hbase/pull/749#issuecomment-554897893 Failed tests are unrelated to this PR, and not reproduced in my local env.  ``` Cannot delete /user/jenkins/test-data/72b993b9-2876-4d2d-be72-f1fcc47a2371. Name node is in safe mode. Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE: If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off. NamenodeHostName:localhost at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.newSafemodeException(FSNamesystem.java:1407) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1395) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2853) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:850) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:793) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:421) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2489) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23269) Hbase crashed due to two versions of regionservers when rolling upgrading
[
https://issues.apache.org/jira/browse/HBASE-23269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976335#comment-16976335
]
HBase QA commented on HBASE-23269:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red} https://github.com/apache/hbase/pull/840 does not apply to
branch-1. Rebase required? Wrong Branch? See
https://yetus.apache.org/documentation/in-progress/precommit-patchnames for
help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| GITHUB PR | https://github.com/apache/hbase/pull/840 |
| JIRA Issue | HBASE-23269 |
| Console output |
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-840/1/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
> Hbase crashed due to two versions of regionservers when rolling upgrading
> -
>
> Key: HBASE-23269
> URL: https://issues.apache.org/jira/browse/HBASE-23269
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 1.4.0, 1.4.2, 1.4.9, 1.4.10, 1.4.11
>Reporter: Jianzhen Xu
>Assignee: Jianzhen Xu
>Priority: Critical
> Attachments: 9.png, image-2019-11-07-14-49-41-253.png,
> image-2019-11-07-14-50-11-877.png, image-2019-11-07-14-51-38-858.png
>
>
> Currently, when hbase turns on the rs_group function and needs to upgrade to
> a higher version, the meta table maybe assign failed, which eventually makes
> the whole cluster unavailable and the availability drops to 0.This applies to
> all versions that introduce rs_group functionality in hbase-1.4.*. Including
> the patch of rs_group is introduced in the version below 1.4, upgrade to
> version 1.4 will also appear.
> When this happens during an upgrade:
> * When rolling upgrading regionservers, it must appear if the first rs of
> the upgrade is not in the same rs_group as the meta table.
> The phenomenon is as follows:
> !image-2019-11-07-14-50-11-877.png!
> !image-2019-11-07-14-51-38-858.png!
> The reason for this is as follows: during a rolling upgrade of the first
> regionserver node (denoted as RS1),RS1 started up and re-registered to
> zk,master triggered the operation through watcher perception in
> RegionServerTracker, and finally came to this
> method-HMaster.checkIfShouldMoveSystemRegionAsync()。
> The logic of this method is as follows:
>
> {code:java}
> // code placeholder
> public void checkIfShouldMoveSystemRegionAsync() {
> new Thread(new Runnable() {
> @Override
> public void run() {
> try {
> synchronized (checkIfShouldMoveSystemRegionLock) {
> // RS register on ZK after reports startup on master
> List regionsShouldMove = new ArrayList<>();
> for (ServerName server : getExcludedServersForSystemTable()) {
> regionsShouldMove.addAll(getCarryingSystemTables(server));
> }
> if (!regionsShouldMove.isEmpty()) {
> List plans = new ArrayList<>();
> for (HRegionInfo regionInfo : regionsShouldMove) {
> RegionPlan plan = getRegionPlan(regionInfo, true);
> if (regionInfo.isMetaRegion()) {
> // Must move meta region first.
> balance(plan);
> } else {
> plans.add(plan);
> }
> }
> for (RegionPlan plan : plans) {
> balance(plan);
> }
> }
> }
> } catch (Throwable t) {
> LOG.error(t);
> }
> }
> }).start();
> }{code}
>
> # First execute getExcludedServersForSystemTable()：Get the highest version
> value in all regionservers and return all RSs below that version value,
> labeled LowVersionRSList
> # If 1 does not return null, iterate.If there is a region with system table
> on rs, add this region to the List that needs move.If the first rs upgraded
> at this point is not in the rs_group where the system table is located, the
> region of the meta table is added to regionsShouldMove
> # Get a Regionplan for the region in regionsShouldMove,, and the parameter
> forceNewPlan is true:
> ## Gets all regionserver which version is below the highest version;
> ## Exclude regionservers from 1) for all rs online status. The result is
> that only the rs has been upgraded will in collection, marked as destServers ;
> ## Since forceNewPlan is set to true, destination server will be obtained
> through balance.randomassignmet (region, destServers). Since
[GitHub] [hbase] Apache-HBase commented on issue #840: HBASE-23269:Hbase crashed due to two versions of regionservers when r…
Apache-HBase commented on issue #840: HBASE-23269:Hbase crashed due to two versions of regionservers when r… URL: https://github.com/apache/hbase/pull/840#issuecomment-554897058 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | patch | 0m 10s | https://github.com/apache/hbase/pull/840 does not apply to branch-1. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/840 | | JIRA Issue | HBASE-23269 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-840/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] xujianzhen opened a new pull request #840: HBASE-23269:Hbase crashed due to two versions of regionservers when r…
xujianzhen opened a new pull request #840: HBASE-23269:Hbase crashed due to two versions of regionservers when r… URL: https://github.com/apache/hbase/pull/840 HBASE-23269:Hbase crashed due to two versions of regionservers when rolling upgrading This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #749: HBASE-23205 Correctly update the position of WALs currently being replicated
Apache-HBase commented on issue #749: HBASE-23205 Correctly update the position of WALs currently being replicated URL: https://github.com/apache/hbase/pull/749#issuecomment-554888674 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 27s | branch-1 passed | | +1 :green_heart: | compile | 0m 41s | branch-1 passed with JDK v1.8.0_232 | | +1 :green_heart: | compile | 0m 47s | branch-1 passed with JDK v1.7.0_242 | | +1 :green_heart: | checkstyle | 1m 49s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | branch-1 passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK v1.7.0_242 | | +0 :ok: | spotbugs | 3m 2s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 59s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 4s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed with JDK v1.7.0_242 | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | checkstyle | 1m 45s | hbase-server: The patch generated 0 new + 42 unchanged - 12 fixed = 42 total (was 54) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 9s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 30s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK v1.7.0_242 | | +1 :green_heart: | findbugs | 3m 5s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 118m 37s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 160m 1s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.procedure.TestFailedProcCleanup | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-749/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/749 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ff551250b60e 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-749/out/precommit/personality/provided.sh | | git revision | branch-1 / 8e60b0c | | Default Java | 1.7.0_242 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-749/8/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-749/8/testReport/ | | Max. process+thread count | 3986 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-749/8/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #817: HBASE-23278 Add a table-level compaction progress display on the UI (for branch-1)
Apache-HBase commented on issue #817: HBASE-23278 Add a table-level compaction progress display on the UI (for branch-1) URL: https://github.com/apache/hbase/pull/817#issuecomment-554884031 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 50s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 30s | branch-1 passed | | +1 :green_heart: | javadoc | 0m 55s | branch-1 passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 0m 56s | branch-1 passed with JDK v1.7.0_242 | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 46s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 0m 58s | the patch passed with JDK v1.7.0_242 | ||| _ Other Tests _ | | -1 :x: | unit | 155m 43s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 175m 27s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.mapreduce.TestTableSnapshotInputFormat | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/817 | | Optional Tests | dupname asflicense javac javadoc unit | | uname | Linux 43594263cd58 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-817/out/precommit/personality/provided.sh | | git revision | branch-1 / 8e60b0c | | Default Java | 1.7.0_242 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/3/testReport/ | | Max. process+thread count | 4239 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/3/console | | versions | git=1.9.1 maven=3.0.5 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554873190 @Apache9 Can you please take a look now, moved into replication peer config. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (HBASE-22978) Online slow response log
[
https://issues.apache.org/jira/browse/HBASE-22978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970182#comment-16970182
]
Viraj Jasani edited comment on HBASE-22978 at 11/18/19 6:19 AM:
Please review the pull request: [https://github.com/apache/hbase/pull/754]
was (Author: vjasani):
Please review the pull request
> Online slow response log
>
>
> Key: HBASE-22978
> URL: https://issues.apache.org/jira/browse/HBASE-22978
> Project: HBase
> Issue Type: New Feature
> Components: Admin, Operability, regionserver, shell
>Affects Versions: 3.0.0, 2.3.0, 1.5.1
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
> Attachments: Screen Shot 2019-10-19 at 2.31.59 AM.png, Screen Shot
> 2019-10-19 at 2.32.54 AM.png, Screen Shot 2019-10-19 at 2.34.11 AM.png,
> Screen Shot 2019-10-19 at 2.36.14 AM.png
>
>
> Today when an individual RPC exceeds a configurable time bound we log a
> complaint by way of the logging subsystem. These log lines look like:
> {noformat}
> 2019-08-30 22:10:36,195 WARN [,queue=15,port=60020] ipc.RpcServer -
> (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1567203007549,
> "responsesize":6819737,
> "method":"Scan",
> "param":"region { type: REGION_NAME value:
> \"tsdb,\\000\\000\\215\\f)o\\024\\302\\220\\000\\000\\000\\000\\000\\001\\000\\000\\000\\000\\000\\006\\000\\000\\000\\000\\000\\005\\000\\000",
> "processingtimems":28646,
> "client":"10.253.196.215:41116",
> "queuetimems":22453,
> "class":"HRegionServer"}
> {noformat}
> Unfortunately we often truncate the request parameters, like in the above
> example. We do this because the human readable representation is verbose, the
> rate of too slow warnings may be high, and the combination of these things
> can overwhelm the log capture system. The truncation is unfortunate because
> it eliminates much of the utility of the warnings. For example, the region
> name, the start and end keys, and the filter hierarchy are all important
> clues for debugging performance problems caused by moderate to low
> selectivity queries or queries made at a high rate.
> We can maintain an in-memory ring buffer of requests that were judged to be
> too slow in addition to the responseTooSlow logging. The in-memory
> representation can be complete and compressed. A new admin API and shell
> command can provide access to the ring buffer for online performance
> debugging. A modest sizing of the ring buffer will prevent excessive memory
> utilization for a minor performance debugging feature by limiting the total
> number of retained records. There is some chance a high rate of requests will
> cause information on other interesting requests to be overwritten before it
> can be read. This is the nature of a ring buffer and an acceptable trade off.
> The write request types do not require us to retain all information submitted
> in the request. We don't need to retain all key-values in the mutation, which
> may be too large to comfortably retain. We only need a unique set of row
> keys, or even a min/max range, and total counts.
> The consumers of this information will be debugging tools. We can afford to
> apply fast compression to ring buffer entries (if codec support is
> available), something like snappy or zstandard, and decompress on the fly
> when servicing the retrieval API request. This will minimize the impact of
> retaining more information about slow requests than we do today.
> This proposal is for retention of request information only, the same
> information provided by responseTooSlow warnings. Total size of response
> serialization, possibly also total cell or row counts, should be sufficient
> to characterize the response.
> Optionally persist new entries added to the ring buffer into one or more
> files in HDFS in a write-behind manner. If the HDFS writer blocks or falls
> behind and we are unable to persist an entry before it is overwritten, that
> is fine. Response too slow logging is best effort. If we can detect this make
> a note of it in the log file. Provide a tool for parsing, dumping, filtering,
> and pretty printing the slow logs written to HDFS. The tool and the shell can
> share and reuse some utility classes and methods for accomplishing that.
> —
> New shell commands:
> {{get_slow_responses [ ... , ] [ , \{
> } ]}}
> Retrieve, decode, and pretty print the contents of the too slow response ring
> buffer maintained by the given list of servers; or all servers in the cluster
> if no list is provided. Optionally provide a map of parameters for filtering
> as additional argument. The TABLE filter, which expects a string
[jira] [Resolved] (HBASE-23301) Generate CHANGES.md and RELEASENOTES.md for 2.1.8
[ https://issues.apache.org/jira/browse/HBASE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-23301. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to branch-2.1. Thanks [~zghao] for reviewing. > Generate CHANGES.md and RELEASENOTES.md for 2.1.8 > - > > Key: HBASE-23301 > URL: https://issues.apache.org/jira/browse/HBASE-23301 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.1.8 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 merged pull request #839: HBASE-23301 Generate CHANGES.md and RELEASENOTES.md for 2.1.8
Apache9 merged pull request #839: HBASE-23301 Generate CHANGES.md and RELEASENOTES.md for 2.1.8 URL: https://github.com/apache/hbase/pull/839 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #839: HBASE-23301 Generate CHANGES.md and RELEASENOTES.md for 2.1.8
Apache-HBase commented on issue #839: HBASE-23301 Generate CHANGES.md and RELEASENOTES.md for 2.1.8 URL: https://github.com/apache/hbase/pull/839#issuecomment-554862640 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.1 Compile Tests _ | ||| _ Patch Compile Tests _ | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 3m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-839/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/839 | | Optional Tests | dupname asflicense markdownlint | | uname | Linux 65437956ac06 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-839/out/precommit/personality/provided.sh | | git revision | branch-2.1 / 855f1b3dce | | Max. process+thread count | 46 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-839/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976297#comment-16976297
]
HBase QA commented on HBASE-23309:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
36s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
20s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
30s{color} | {color:red} hbase-server: The patch generated 3 new + 0 unchanged
- 0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}258m 56s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}320m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.TestReplicationSyncUpToolWithBulkLoadedData |
| | hadoop.hbase.replication.TestAddToSerialReplicationPeer |
| | hadoop.hbase.replication.TestReplicationSmallTestsSync |
| | hadoop.hbase.security.visibility.TestVisibilityLabelsReplication |
| | hadoop.hbase.replication.regionserver.TestSerialReplicationEndpoint |
| |
hadoop.hbase.security.visibility.TestVisibilityLabelReplicationWithExpAsString |
| | hadoop.hbase.replication.TestNamespaceReplication |
| | hadoop.hbase.client.TestAsyncReplicationAdminApiWithClusters |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1026/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23309 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976295#comment-16976295
]
HBase QA commented on HBASE-23309:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
16s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
28s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
25s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
17s{color} | {color:red} hbase-server: The patch generated 3 new + 1 unchanged
- 0 fixed = 4 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}223m 40s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}276m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.regionserver.TestReplicator |
| | hadoop.hbase.replication.TestReplicationWithTags |
| | hadoop.hbase.replication.regionserver.TestGlobalReplicationThrottler |
| |
hadoop.hbase.client.replication.TestReplicationAdminWithTwoDifferentZKClusters |
| | hadoop.hbase.client.TestAsyncReplicationAdminApiWithClusters |
| |
hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleAsyncWAL
|
| | hadoop.hbase.replication.TestReplicationKillMasterRSWithSeparateOldWALs |
| | hadoop.hbase.client.replication.TestReplicationAdminWithClusters |
| | hadoop.hbase.replication.TestSerialReplication |
| | hadoop.hbase.replication.TestReplicationSyncUpTool |
| | hadoop.hbase.replication.TestReplicationSmallTests |
| |
[jira] [Commented] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976276#comment-16976276
]
HBase QA commented on HBASE-23309:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
2s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
37s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
27s{color} | {color:red} hbase-server: The patch generated 3 new + 0 unchanged
- 0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}177m 38s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}238m 41s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.regionserver.TestSerialReplicationEndpoint |
| | hadoop.hbase.replication.TestAddToSerialReplicationPeer |
| | hadoop.hbase.replication.TestPerTableCFReplication |
| | hadoop.hbase.replication.TestReplicationSyncUpToolWithBulkLoadedData |
| |
hadoop.hbase.security.visibility.TestVisibilityLabelReplicationWithExpAsString |
| | hadoop.hbase.replication.TestReplicationSmallTestsSync |
| | hadoop.hbase.security.visibility.TestVisibilityLabelsReplication |
| | hadoop.hbase.replication.TestNamespaceReplication |
| | hadoop.hbase.client.TestAsyncReplicationAdminApiWithClusters |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.4 Server=19.03.4 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/2/artifact/out/Dockerfile
|
| GITHUB PR |
[GitHub] [hbase] Apache-HBase commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
Apache-HBase commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554846914 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 54s | master passed | | +1 :green_heart: | compile | 0m 55s | master passed | | +1 :green_heart: | checkstyle | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 5m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | | +0 :ok: | spotbugs | 4m 37s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 34s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 24s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | -1 :x: | checkstyle | 1m 27s | hbase-server: The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 0s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | | +1 :green_heart: | findbugs | 4m 25s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 177m 38s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 238m 41s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.regionserver.TestSerialReplicationEndpoint | | | hadoop.hbase.replication.TestAddToSerialReplicationPeer | | | hadoop.hbase.replication.TestPerTableCFReplication | | | hadoop.hbase.replication.TestReplicationSyncUpToolWithBulkLoadedData | | | hadoop.hbase.security.visibility.TestVisibilityLabelReplicationWithExpAsString | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.security.visibility.TestVisibilityLabelsReplication | | | hadoop.hbase.replication.TestNamespaceReplication | | | hadoop.hbase.client.TestAsyncReplicationAdminApiWithClusters | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/837 | | JIRA Issue | HBASE-23309 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 9d2ed766ee4e 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-837/out/precommit/personality/provided.sh | | git revision | master / a3efa5911d | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/2/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/2/testReport/ | | Max. process+thread count | 4158 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL
[GitHub] [hbase] Apache-HBase commented on issue #817: HBASE-23278 Add a table-level compaction progress display on the UI (for branch-1)
Apache-HBase commented on issue #817: HBASE-23278 Add a table-level compaction progress display on the UI (for branch-1) URL: https://github.com/apache/hbase/pull/817#issuecomment-554841623 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 16m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 38s | branch-1 passed | | +1 :green_heart: | javadoc | 0m 40s | branch-1 passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 0m 43s | branch-1 passed with JDK v1.7.0_242 | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 1m 11s | root in the patch failed. | | -1 :x: | whitespace | 0m 0s | The patch 4 line(s) with tabs. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 0m 42s | the patch passed with JDK v1.7.0_242 | ||| _ Other Tests _ | | -1 :x: | unit | 0m 36s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 30m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/817 | | Optional Tests | dupname asflicense javac javadoc unit | | uname | Linux 47f14321b1ee 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-817/out/precommit/personality/provided.sh | | git revision | branch-1 / 8e60b0c | | Default Java | 1.7.0_242 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/2/artifact/out/patch-mvninstall-root.txt | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/2/artifact/out/whitespace-tabs.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/2/testReport/ | | Max. process+thread count | 77 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-817/2/console | | versions | git=1.9.1 maven=3.0.5 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] JeongDaeKim commented on a change in pull request #749: HBASE-23205 Correctly update the position of WALs currently being replicated
JeongDaeKim commented on a change in pull request #749: HBASE-23205 Correctly
update the position of WALs currently being replicated
URL: https://github.com/apache/hbase/pull/749#discussion_r347191344
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReaderThread.java
##
@@ -135,59 +127,46 @@ public void run() {
try (WALEntryStream entryStream =
new WALEntryStream(logQueue, fs, conf, currentPosition, metrics)) {
while (isReaderRunning()) { // loop here to keep reusing stream while
we can
- if (!checkQuota()) {
+ if (manager.isBufferQuotaReached()) {
+Threads.sleep(sleepForRetries);
continue;
}
- WALEntryBatch batch = null;
- while (entryStream.hasNext()) {
-if (batch == null) {
- batch = new WALEntryBatch(replicationBatchCountCapacity,
entryStream.getCurrentPath());
-}
+ WALEntryBatch batch =
+ new WALEntryBatch(replicationBatchCountCapacity,
replicationBatchSizeCapacity);
+ boolean hasNext;
+ while ((hasNext = entryStream.hasNext()) == true) {
Entry entry = entryStream.next();
entry = filterEntry(entry);
if (entry != null) {
WALEdit edit = entry.getEdit();
if (edit != null && !edit.isEmpty()) {
-long entrySize = getEntrySizeIncludeBulkLoad(entry);
-long entrySizeExlucdeBulkLoad =
getEntrySizeExcludeBulkLoad(entry);
-batch.addEntry(entry);
-replicationSourceManager.setPendingShipment(true);
-updateBatchStats(batch, entry, entryStream.getPosition(),
entrySize);
-boolean totalBufferTooLarge =
acquireBufferQuota(entrySizeExlucdeBulkLoad);
+long entrySizeExcludeBulkLoad = batch.addEntry(entry);
+boolean totalBufferTooLarge =
manager.acquireBufferQuota(entrySizeExcludeBulkLoad);
// Stop if too many entries or too big
-if (totalBufferTooLarge || batch.getHeapSize() >=
replicationBatchSizeCapacity
-|| batch.getNbEntries() >= replicationBatchCountCapacity) {
+if (totalBufferTooLarge || batch.isLimitReached()) {
break;
}
}
-} else {
-
replicationSourceManager.logPositionAndCleanOldLogs(entryStream.getCurrentPath(),
-this.replicationQueueInfo.getPeerClusterZnode(),
-entryStream.getPosition(),
-this.replicationQueueInfo.isQueueRecovered(), false);
}
}
- if (batch != null && (!batch.getLastSeqIds().isEmpty() ||
batch.getNbEntries() > 0)) {
-if (LOG.isTraceEnabled()) {
- LOG.trace(String.format("Read %s WAL entries eligible for
replication",
-batch.getNbEntries()));
-}
-entryBatchQueue.put(batch);
+
+ if (LOG.isTraceEnabled()) {
Review comment:
I see. I'll retain the original message.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] JeongDaeKim commented on a change in pull request #749: HBASE-23205 Correctly update the position of WALs currently being replicated
JeongDaeKim commented on a change in pull request #749: HBASE-23205 Correctly
update the position of WALs currently being replicated
URL: https://github.com/apache/hbase/pull/749#discussion_r347190247
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReaderThread.java
##
@@ -135,59 +127,46 @@ public void run() {
try (WALEntryStream entryStream =
new WALEntryStream(logQueue, fs, conf, currentPosition, metrics)) {
while (isReaderRunning()) { // loop here to keep reusing stream while
we can
- if (!checkQuota()) {
+ if (manager.isBufferQuotaReached()) {
+Threads.sleep(sleepForRetries);
continue;
}
- WALEntryBatch batch = null;
- while (entryStream.hasNext()) {
-if (batch == null) {
- batch = new WALEntryBatch(replicationBatchCountCapacity,
entryStream.getCurrentPath());
-}
+ WALEntryBatch batch =
+ new WALEntryBatch(replicationBatchCountCapacity,
replicationBatchSizeCapacity);
+ boolean hasNext;
+ while ((hasNext = entryStream.hasNext()) == true) {
Entry entry = entryStream.next();
entry = filterEntry(entry);
if (entry != null) {
WALEdit edit = entry.getEdit();
if (edit != null && !edit.isEmpty()) {
-long entrySize = getEntrySizeIncludeBulkLoad(entry);
-long entrySizeExlucdeBulkLoad =
getEntrySizeExcludeBulkLoad(entry);
-batch.addEntry(entry);
-replicationSourceManager.setPendingShipment(true);
-updateBatchStats(batch, entry, entryStream.getPosition(),
entrySize);
-boolean totalBufferTooLarge =
acquireBufferQuota(entrySizeExlucdeBulkLoad);
+long entrySizeExcludeBulkLoad = batch.addEntry(entry);
+boolean totalBufferTooLarge =
manager.acquireBufferQuota(entrySizeExcludeBulkLoad);
// Stop if too many entries or too big
-if (totalBufferTooLarge || batch.getHeapSize() >=
replicationBatchSizeCapacity
-|| batch.getNbEntries() >= replicationBatchCountCapacity) {
+if (totalBufferTooLarge || batch.isLimitReached()) {
break;
}
}
-} else {
Review comment:
> any sub-sequent non-filterable one would need to wait for a log roll? That
could take too long for some use cases.
If `some use cases` means "no mutations come for a long time, but a batch
has entries", this case is the one of the 1) case i mentioned `a batch has
entries when eof reached`. reader would reach the eof, and log position would
be updated.
In addition, even while testing this issue with heavy writes, I observed the
reader frequently reached EOF.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #839: HBASE-23301 Generate CHANGES.md and RELEASENOTES.md for 2.1.8
Apache-HBase commented on issue #839: HBASE-23301 Generate CHANGES.md and RELEASENOTES.md for 2.1.8 URL: https://github.com/apache/hbase/pull/839#issuecomment-554831345 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.1 Compile Tests _ | ||| _ Patch Compile Tests _ | | -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 5m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-839/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/839 | | Optional Tests | dupname asflicense markdownlint | | uname | Linux 26cc4a7e3b4c 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-839/out/precommit/personality/provided.sh | | git revision | branch-2.1 / 855f1b3dce | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-839/1/artifact/out/whitespace-eol.txt | | Max. process+thread count | 46 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-839/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 opened a new pull request #839: HBASE-23301 Generate CHANGES.md and RELEASENOTES.md for 2.1.8
Apache9 opened a new pull request #839: HBASE-23301 Generate CHANGES.md and RELEASENOTES.md for 2.1.8 URL: https://github.com/apache/hbase/pull/839 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #838: Adding tests for WALCellFilter to cover all the scenarios
Apache-HBase commented on issue #838: Adding tests for WALCellFilter to cover all the scenarios URL: https://github.com/apache/hbase/pull/838#issuecomment-554829209 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 23s | master passed | | +1 :green_heart: | compile | 0m 55s | master passed | | +1 :green_heart: | checkstyle | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 4m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | | +0 :ok: | spotbugs | 4m 9s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 7s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 59s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | -1 :x: | checkstyle | 1m 16s | hbase-server: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 33s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 49s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | | +1 :green_heart: | findbugs | 4m 3s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 162m 14s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 218m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-838/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/838 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux b2cc2bdbe231 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-838/out/precommit/personality/provided.sh | | git revision | master / a3efa5911d | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-838/1/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-838/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-838/1/testReport/ | | Max. process+thread count | 4849 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-838/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-23306) Add sanity check for Admin#deleteTables and other operations
[ https://issues.apache.org/jira/browse/HBASE-23306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Kesharwani resolved HBASE-23306. -- Resolution: Not A Bug The submitted behavior is working as expected hence marking this issue resolved. > Add sanity check for Admin#deleteTables and other operations > > > Key: HBASE-23306 > URL: https://issues.apache.org/jira/browse/HBASE-23306 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Rahul Kesharwani >Priority: Minor > > h3. Description: > After taking a brief observation, It seems to be the following operation > behaves in the opposite manner of what they should be doing in case `null > argument. > *For example:* `Admin#listTables(Pattern)` If its argument is `null` then all > the tables would be returned. Also this very method is being used by rest of > the listed operation like `deleteTables()`, `enableTables()`, > `disableTables()` etc. > h3. Affected operation: > * Admin.listTables(Pattern) > * Admin.listTableNames(Pattern) > * Admin.deleteTables(Pattern) > * Admin.enableTables(Pattern) > * Admin.disableTables(Pattern) > This could be a very rare scenario but I think it should either throw an > exception or behave opposite of what current behavior is. > *If this was not intentional* then I can quickly put together a patch with > precondition check for these operations. > > Note: I tried to look for answers but I was not able to find any relevant > issue/comments hence logging this issue. Feel free to close if this is > intentional behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554828314 In ReplicationPeerConfig we have a configuration field, which can be used I think. And also, I prefer we make the filterEmptyEntry field final, which means it will be set in constructor, as I do not think we want to change it on the fly. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23306) Add sanity check for Admin#deleteTables and other operations
[ https://issues.apache.org/jira/browse/HBASE-23306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976245#comment-16976245 ] Rahul Kesharwani commented on HBASE-23306: -- Thanks, [~wchevreuil] for explanation and references. I am closing this issue now. > Add sanity check for Admin#deleteTables and other operations > > > Key: HBASE-23306 > URL: https://issues.apache.org/jira/browse/HBASE-23306 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Rahul Kesharwani >Priority: Minor > > h3. Description: > After taking a brief observation, It seems to be the following operation > behaves in the opposite manner of what they should be doing in case `null > argument. > *For example:* `Admin#listTables(Pattern)` If its argument is `null` then all > the tables would be returned. Also this very method is being used by rest of > the listed operation like `deleteTables()`, `enableTables()`, > `disableTables()` etc. > h3. Affected operation: > * Admin.listTables(Pattern) > * Admin.listTableNames(Pattern) > * Admin.deleteTables(Pattern) > * Admin.enableTables(Pattern) > * Admin.disableTables(Pattern) > This could be a very rare scenario but I think it should either throw an > exception or behave opposite of what current behavior is. > *If this was not intentional* then I can quickly put together a patch with > precondition check for these operations. > > Note: I tried to look for answers but I was not able to find any relevant > issue/comments hence logging this issue. Feel free to close if this is > intentional behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554827477 @Apache9 I see what you mean, and make sense to me. I can make a configuration in [ReplicationPeerConfig](https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerConfig.java) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554826836 A replication peer could also have configurations I think? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Reopened] (HBASE-23301) Generate CHANGES.md and RELEASENOTES.md for 2.1.8
[ https://issues.apache.org/jira/browse/HBASE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-23301: --- > Generate CHANGES.md and RELEASENOTES.md for 2.1.8 > - > > Key: HBASE-23301 > URL: https://issues.apache.org/jira/browse/HBASE-23301 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.1.8 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554825360 @Apache9 I am not sure if adding this in Configuration will help us: for example: if we have 5 replication peers, and 2 of them have custom replication endpoints and also wants to have this flag set, where will we add such configuration? Won't that make the configurations so complex unless I am missing something here? We don't want to set this flag at a region server level as a whole, this is at each ChainWalFilter level. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23301) Generate CHANGES.md and RELEASENOTES.md for 2.1.8
[ https://issues.apache.org/jira/browse/HBASE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23301: -- Fix Version/s: 2.1.8 > Generate CHANGES.md and RELEASENOTES.md for 2.1.8 > - > > Key: HBASE-23301 > URL: https://issues.apache.org/jira/browse/HBASE-23301 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.1.8 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23301) Generate CHANGES.md and RELEASENOTES.md for 2.1.8
[ https://issues.apache.org/jira/browse/HBASE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23301: -- Component/s: documentation > Generate CHANGES.md and RELEASENOTES.md for 2.1.8 > - > > Key: HBASE-23301 > URL: https://issues.apache.org/jira/browse/HBASE-23301 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23301) Generate CHANGES.md and RELEASENOTES.md for 2.1.8
[ https://issues.apache.org/jira/browse/HBASE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-23301. --- Resolution: Fixed For generating changes. > Generate CHANGES.md and RELEASENOTES.md for 2.1.8 > - > > Key: HBASE-23301 > URL: https://issues.apache.org/jira/browse/HBASE-23301 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554823872 I mean can we add a configuration for this flag? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sandeepvinayak edited a comment on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak edited a comment on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554823546 @Apache9 That's correct, if hbase wants to use this flag in all the endpoints. But this flag provides more control/flexibility to custom replication endpoints which we can hook to any replication peers. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554823546 @Apache9 That's correct, if hbase wants to use this flag in all the points. But this flag provides more control/flexibility to custom replication endpoints which we can hook to any replication peers. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554821495 This means you need to change the code of HBase right itelf and rebuild HBase right? What I say is 'Configuration'. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to
ChainWalEntryFilter to filter the whole entry if all cells get filtered
URL: https://github.com/apache/hbase/pull/837#issuecomment-554821242
For example
[getWALEntryfilter](https://github.com/Apache9/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseReplicationEndpoint.java#L70)
in BaseReplicationEndpoint can set this flag if it needs to by the following:
```java
if !filters.isEmpty() {
ChainWALEntryFilter chainFilter = new ChainWALEntryFilter(filters);
chainFilter.setFilterEmptyEntry(true);
return chainFilter;
}
```
instead of
```
return filters.isEmpty() ? null : new ChainWALEntryFilter(filters);
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] sandeepvinayak edited a comment on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak edited a comment on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554820318 @Apache9 Just like I set the flag in test ```java ChainWALEntryFilter filterAllCells = new ChainWALEntryFilter(new FilterAllCellsWALCellFilter()); // let's set the filter empty entry flag to true now for the above case filterAllCells.setFilterEmptyEntry(true); ``` Let me know if this answers your concern. Or do you prefer to set this flag in a new constructor? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554820318 @Apache9 Just like I set the flag in test ```java ChainWALEntryFilter filterAllCells = new ChainWALEntryFilter(new FilterAllCellsWALCellFilter()); // let's set the filter empty entry flag to true now for the above case filterAllCells.setFilterEmptyEntry(true); ``` Let me know if this answers question concern. Or do you prefer to set this flag in a new constructor? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #832: HBASE-23298 Refactor LogRecoveredEditsOutputSink and BoundedLogWriter…
infraio commented on a change in pull request #832: HBASE-23298 Refactor
LogRecoveredEditsOutputSink and BoundedLogWriter…
URL: https://github.com/apache/hbase/pull/832#discussion_r347177698
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BaseRecoveredEditsOutputSink.java
##
@@ -0,0 +1,281 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.wal;
+
+import static
org.apache.hadoop.hbase.wal.WALSplitUtil.getCompletedRecoveredEditsFilePath;
+import static org.apache.hadoop.hbase.wal.WALSplitUtil.getRegionSplitEditsPath;
+
+import java.io.EOFException;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.log.HBaseMarkers;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.ipc.RemoteException;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.org.apache.commons.collections4.MapUtils;
+
+@InterfaceAudience.Private
+public abstract class BaseRecoveredEditsOutputSink extends OutputSink {
+ private static final Logger LOG =
LoggerFactory.getLogger(RecoveredEditsOutputSink.class);
+ private WALSplitter walSplitter;
+ private FileSystem walFS;
+ private Configuration conf;
+ private final ConcurrentHashMap regionMaximumEditLogSeqNum =
+ new ConcurrentHashMap<>();
+
+ public BaseRecoveredEditsOutputSink(WALSplitter walSplitter,
+ WALSplitter.PipelineController controller, EntryBuffers entryBuffers,
int numWriters) {
+super(controller, entryBuffers, numWriters);
+this.walSplitter = walSplitter;
+this.walFS = walSplitter.walFS;
+this.conf = walSplitter.conf;
+ }
+
+ /**
+ * @return a path with a write for that path. caller should close.
+ */
+ protected RecoveredEditsWriter createWriter(byte[] region, WAL.Entry entry)
throws IOException {
Review comment:
Will change to pass the tablename, regionname, and the sequenceid.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work started] (HBASE-23301) Generate CHANGES.md and RELEASENOTES.md for 2.1.8
[ https://issues.apache.org/jira/browse/HBASE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-23301 started by Duo Zhang. - > Generate CHANGES.md and RELEASENOTES.md for 2.1.8 > - > > Key: HBASE-23301 > URL: https://issues.apache.org/jira/browse/HBASE-23301 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23301) Generate CHANGES.md and RELEASENOTES.md for 2.1.8
[ https://issues.apache.org/jira/browse/HBASE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-23301: - Assignee: Duo Zhang > Generate CHANGES.md and RELEASENOTES.md for 2.1.8 > - > > Key: HBASE-23301 > URL: https://issues.apache.org/jira/browse/HBASE-23301 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] infraio commented on a change in pull request #832: HBASE-23298 Refactor LogRecoveredEditsOutputSink and BoundedLogWriter…
infraio commented on a change in pull request #832: HBASE-23298 Refactor
LogRecoveredEditsOutputSink and BoundedLogWriter…
URL: https://github.com/apache/hbase/pull/832#discussion_r347177615
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedEntryBuffers.java
##
@@ -0,0 +1,40 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.wal;
+
+import org.apache.yetus.audience.InterfaceAudience;
+
+@InterfaceAudience.Private
+public class BoundedEntryBuffers extends EntryBuffers {
Review comment:
Ok.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #832: HBASE-23298 Refactor LogRecoveredEditsOutputSink and BoundedLogWriter…
infraio commented on a change in pull request #832: HBASE-23298 Refactor
LogRecoveredEditsOutputSink and BoundedLogWriter…
URL: https://github.com/apache/hbase/pull/832#discussion_r347177592
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BaseRecoveredEditsOutputSink.java
##
@@ -0,0 +1,281 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.wal;
+
+import static
org.apache.hadoop.hbase.wal.WALSplitUtil.getCompletedRecoveredEditsFilePath;
+import static org.apache.hadoop.hbase.wal.WALSplitUtil.getRegionSplitEditsPath;
+
+import java.io.EOFException;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.log.HBaseMarkers;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.ipc.RemoteException;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.org.apache.commons.collections4.MapUtils;
+
+@InterfaceAudience.Private
+public abstract class BaseRecoveredEditsOutputSink extends OutputSink {
+ private static final Logger LOG =
LoggerFactory.getLogger(RecoveredEditsOutputSink.class);
+ private WALSplitter walSplitter;
+ private FileSystem walFS;
+ private Configuration conf;
+ private final ConcurrentHashMap regionMaximumEditLogSeqNum =
+ new ConcurrentHashMap<>();
+
+ public BaseRecoveredEditsOutputSink(WALSplitter walSplitter,
+ WALSplitter.PipelineController controller, EntryBuffers entryBuffers,
int numWriters) {
+super(controller, entryBuffers, numWriters);
+this.walSplitter = walSplitter;
+this.walFS = walSplitter.walFS;
+this.conf = walSplitter.conf;
+ }
+
+ /**
+ * @return a path with a write for that path. caller should close.
+ */
+ protected RecoveredEditsWriter createWriter(byte[] region, WAL.Entry entry)
throws IOException {
+// If we already decided that this region doesn't get any output
+// we don't need to check again.
+if (blacklistedRegions.contains(region)) {
+ return null;
+}
+String tmpDirName =
walSplitter.conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
+HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
+Path regionedits = getRegionSplitEditsPath(entry,
+walSplitter.getFileBeingSplit().getPath().getName(), tmpDirName, conf);
+if (regionedits == null) {
+ blacklistedRegions.add(region);
+ return null;
+}
+FileSystem walFs = FSUtils.getWALFileSystem(conf);
+if (walFs.exists(regionedits)) {
+ LOG.warn("Found old edits file. It could be the "
+ + "result of a previous failed split attempt. Deleting " +
regionedits + ", length="
+ + walFs.getFileStatus(regionedits).getLen());
+ if (!walFs.delete(regionedits, false)) {
+LOG.warn("Failed delete of old {}", regionedits);
+ }
+}
+WALProvider.Writer w = walSplitter.createWriter(regionedits);
+LOG.info("Creating recovered edits writer path={}", regionedits);
+return new RecoveredEditsWriter(regionedits, w,
entry.getKey().getSequenceId());
+ }
+
+ protected Path closeWriter(String encodedRegionName, RecoveredEditsWriter
wap,
+ List thrown) throws IOException {
+try {
+ wap.writer.close();
+} catch (IOException ioe) {
+ LOG.error("Could not close recovered edits at {}", wap.path, ioe);
+ thrown.add(ioe);
+ return null;
+}
+LOG.info("Closed recovered edits writer path={} (wrote {} edits, skipped
{} edits in {} ms",
+wap.path, wap.editsWritten, wap.editsSkipped, wap.nanosSpent / 1000 /
1000);
+if (wap.editsWritten == 0) {
+ // just remove the empty recovered.edits file
+ if (walFS.exists(wap.path) && !walFS.delete(wap.path, false)) {
+
[GitHub] [hbase] Apache-HBase commented on issue #722: HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs
Apache-HBase commented on issue #722: HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs URL: https://github.com/apache/hbase/pull/722#issuecomment-554814327 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 23s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 40s | master passed | | +1 :green_heart: | compile | 3m 14s | master passed | | +1 :green_heart: | checkstyle | 3m 1s | master passed | | +1 :green_heart: | shadedjars | 5m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 3s | master passed | | +0 :ok: | spotbugs | 4m 28s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 10m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 27s | the patch passed | | +1 :green_heart: | compile | 3m 20s | the patch passed | | +1 :green_heart: | cc | 3m 20s | the patch passed | | +1 :green_heart: | javac | 3m 20s | the patch passed | | -1 :x: | checkstyle | 0m 41s | hbase-client: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -1 :x: | checkstyle | 1m 41s | hbase-server: The patch generated 4 new + 254 unchanged - 0 fixed = 258 total (was 254) | | -1 :x: | checkstyle | 0m 14s | hbase-hbtop: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 51s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 3m 15s | the patch passed | | +1 :green_heart: | javadoc | 2m 15s | the patch passed | | +1 :green_heart: | findbugs | 12m 49s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 41s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 30s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 0m 37s | hbase-hadoop2-compat in the patch passed. | | +1 :green_heart: | unit | 0m 26s | hbase-protocol in the patch passed. | | +1 :green_heart: | unit | 1m 50s | hbase-client in the patch passed. | | -1 :x: | unit | 33m 32s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 0m 49s | hbase-hbtop in the patch passed. | | +1 :green_heart: | asflicense | 1m 33s | The patch does not generate ASF License warnings. | | | | 136m 27s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.TestClientClusterMetrics | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-722/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/722 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux a0d501a746bb 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-722/out/precommit/personality/provided.sh | | git revision | master / a3efa5911d | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-722/7/artifact/out/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-722/7/artifact/out/diff-checkstyle-hbase-server.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-722/7/artifact/out/diff-checkstyle-hbase-hbtop.txt | | unit |
[GitHub] [hbase] Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
Apache9 commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554814262 I think usually the filter chain is set up by configuration? Then how do you set this flag? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sandeepvinayak closed pull request #838: Adding tests for WALCellFilter to cover all the scenarios
sandeepvinayak closed pull request #838: Adding tests for WALCellFilter to cover all the scenarios URL: https://github.com/apache/hbase/pull/838 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-23310) Add tests for WALCellFilter
[ https://issues.apache.org/jira/browse/HBASE-23310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal resolved HBASE-23310. - Resolution: Duplicate Added as a part of HBASE-23309 > Add tests for WALCellFilter > > > Key: HBASE-23310 > URL: https://issues.apache.org/jira/browse/HBASE-23310 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 3.0.0 >Reporter: Sandeep Pal >Assignee: Sandeep Pal >Priority: Minor > > There is no test coverage to test the WALCellFilter in the replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Attachment: HBASE-23309.branch-2.patch
> Add support in ChainWalEntryFilter to filter Entry if all cells get filtered
> through WalCellFilter
> --
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
> Attachments: HBASE-23309.branch-1.patch, HBASE-23309.branch-2.patch,
> HBASE-23309.patch
>
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should add an
> option in chainwalentryfilter to filter the entry as well.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
> Customer replication endpoints may use this flag.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Attachment: HBASE-23309.branch-1.patch
> Add support in ChainWalEntryFilter to filter Entry if all cells get filtered
> through WalCellFilter
> --
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
> Attachments: HBASE-23309.branch-1.patch, HBASE-23309.patch
>
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should add an
> option in chainwalentryfilter to filter the entry as well.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
> Customer replication endpoints may use this flag.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered
Apache-HBase commented on issue #837: HBASE-23309: Adding the flexibility to ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837#issuecomment-554806538 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 5s | master passed | | +1 :green_heart: | compile | 1m 15s | master passed | | +1 :green_heart: | checkstyle | 1m 49s | master passed | | +1 :green_heart: | shadedjars | 6m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | master passed | | +0 :ok: | spotbugs | 5m 56s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 52s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 48s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed | | +1 :green_heart: | javac | 1m 10s | the patch passed | | -1 :x: | checkstyle | 1m 38s | hbase-server: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 19m 49s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed | | +1 :green_heart: | findbugs | 6m 2s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 33m 57s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 108m 22s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestReplicationWALEntryFilters | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/837 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d66d55e1934c 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-837/out/precommit/personality/provided.sh | | git revision | master / a3efa5911d | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/1/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/1/testReport/ | | Max. process+thread count | 671 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-837/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Attachment: HBASE-23309.patch
Status: Patch Available (was: In Progress)
> Add support in ChainWalEntryFilter to filter Entry if all cells get filtered
> through WalCellFilter
> --
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.3.6, 3.0.0, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
> Attachments: HBASE-23309.patch
>
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should add an
> option in chainwalentryfilter to filter the entry as well.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
> Customer replication endpoints may use this flag.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Description:
ChainWalEntryFilter applies the filter on entry followed by filter on cells.
If filter on cells remove all the cells from the entry, we should add an
option in chainwalentryfilter to filter the entry as well.
Here is the snippet for ChainWalEntryFilter filter. After filterCells we should
check if there is any cells remaining in the entry.
{code:java}
@Override
public Entry filter(Entry entry) {
for (WALEntryFilter filter : filters) {
if (entry == null) {
return null;
}
entry = filter.filter(entry);
}
filterCells(entry);
return entry;
}{code}
Customer replication endpoints may use this flag.
was:
ChainWalEntryFilter applies the filter on entry followed by filter on cells.
If filter on cells remove all the cells from the entry, we should add an
option in chainwalentryfilter to filter the entry as well.
Here is the snippet for ChainWalEntryFilter filter. After filterCells we should
check if there is any cells remaining in the entry.
{code:java}
@Override
public Entry filter(Entry entry) {
for (WALEntryFilter filter : filters) {
if (entry == null) {
return null;
}
entry = filter.filter(entry);
}
filterCells(entry);
return entry;
}{code}
> Add support in ChainWalEntryFilter to filter Entry if all cells get filtered
> through WalCellFilter
> --
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should add an
> option in chainwalentryfilter to filter the entry as well.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
> Customer replication endpoints may use this flag.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Summary: Add support in ChainWalEntryFilter to filter Entry if all cells
get filtered through WalCellFilter (was: Entries get replicated even though
all cells get filtered through WalCellFilter)
> Add support in ChainWalEntryFilter to filter Entry if all cells get filtered
> through WalCellFilter
> --
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should add an
> option in chainwalentryfilter to filter the entry as well.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23309) Entries get replicated even though all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Description:
ChainWalEntryFilter applies the filter on entry followed by filter on cells.
If filter on cells remove all the cells from the entry, we should add an
option in chainwalentryfilter to filter the entry as well.
Here is the snippet for ChainWalEntryFilter filter. After filterCells we should
check if there is any cells remaining in the entry.
{code:java}
@Override
public Entry filter(Entry entry) {
for (WALEntryFilter filter : filters) {
if (entry == null) {
return null;
}
entry = filter.filter(entry);
}
filterCells(entry);
return entry;
}{code}
was:
ChainWalEntryFilter applies the filter on entry followed by filter on cells.
If filter on cells remove all the cells from the entry, we should't replicate
the entry.
Here is the snippet for ChainWalEntryFilter filter. After filterCells we should
check if there is any cells remaining in the entry.
{code:java}
@Override
public Entry filter(Entry entry) {
for (WALEntryFilter filter : filters) {
if (entry == null) {
return null;
}
entry = filter.filter(entry);
}
filterCells(entry);
return entry;
}{code}
> Entries get replicated even though all cells get filtered through
> WalCellFilter
> ---
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should add an
> option in chainwalentryfilter to filter the entry as well.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23309) Entries get replicated even though all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976204#comment-16976204
]
Sandeep Pal commented on HBASE-23309:
-
[https://github.com/apache/hbase/pull/837|https://github.com/apache/hbase/pull/837/files]
> Entries get replicated even though all cells get filtered through
> WalCellFilter
> ---
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should't
> replicate the entry.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23309) Entries get replicated even though all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Issue Type: Improvement (was: Bug)
> Entries get replicated even though all cells get filtered through
> WalCellFilter
> ---
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should't
> replicate the entry.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HBASE-23309) Entries get replicated even though all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-23309 started by Sandeep Pal.
---
> Entries get replicated even though all cells get filtered through
> WalCellFilter
> ---
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should't
> replicate the entry.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23310) Add tests for WALCellFilter
[ https://issues.apache.org/jira/browse/HBASE-23310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976195#comment-16976195 ] Sandeep Pal commented on HBASE-23310: - [https://github.com/apache/hbase/pull/838] > Add tests for WALCellFilter > > > Key: HBASE-23310 > URL: https://issues.apache.org/jira/browse/HBASE-23310 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 3.0.0 >Reporter: Sandeep Pal >Assignee: Sandeep Pal >Priority: Minor > > There is no test coverage to test the WALCellFilter in the replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] sandeepvinayak opened a new pull request #838: Adding tests for WALCellFilter to cover all the scenarios
sandeepvinayak opened a new pull request #838: Adding tests for WALCellFilter to cover all the scenarios URL: https://github.com/apache/hbase/pull/838 Scenarios to cover the WALCellFilter: 1. Filter some cells walcellfiter 2. Filter all cells 3. Filter some cells walcellfilter but nothing get filtered This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ankitsinghal commented on issue #722: HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs
ankitsinghal commented on issue #722: HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs URL: https://github.com/apache/hbase/pull/722#issuecomment-554798490 @brfrn169 , have taken the review comments in the latest push. Let me know how it looks? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23310) Add tests for WALCellFilter
Sandeep Pal created HBASE-23310: --- Summary: Add tests for WALCellFilter Key: HBASE-23310 URL: https://issues.apache.org/jira/browse/HBASE-23310 Project: HBase Issue Type: Improvement Components: Replication Affects Versions: 3.0.0 Reporter: Sandeep Pal Assignee: Sandeep Pal There is no test coverage to test the WALCellFilter in the replication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23309) Entries get replicated even though all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Description:
ChainWalEntryFilter applies the filter on entry followed by filter on cells.
If filter on cells remove all the cells from the entry, we should't replicate
the entry.
Here is the snippet for ChainWalEntryFilter filter. After filterCells we should
check if there is any cells remaining in the entry.
{code:java}
@Override
public Entry filter(Entry entry) {
for (WALEntryFilter filter : filters) {
if (entry == null) {
return null;
}
entry = filter.filter(entry);
}
filterCells(entry);
return entry;
}{code}
was:
ChainWalEntryFilter applies the filter on entry followed by filter on cells.
If filter on cells remove all the cells from the entry, we should't replicate
the entry.
Here is the snippet for ChainWalEntryFilter filter. After filterCells we should
check if there is any cells remaining in the entry.
{code:java}
@Override
public Entry filter(Entry entry) {
for (WALEntryFilter filter : filters) {
if (entry == null) {
return null;
}
entry = filter.filter(entry);
}
filterCells(entry);
return entry;
}{code}
Also, there are not tests for WAlCellFilter, adding test for that as well.
> Entries get replicated even though all cells get filtered through
> WalCellFilter
> ---
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should't
> replicate the entry.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] sandeepvinayak opened a new pull request #837: Fixing the ChainWalEntryFilter to filter the whole entry if all cells get filtered
sandeepvinayak opened a new pull request #837: Fixing the ChainWalEntryFilter to filter the whole entry if all cells get filtered URL: https://github.com/apache/hbase/pull/837 Fixing the ChainWalEntryFilter to filter the whole entry if all cell get filtered. Also, adding test cases for WALCellFilter. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23309) Entries get replicated even though all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Description:
ChainWalEntryFilter applies the filter on entry followed by filter on cells.
If filter on cells remove all the cells from the entry, we should't replicate
the entry.
Here is the snippet for ChainWalEntryFilter filter. After filterCells we should
check if there is any cells remaining in the entry.
{code:java}
@Override
public Entry filter(Entry entry) {
for (WALEntryFilter filter : filters) {
if (entry == null) {
return null;
}
entry = filter.filter(entry);
}
filterCells(entry);
return entry;
}{code}
Also, there are not tests for WAlCellFilter, adding test for that as well.
was:
ChainWalEntryFilter applies the filter on entry followed by filter on cells.
If filter on cells remove all the cells from the entry, we should't replicate
the entry.
Here is the snippet for ChainWalEntryFilter filter. After filterCells we should
check if there is any cells remaining in the entry.
{code:java}
@Override
public Entry filter(Entry entry) {
for (WALEntryFilter filter : filters) {
if (entry == null) {
return null;
}
entry = filter.filter(entry);
}
filterCells(entry);
return entry;
}{code}
> Entries get replicated even though all cells get filtered through
> WalCellFilter
> ---
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should't
> replicate the entry.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
> Also, there are not tests for WAlCellFilter, adding test for that as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23308) Review of NullPointerExceptions
[
https://issues.apache.org/jira/browse/HBASE-23308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976173#comment-16976173
]
HBase QA commented on HBASE-23308:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
12s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} The patch passed checkstyle in hbase-common {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} hbase-server: The patch generated 0 new + 247
unchanged - 9 fixed = 247 total (was 256) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch passed checkstyle in hbase-backup {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 33s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
8s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
54s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}162m
[GitHub] [hbase] Apache-HBase commented on issue #836: HBASE-23308: Review of NullPointerExceptions
Apache-HBase commented on issue #836: HBASE-23308: Review of NullPointerExceptions URL: https://github.com/apache/hbase/pull/836#issuecomment-554791271 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 16s | master passed | | +1 :green_heart: | compile | 2m 9s | master passed | | +1 :green_heart: | checkstyle | 2m 46s | master passed | | +1 :green_heart: | shadedjars | 4m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 40s | master passed | | +0 :ok: | spotbugs | 4m 12s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 49s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 56s | the patch passed | | +1 :green_heart: | compile | 2m 12s | the patch passed | | +1 :green_heart: | javac | 2m 12s | the patch passed | | +1 :green_heart: | checkstyle | 0m 27s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 41s | The patch passed checkstyle in hbase-client | | +1 :green_heart: | checkstyle | 1m 24s | hbase-server: The patch generated 0 new + 247 unchanged - 9 fixed = 247 total (was 256) | | +1 :green_heart: | checkstyle | 0m 16s | The patch passed checkstyle in hbase-backup | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 33s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 38s | the patch passed | | +1 :green_heart: | findbugs | 7m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 8s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 54s | hbase-client in the patch passed. | | -1 :x: | unit | 162m 43s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 14m 19s | hbase-backup in the patch passed. | | +1 :green_heart: | asflicense | 2m 20s | The patch does not generate ASF License warnings. | | | | 255m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-836/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/836 | | JIRA Issue | HBASE-23308 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux f4e3a2c117c3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-836/out/precommit/personality/provided.sh | | git revision | master / a3efa5911d | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-836/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-836/2/testReport/ | | Max. process+thread count | 4643 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-server hbase-backup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-836/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries
[jira] [Created] (HBASE-23309) Entries get replicated even though all cells get filtered through WalCellFilter
Sandeep Pal created HBASE-23309:
---
Summary: Entries get replicated even though all cells get filtered
through WalCellFilter
Key: HBASE-23309
URL: https://issues.apache.org/jira/browse/HBASE-23309
Project: HBase
Issue Type: Bug
Affects Versions: 1.3.6, 3.0.0, 2.3.3
Reporter: Sandeep Pal
Assignee: Sandeep Pal
ChainWalEntryFilter applies the filter on entry followed by filter on cells.
If filter on cells remove all the cells from the entry, we should't replicate
the entry.
Here is the snippet for ChainWalEntryFilter filter. After filterCells we should
check if there is any cells remaining in the entry.
{code:java}
@Override
public Entry filter(Entry entry) {
for (WALEntryFilter filter : filters) {
if (entry == null) {
return null;
}
entry = filter.filter(entry);
}
filterCells(entry);
return entry;
}{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] belugabehr commented on issue #836: HBASE-23308: Review of NullPointerExceptions
belugabehr commented on issue #836: HBASE-23308: Review of NullPointerExceptions URL: https://github.com/apache/hbase/pull/836#issuecomment-554781113 Please node that I changed a couple of NPE to be IllegalStateException (Guava Preconditions#checkState) as these are more appropriate exceptions given the circumstances. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] belugabehr edited a comment on issue #836: HBASE-23308: Review of NullPointerExceptions
belugabehr edited a comment on issue #836: HBASE-23308: Review of NullPointerExceptions URL: https://github.com/apache/hbase/pull/836#issuecomment-554781113 Please note that I changed a couple of NPE to be IllegalStateException (Guava Preconditions#checkState) as these are more appropriate exceptions given the circumstances. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] karthikhw commented on a change in pull request #834: HBASE-23237 Negative sign in requestsPerSecond
karthikhw commented on a change in pull request #834: HBASE-23237 Negative sign
in requestsPerSecond
URL: https://github.com/apache/hbase/pull/834#discussion_r347149839
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRequestsPerSecondMetric.java
##
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+
+import java.io.IOException;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.testclassification.RegionServerTests;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+/**
+ * Validate no negative value in requestsPerSecond metric.
+ */
+@Category({ RegionServerTests.class, MediumTests.class })
+public class TestRequestsPerSecondMetric {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestRequestsPerSecondMetric.class);
+
+ private static final HBaseTestingUtility UTIL = new HBaseTestingUtility();
+ private static final long METRICS_PERIOD = 2000L;
+ private static Configuration conf;
+
+
+ @BeforeClass
+ public static void setup() throws Exception {
+conf = UTIL.getConfiguration();
+conf.setLong(HConstants.REGIONSERVER_METRICS_PERIOD, METRICS_PERIOD);
+UTIL.startMiniCluster(1);
+ }
+
+ @AfterClass
+ public static void teardown() throws Exception {
+UTIL.shutdownMiniCluster();
+ }
+
+ @Test
+ public void testNoNegativeSignAtRequestsPerSecond() throws IOException,
InterruptedException {
+final TableName TABLENAME = TableName.valueOf("t");
+final String FAMILY = "f";
+Admin admin = UTIL.getAdmin();
+UTIL.createMultiRegionTable(TABLENAME, FAMILY.getBytes(),25);
+Table table = admin.getConnection().getTable(TABLENAME);
+ServerName serverName = admin.getRegionServers().iterator().next();
+HRegionServer regionServer =
UTIL.getMiniHBaseCluster().getRegionServer(serverName);
+MetricsRegionServerWrapperImpl metricsWrapper =
+new MetricsRegionServerWrapperImpl(regionServer);
+MetricsRegionServerWrapperImpl.RegionServerMetricsWrapperRunnable
metricsServer
+= metricsWrapper.new RegionServerMetricsWrapperRunnable();
+metricsServer.run();
+UTIL.loadRandomRows(table, FAMILY.getBytes(), 1, 2000);
+Thread.sleep(METRICS_PERIOD);
+metricsServer.run();
+admin.disableTable(TABLENAME);
+Thread.sleep(METRICS_PERIOD);
+metricsServer.run();
+Assert.assertTrue(metricsWrapper.getRequestsPerSecond() > -1);
+ }
+}
Review comment:
Thank you very much @saintstack. I am disabling table(the equivalent of
region transition in my case) so all of his regions will be closed and removed
from HRegionServer#onlineregion map. But metricsystem still have these closed
regions metric and it never removed from metricsystem. so if we compute new
metric then it will be a negative value. This can be re-produced running this
test case without patch.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] karthikhw commented on a change in pull request #834: HBASE-23237 Negative sign in requestsPerSecond
karthikhw commented on a change in pull request #834: HBASE-23237 Negative sign
in requestsPerSecond
URL: https://github.com/apache/hbase/pull/834#discussion_r347149839
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRequestsPerSecondMetric.java
##
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+
+import java.io.IOException;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.testclassification.RegionServerTests;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+/**
+ * Validate no negative value in requestsPerSecond metric.
+ */
+@Category({ RegionServerTests.class, MediumTests.class })
+public class TestRequestsPerSecondMetric {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestRequestsPerSecondMetric.class);
+
+ private static final HBaseTestingUtility UTIL = new HBaseTestingUtility();
+ private static final long METRICS_PERIOD = 2000L;
+ private static Configuration conf;
+
+
+ @BeforeClass
+ public static void setup() throws Exception {
+conf = UTIL.getConfiguration();
+conf.setLong(HConstants.REGIONSERVER_METRICS_PERIOD, METRICS_PERIOD);
+UTIL.startMiniCluster(1);
+ }
+
+ @AfterClass
+ public static void teardown() throws Exception {
+UTIL.shutdownMiniCluster();
+ }
+
+ @Test
+ public void testNoNegativeSignAtRequestsPerSecond() throws IOException,
InterruptedException {
+final TableName TABLENAME = TableName.valueOf("t");
+final String FAMILY = "f";
+Admin admin = UTIL.getAdmin();
+UTIL.createMultiRegionTable(TABLENAME, FAMILY.getBytes(),25);
+Table table = admin.getConnection().getTable(TABLENAME);
+ServerName serverName = admin.getRegionServers().iterator().next();
+HRegionServer regionServer =
UTIL.getMiniHBaseCluster().getRegionServer(serverName);
+MetricsRegionServerWrapperImpl metricsWrapper =
+new MetricsRegionServerWrapperImpl(regionServer);
+MetricsRegionServerWrapperImpl.RegionServerMetricsWrapperRunnable
metricsServer
+= metricsWrapper.new RegionServerMetricsWrapperRunnable();
+metricsServer.run();
+UTIL.loadRandomRows(table, FAMILY.getBytes(), 1, 2000);
+Thread.sleep(METRICS_PERIOD);
+metricsServer.run();
+admin.disableTable(TABLENAME);
+Thread.sleep(METRICS_PERIOD);
+metricsServer.run();
+Assert.assertTrue(metricsWrapper.getRequestsPerSecond() > -1);
+ }
+}
Review comment:
Thank you very much @saintstack. I am disabling table(the equivalent of
region transition in my case) so all of his regions will be closed and removed
from HRegionServer#onlineregion map. But metricsystem still have these closed
regions metric and it never removed from metricsystem. so if we compute new
metric then it will be a negative value. This can be re-produced running this
test case without patch.
Hope, I am answering correct answer @saintstack Sorry If I am wrong :). I
am still fine if you would like to change the test case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] belugabehr commented on a change in pull request #836: HBASE-23308: Review of NullPointerExceptions
belugabehr commented on a change in pull request #836: HBASE-23308: Review of
NullPointerExceptions
URL: https://github.com/apache/hbase/pull/836#discussion_r347149074
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -7322,13 +7323,14 @@ public static HRegion openHRegion(final Configuration
conf, final FileSystem fs,
* @param rsServices An interface we can request flushes against.
* @param reporter An interface we can report progress against.
* @return new HRegion
+ * @throws NullPointerException if {@code info} is {@code null}
*/
public static HRegion openHRegion(final Configuration conf, final FileSystem
fs,
final Path rootDir, final Path tableDir, final RegionInfo info, final
TableDescriptor htd,
final WAL wal, final RegionServerServices rsServices,
final CancelableProgressable reporter)
throws IOException {
-if (info == null) throw new NullPointerException("Passed region info is
null");
+Objects.requireNonNull(info, "RegionInfo cannot be null");
Review comment:
As long as there's a helpful JavaDoc, that should be good enough for now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] belugabehr commented on issue #836: HBASE-23308: Review of NullPointerExceptions
belugabehr commented on issue #836: HBASE-23308: Review of NullPointerExceptions URL: https://github.com/apache/hbase/pull/836#issuecomment-554768282 @saintstack Hey, sorry, didn't see that you reviewed this code already so I tried to put in a force-push to keep things cleaner. Ended up messing up the review. Good way to introduce myself back. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] belugabehr commented on a change in pull request #836: HBASE-23308: Review of NullPointerExceptions
belugabehr commented on a change in pull request #836: HBASE-23308: Review of
NullPointerExceptions
URL: https://github.com/apache/hbase/pull/836#discussion_r347148986
##
File path:
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupDriver.java
##
@@ -187,10 +188,7 @@ public static void main(String[] args) throws Exception {
@Override
public int run(String[] args) throws IOException {
-if (conf == null) {
- LOG.error("Tool configuration is not initialized");
- throw new NullPointerException("conf");
-}
+Objects.requireNonNull(conf, "Tool configuration is not initialized");
Review comment:
@saintstack Less code. I also like the JavaDoc for the method. I copy &
paste its message to use the same helpful formatting.
Trying to introduce it so that others may stumble upon and learn. Best used
in this way:
```
public MyClass(String val) {
this.val = Objects.requireNonNull(val);
}
```
More concise than the alternative:
```
public MyClass(String val) {
if (val == null) {
throw new NullPointerException();
}
this.val = val;
}
```
But if it's going to be used one way, might as well be used everywhere.
Pretty versatile that way.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-19450) Add log about average execution time for ScheduledChore
[
https://issues.apache.org/jira/browse/HBASE-19450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976057#comment-16976057
]
Hudson commented on HBASE-19450:
Results for branch master
[build #1539 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1539/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1539//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1539//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1539//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add log about average execution time for ScheduledChore
> ---
>
> Key: HBASE-19450
> URL: https://issues.apache.org/jira/browse/HBASE-19450
> Project: HBase
> Issue Type: Improvement
> Components: Operability
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.1.8, 2.2.3
>
> Attachments: HBASE-19450.addendum1.0.patch,
> HBASE-19450.addendum1.1.patch, HBASE-19450.master.001.patch,
> HBASE-19450.master.002.patch, HBASE-19450.master.003.patch,
> HBASE-19450.master.004.patch
>
>
> So far, there is no information about the exact execution time for a chore,
> we can provide log information about it. It also brings other benefits, like
> discovering inefficient chores which show rooms for improvement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23289) Update links to Hadoop wiki in code and book
[
https://issues.apache.org/jira/browse/HBASE-23289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976032#comment-16976032
]
HBase QA commented on HBASE-23289:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m
29s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
30s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
31s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 13m
36s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 15m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
58s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 47s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 16m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} |
[GitHub] [hbase] wchevreuil commented on a change in pull request #749: HBASE-23205 Correctly update the position of WALs currently being replicated
wchevreuil commented on a change in pull request #749: HBASE-23205 Correctly
update the position of WALs currently being replicated
URL: https://github.com/apache/hbase/pull/749#discussion_r347131612
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReaderThread.java
##
@@ -135,59 +127,46 @@ public void run() {
try (WALEntryStream entryStream =
new WALEntryStream(logQueue, fs, conf, currentPosition, metrics)) {
while (isReaderRunning()) { // loop here to keep reusing stream while
we can
- if (!checkQuota()) {
+ if (manager.isBufferQuotaReached()) {
+Threads.sleep(sleepForRetries);
continue;
}
- WALEntryBatch batch = null;
- while (entryStream.hasNext()) {
-if (batch == null) {
- batch = new WALEntryBatch(replicationBatchCountCapacity,
entryStream.getCurrentPath());
-}
+ WALEntryBatch batch =
+ new WALEntryBatch(replicationBatchCountCapacity,
replicationBatchSizeCapacity);
+ boolean hasNext;
+ while ((hasNext = entryStream.hasNext()) == true) {
Entry entry = entryStream.next();
entry = filterEntry(entry);
if (entry != null) {
WALEdit edit = entry.getEdit();
if (edit != null && !edit.isEmpty()) {
-long entrySize = getEntrySizeIncludeBulkLoad(entry);
-long entrySizeExlucdeBulkLoad =
getEntrySizeExcludeBulkLoad(entry);
-batch.addEntry(entry);
-replicationSourceManager.setPendingShipment(true);
-updateBatchStats(batch, entry, entryStream.getPosition(),
entrySize);
-boolean totalBufferTooLarge =
acquireBufferQuota(entrySizeExlucdeBulkLoad);
+long entrySizeExcludeBulkLoad = batch.addEntry(entry);
+boolean totalBufferTooLarge =
manager.acquireBufferQuota(entrySizeExcludeBulkLoad);
// Stop if too many entries or too big
-if (totalBufferTooLarge || batch.getHeapSize() >=
replicationBatchSizeCapacity
-|| batch.getNbEntries() >= replicationBatchCountCapacity) {
+if (totalBufferTooLarge || batch.isLimitReached()) {
break;
}
}
-} else {
Review comment:
So if a filtered edit came, any sub-sequent non-filterable one would need to
wait for a log roll? That could take too long for some use cases.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #749: HBASE-23205 Correctly update the position of WALs currently being replicated
wchevreuil commented on a change in pull request #749: HBASE-23205 Correctly
update the position of WALs currently being replicated
URL: https://github.com/apache/hbase/pull/749#discussion_r347130863
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReaderThread.java
##
@@ -135,59 +127,46 @@ public void run() {
try (WALEntryStream entryStream =
new WALEntryStream(logQueue, fs, conf, currentPosition, metrics)) {
while (isReaderRunning()) { // loop here to keep reusing stream while
we can
- if (!checkQuota()) {
+ if (manager.isBufferQuotaReached()) {
+Threads.sleep(sleepForRetries);
continue;
}
- WALEntryBatch batch = null;
- while (entryStream.hasNext()) {
-if (batch == null) {
- batch = new WALEntryBatch(replicationBatchCountCapacity,
entryStream.getCurrentPath());
-}
+ WALEntryBatch batch =
+ new WALEntryBatch(replicationBatchCountCapacity,
replicationBatchSizeCapacity);
+ boolean hasNext;
+ while ((hasNext = entryStream.hasNext()) == true) {
Entry entry = entryStream.next();
entry = filterEntry(entry);
if (entry != null) {
WALEdit edit = entry.getEdit();
if (edit != null && !edit.isEmpty()) {
-long entrySize = getEntrySizeIncludeBulkLoad(entry);
-long entrySizeExlucdeBulkLoad =
getEntrySizeExcludeBulkLoad(entry);
-batch.addEntry(entry);
-replicationSourceManager.setPendingShipment(true);
-updateBatchStats(batch, entry, entryStream.getPosition(),
entrySize);
-boolean totalBufferTooLarge =
acquireBufferQuota(entrySizeExlucdeBulkLoad);
+long entrySizeExcludeBulkLoad = batch.addEntry(entry);
+boolean totalBufferTooLarge =
manager.acquireBufferQuota(entrySizeExcludeBulkLoad);
// Stop if too many entries or too big
-if (totalBufferTooLarge || batch.getHeapSize() >=
replicationBatchSizeCapacity
-|| batch.getNbEntries() >= replicationBatchCountCapacity) {
+if (totalBufferTooLarge || batch.isLimitReached()) {
break;
}
}
-} else {
-
replicationSourceManager.logPositionAndCleanOldLogs(entryStream.getCurrentPath(),
-this.replicationQueueInfo.getPeerClusterZnode(),
-entryStream.getPosition(),
-this.replicationQueueInfo.isQueueRecovered(), false);
}
}
- if (batch != null && (!batch.getLastSeqIds().isEmpty() ||
batch.getNbEntries() > 0)) {
-if (LOG.isTraceEnabled()) {
- LOG.trace(String.format("Read %s WAL entries eligible for
replication",
-batch.getNbEntries()));
-}
-entryBatchQueue.put(batch);
+
+ if (LOG.isTraceEnabled()) {
Review comment:
This adds extra complexity for troubleshooting and is even misleading, as
the message says the entries are eligible for replication, even if they got
filtered, which means just the opposite.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #749: HBASE-23205 Correctly update the position of WALs currently being replicated
wchevreuil commented on a change in pull request #749: HBASE-23205 Correctly
update the position of WALs currently being replicated
URL: https://github.com/apache/hbase/pull/749#discussion_r347130863
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReaderThread.java
##
@@ -135,59 +127,46 @@ public void run() {
try (WALEntryStream entryStream =
new WALEntryStream(logQueue, fs, conf, currentPosition, metrics)) {
while (isReaderRunning()) { // loop here to keep reusing stream while
we can
- if (!checkQuota()) {
+ if (manager.isBufferQuotaReached()) {
+Threads.sleep(sleepForRetries);
continue;
}
- WALEntryBatch batch = null;
- while (entryStream.hasNext()) {
-if (batch == null) {
- batch = new WALEntryBatch(replicationBatchCountCapacity,
entryStream.getCurrentPath());
-}
+ WALEntryBatch batch =
+ new WALEntryBatch(replicationBatchCountCapacity,
replicationBatchSizeCapacity);
+ boolean hasNext;
+ while ((hasNext = entryStream.hasNext()) == true) {
Entry entry = entryStream.next();
entry = filterEntry(entry);
if (entry != null) {
WALEdit edit = entry.getEdit();
if (edit != null && !edit.isEmpty()) {
-long entrySize = getEntrySizeIncludeBulkLoad(entry);
-long entrySizeExlucdeBulkLoad =
getEntrySizeExcludeBulkLoad(entry);
-batch.addEntry(entry);
-replicationSourceManager.setPendingShipment(true);
-updateBatchStats(batch, entry, entryStream.getPosition(),
entrySize);
-boolean totalBufferTooLarge =
acquireBufferQuota(entrySizeExlucdeBulkLoad);
+long entrySizeExcludeBulkLoad = batch.addEntry(entry);
+boolean totalBufferTooLarge =
manager.acquireBufferQuota(entrySizeExcludeBulkLoad);
// Stop if too many entries or too big
-if (totalBufferTooLarge || batch.getHeapSize() >=
replicationBatchSizeCapacity
-|| batch.getNbEntries() >= replicationBatchCountCapacity) {
+if (totalBufferTooLarge || batch.isLimitReached()) {
break;
}
}
-} else {
-
replicationSourceManager.logPositionAndCleanOldLogs(entryStream.getCurrentPath(),
-this.replicationQueueInfo.getPeerClusterZnode(),
-entryStream.getPosition(),
-this.replicationQueueInfo.isQueueRecovered(), false);
}
}
- if (batch != null && (!batch.getLastSeqIds().isEmpty() ||
batch.getNbEntries() > 0)) {
-if (LOG.isTraceEnabled()) {
- LOG.trace(String.format("Read %s WAL entries eligible for
replication",
-batch.getNbEntries()));
-}
-entryBatchQueue.put(batch);
+
+ if (LOG.isTraceEnabled()) {
Review comment:
This can lead to confusion, adds extra complexity for troubleshooting and is
even misleading, as the message says the entries are eligible for replication,
even if they got filtered, which means just the opposite.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23306) Add sanity check for Admin#deleteTables and other operations
[
https://issues.apache.org/jira/browse/HBASE-23306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975968#comment-16975968
]
Wellington Chevreuil commented on HBASE-23306:
--
Apart from _ listTableNames_, all these mentioned methods were deprecated and
already been removed from *Admin* for the next major release. The *Admin*
interface is mainly a client facade for *HMaster* admin methods, and if we look
at [_HMaster.listTableNames_ javadoc
description|https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/master/HMaster.html#listTableNames-java.lang.String-java.lang.String-boolean-],
it's pretty clear about the parameter behaviour:
{noformat}
public List listTableNames(String namespace,
String regex,
boolean includeSysTables)
throws IOException
Returns the list of table names that match the specified request
Parameters:
regex - The regular expression to match against, or null if querying for all
namespace - the namespace to query, or null if querying for all
includeSysTables - False to match only against userspace tables
Returns:
the list of table names
Throws:
IOException
{noformat}
> Add sanity check for Admin#deleteTables and other operations
>
>
> Key: HBASE-23306
> URL: https://issues.apache.org/jira/browse/HBASE-23306
> Project: HBase
> Issue Type: Bug
> Components: Admin
>Reporter: Rahul Kesharwani
>Priority: Minor
>
> h3. Description:
> After taking a brief observation, It seems to be the following operation
> behaves in the opposite manner of what they should be doing in case `null
> argument.
> *For example:* `Admin#listTables(Pattern)` If its argument is `null` then all
> the tables would be returned. Also this very method is being used by rest of
> the listed operation like `deleteTables()`, `enableTables()`,
> `disableTables()` etc.
> h3. Affected operation:
> * Admin.listTables(Pattern)
> * Admin.listTableNames(Pattern)
> * Admin.deleteTables(Pattern)
> * Admin.enableTables(Pattern)
> * Admin.disableTables(Pattern)
> This could be a very rare scenario but I think it should either throw an
> exception or behave opposite of what current behavior is.
> *If this was not intentional* then I can quickly put together a patch with
> precondition check for these operations.
>
> Note: I tried to look for answers but I was not able to find any relevant
> issue/comments hence logging this issue. Feel free to close if this is
> intentional behavior.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23102) Improper Usage of Map putIfAbsent
[
https://issues.apache.org/jira/browse/HBASE-23102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil updated HBASE-23102:
-
Fix Version/s: 3.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Merged PR into master branch. Thanks for the contribution, [~belugabehr]!
> Improper Usage of Map putIfAbsent
> -
>
> Key: HBASE-23102
> URL: https://issues.apache.org/jira/browse/HBASE-23102
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.3.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: newbie, noob
> Fix For: 3.0.0
>
> Attachments: HBASE-23102.1.patch
>
>
> When using {{Map#putIfAbsent}}, the argument should not be a {{new}} object.
> Otherwise, if the item is present, the object that was instantiated is
> immediately thrown away. Instead, use {{Map#computeIfAbsent}} so that the
> object is only instantiated if it is needed.
> There exists a good example in the {{Map}} JavaDoc:
> https://docs.oracle.com/javase/8/docs/api/java/util/Map.html#computeIfAbsent-K-java.util.function.Function-
>
> h2. Locations
> https://github.com/apache/hbase/blob/9370347efea5b09e2fa8f4e5d82fa32491e1181b/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java#L227-L236
> https://github.com/apache/hbase/blob/025ddce868eb06b4072b5152c5ffae5a01e7ae30/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/throttle/StoreHotnessProtector.java#L124-L129
> https://github.com/apache/hbase/blob/1170f28122d9d36e511ba504a5263ec62e11ef6a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java#L555
> https://github.com/apache/hbase/blob/4ca760fe9dd373b8d8a4c48db15e42424920653c/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java#L584-L586
> https://github.com/apache/hbase/blob/4ca760fe9dd373b8d8a4c48db15e42424920653c/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java#L585
> https://github.com/apache/hbase/blob/5b01e613fbbb92e243e99a1d199b4ffbb21ed2d9/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java#L834
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] wchevreuil merged pull request #828: HBASE-23102: Improper Usage of Map putIfAbsent
wchevreuil merged pull request #828: HBASE-23102: Improper Usage of Map putIfAbsent URL: https://github.com/apache/hbase/pull/828 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975942#comment-16975942
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #182 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/182/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/182//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/182//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/182//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] chenxu14 commented on a change in pull request #831: HBASE-23297 [RSGROUP] build RegionPlan per group not per table
chenxu14 commented on a change in pull request #831: HBASE-23297 [RSGROUP]
build RegionPlan per group not per table
URL: https://github.com/apache/hbase/pull/831#discussion_r347123564
##
File path:
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java
##
@@ -607,6 +613,22 @@ public void removeServers(Set servers) throws
IOException {
return result;
}
+ private Map> getAssignmentsByGroup(String
groupName)
Review comment:
Now balanceRSGroup will perform load balancing of group server dimensions.
Suppose we have two groups(groupA and groupB), if tableA's region mis-placed
on groupB, we can execute balanceRSGroup(GroupB) to move it back to the right
place. @guangxuCheng
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23297) [RSGROUP] build RegionPlan per group not per table
[ https://issues.apache.org/jira/browse/HBASE-23297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975941#comment-16975941 ] chenxu commented on HBASE-23297: ya, it's an improvement, the orginal balanceRSGroup will have a final balance, but will take too long time > [RSGROUP] build RegionPlan per group not per table > -- > > Key: HBASE-23297 > URL: https://issues.apache.org/jira/browse/HBASE-23297 > Project: HBase > Issue Type: Improvement > Components: Balancer, rsgroup >Reporter: chenxu >Assignee: chenxu >Priority: Major > > RSGroupAdminServer#balanceRSGroup will take too long time, if the target > group has thousands of tables, because we will build RegionPlan for each > table, I think it’s better to balance per group instead of per table. -- This message was sent by Atlassian Jira (v8.3.4#803005)
