[jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
[ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895556#comment-15895556 ] Andrew Purtell commented on HBASE-12790: I thought we might get somewhere with adding the ability for a client to tag a RPC call and then let the server side plugin do something special like manage multiple queues. As I recall this could work for scans but our multi ops presented a real problem if trying to generalize it for all RPC including mutations. Maybe we don't do that. Maybe we just address the need at hand and stick to reads. Maybe we need to plumb a bit on the server for it to really work. A new issue scoped to that referring back to here would be a fresh start. > Support fairness across parallelized scans > -- > > Key: HBASE-12790 > URL: https://issues.apache.org/jira/browse/HBASE-12790 > Project: HBase > Issue Type: New Feature >Reporter: James Taylor >Assignee: ramkrishna.s.vasudevan > Labels: Phoenix > Attachments: AbstractRoundRobinQueue.java, HBASE-12790_1.patch, > HBASE-12790_5.patch, HBASE-12790_callwrapper.patch, HBASE-12790.patch, > HBASE-12790_trunk_1.patch, PHOENIX_4.5.3-HBase-0.98-2317-SNAPSHOT.zip > > > Some HBase clients parallelize the execution of a scan to reduce latency in > getting back results. This can lead to starvation with a loaded cluster and > interleaved scans, since the RPC queue will be ordered and processed on a > FIFO basis. For example, if there are two clients, A & B that submit largish > scans at the same time. Say each scan is broken down into 100 scans by the > client (broken down into equal depth chunks along the row key), and the 100 > scans of client A are queued first, followed immediately by the 100 scans of > client B. In this case, client B will be starved out of getting any results > back until the scans for client A complete. > One solution to this is to use the attached AbstractRoundRobinQueue instead > of the standard FIFO queue. The queue to be used could be (maybe it already > is) configurable based on a new config parameter. Using this queue would > require the client to have the same identifier for all of the 100 parallel > scans that represent a single logical scan from the clients point of view. > With this information, the round robin queue would pick off a task from the > queue in a round robin fashion (instead of a strictly FIFO manner) to prevent > starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
[ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895552#comment-15895552 ] Lars Hofhansl commented on HBASE-12790: --- bq. RpcScheduler is pluggable. You need more than that? Is that all we need [~jamestaylor]? Let me summarize how the Phoenix folks got here: # HBase scan contract is serial per scan. I.e. a scan will already return all keys in order whether the client needs it that way or not. Hence no parallel execution on behalf of a single scan (both [~stack] and I had made attempts to improve that but did not finish) # Scans cannot easily be broken down to units smaller than a region (it's certainly possible to do that, but there's no information about internal data skew inside a region) # For this Phoenix adds "guideposts". These are equidistant markers, so that Phoenix can know about the key distribution inside a region. # Phoenix uses guideposts to schedule many small scans. The units are fairly small (100MB-1GB worth of cells) to allow for fairness between queries. # If many query-chunks - a.k.a. scans - of a large query can hog the RPC queues than much of the advantage is lost. # Hence the desire for a this type of "group based" scheduling so that small queries can finish before all large queries in the queue need to finish. The group is a Phoenix query. So it is simply the desire to extend the fair queuing that HBase already has (HBASE-10993) to a query in Phoenix which may issue 1000's of scans to as many region servers. That just for the history... I do agree that the patch proposed here is too complex and perhaps wants to do too much. Now perhaps hbase.ipc.server.callqueue.scan.ratio from HBASE-11355 and HBASE-11724 gives us what we need _if_ we can use this for small scans, so that small scans can land on the "Get" queue. That way we can reserve that queue for small scans and Gets, and other queues for large scans. It's not ideal, though. The best is to allow somehow to round-robin between the queries on behave which the scans are operating. That abstraction is not available in HBase. > Support fairness across parallelized scans > -- > > Key: HBASE-12790 > URL: https://issues.apache.org/jira/browse/HBASE-12790 > Project: HBase > Issue Type: New Feature >Reporter: James Taylor >Assignee: ramkrishna.s.vasudevan > Labels: Phoenix > Attachments: AbstractRoundRobinQueue.java, HBASE-12790_1.patch, > HBASE-12790_5.patch, HBASE-12790_callwrapper.patch, HBASE-12790.patch, > HBASE-12790_trunk_1.patch, PHOENIX_4.5.3-HBase-0.98-2317-SNAPSHOT.zip > > > Some HBase clients parallelize the execution of a scan to reduce latency in > getting back results. This can lead to starvation with a loaded cluster and > interleaved scans, since the RPC queue will be ordered and processed on a > FIFO basis. For example, if there are two clients, A & B that submit largish > scans at the same time. Say each scan is broken down into 100 scans by the > client (broken down into equal depth chunks along the row key), and the 100 > scans of client A are queued first, followed immediately by the 100 scans of > client B. In this case, client B will be starved out of getting any results > back until the scans for client A complete. > One solution to this is to use the attached AbstractRoundRobinQueue instead > of the standard FIFO queue. The queue to be used could be (maybe it already > is) configurable based on a new config parameter. Using this queue would > require the client to have the same identifier for all of the 100 parallel > scans that represent a single logical scan from the clients point of view. > With this information, the round robin queue would pick off a task from the > queue in a round robin fashion (instead of a strictly FIFO manner) to prevent > starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16630) Fragmentation in long running Bucket Cache
[ https://issues.apache.org/jira/browse/HBASE-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895545#comment-15895545 ] Hudson commented on HBASE-16630: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #844 (See [https://builds.apache.org/job/HBase-1.2-IT/844/]) HBASE-16630 Fragmentation in long running Bucket Cache (ramkrishna: rev ea5ac3856cde31b17615bbf38f54f2808a5647ff) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketAllocator.java > Fragmentation in long running Bucket Cache > -- > > Key: HBASE-16630 > URL: https://issues.apache.org/jira/browse/HBASE-16630 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.3 >Reporter: deepankar >Assignee: deepankar >Priority: Critical > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.6 > > Attachments: 16630-v2-suggest.patch, 16630-v3-suggest.patch, > HBASE-16630.patch, HBASE-16630-v2.patch, HBASE-16630-v3-branch-1.patch, > HBASE-16630-v3-branch-1.X.patch, HBASE-16630-v3.patch, > HBASE-16630-v4-branch-1.X.patch > > > As we are running bucket cache for a long time in our system, we are > observing cases where some nodes after some time does not fully utilize the > bucket cache, in some cases it is even worse in the sense they get stuck at a > value < 0.25 % of the bucket cache (DEFAULT_MEMORY_FACTOR as all our tables > are configured in-memory for simplicity sake). > We took a heap dump and analyzed what is happening and saw that is classic > case of fragmentation, current implementation of BucketCache (mainly > BucketAllocator) relies on the logic that fullyFreeBuckets are available for > switching/adjusting cache usage between different bucketSizes . But once a > compaction / bulkload happens and the blocks are evicted from a bucket size , > these are usually evicted from random places of the buckets of a bucketSize > and thus locking the number of buckets associated with a bucketSize and in > the worst case of the fragmentation we have seen some bucketSizes with > occupancy ratio of < 10 % But they dont have any completelyFreeBuckets to > share with the other bucketSize. > Currently the existing eviction logic helps in the cases where cache used is > more the MEMORY_FACTOR or MULTI_FACTOR and once those evictions are also > done, the eviction (freeSpace function) will not evict anything and the cache > utilization will be stuck at that value without any allocations for other > required sizes. > The fix for this we came up with is simple that we do deFragmentation ( > compaction) of the bucketSize and thus increasing the occupancy ratio and > also freeing up the buckets to be fullyFree, this logic itself is not > complicated as the bucketAllocator takes care of packing the blocks in the > buckets, we need evict and re-allocate the blocks for all the BucketSizes > that dont fit the criteria. > I am attaching an initial patch just to give an idea of what we are thinking > and I'll improve it based on the comments from the community. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17733) Undo registering regionservers in zk with ephemeral nodes; its more trouble than its worth
stack created HBASE-17733: - Summary: Undo registering regionservers in zk with ephemeral nodes; its more trouble than its worth Key: HBASE-17733 URL: https://issues.apache.org/jira/browse/HBASE-17733 Project: HBase Issue Type: Brainstorming Reporter: stack Elsewhere, we are undoing the use of ZK (replication current WAL offset, regions-in-transition, etc). I have another case where using ZK, while convenient (call-backs), has holes. The scenario is prompted by review of HBASE-9593. Currently, a RS registers with the Master by calling the Master's reportForDuty. After the Master responds with the name we are to use for ourselves (as well as other properties we need to 'run'), we then turnaround and do a new RPC out to the zk ensemble to register an ephemeral znode for the RS. We notice a RS has gone away -- crashed -- because its znode evaporates and the Master has a watcher triggered notifying it the RS has gone (after a zk session timeout of tens of seconds). Cumbersome (Setting watchers, zk session timeouts) and indirect. Master then trips the server shutdown handler which does reassign of regions from the crashed server. In HBASE-9593, we were trying to handle the rare but possible case where the RS would die after registering w/ the Master but before we put up our ephemeral znode. In this case a RS would live in the Master's internals forever because there is no ephemeral znode to expire to do cleanup and removal of the never-started RS. Lets get ZK out of the loop. Then only the Master and RS involved heartbeating each other. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
[ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895521#comment-15895521 ] stack commented on HBASE-12790: --- bq. How about a new JIRA, Lars Hofhansl? Nah. Why would we drop history/context? There is good stuff in here: e.g. "apurtell Andrew Purtell added a comment - 09/Nov/15 10:25". bq. James Taylor suggests to simply make the queue class pluggable. RpcScheduler is pluggable. You need more than that? > Support fairness across parallelized scans > -- > > Key: HBASE-12790 > URL: https://issues.apache.org/jira/browse/HBASE-12790 > Project: HBase > Issue Type: New Feature >Reporter: James Taylor >Assignee: ramkrishna.s.vasudevan > Labels: Phoenix > Attachments: AbstractRoundRobinQueue.java, HBASE-12790_1.patch, > HBASE-12790_5.patch, HBASE-12790_callwrapper.patch, HBASE-12790.patch, > HBASE-12790_trunk_1.patch, PHOENIX_4.5.3-HBase-0.98-2317-SNAPSHOT.zip > > > Some HBase clients parallelize the execution of a scan to reduce latency in > getting back results. This can lead to starvation with a loaded cluster and > interleaved scans, since the RPC queue will be ordered and processed on a > FIFO basis. For example, if there are two clients, A & B that submit largish > scans at the same time. Say each scan is broken down into 100 scans by the > client (broken down into equal depth chunks along the row key), and the 100 > scans of client A are queued first, followed immediately by the 100 scans of > client B. In this case, client B will be starved out of getting any results > back until the scans for client A complete. > One solution to this is to use the attached AbstractRoundRobinQueue instead > of the standard FIFO queue. The queue to be used could be (maybe it already > is) configurable based on a new config parameter. Using this queue would > require the client to have the same identifier for all of the 100 parallel > scans that represent a single logical scan from the clients point of view. > With this information, the round robin queue would pick off a task from the > queue in a round robin fashion (instead of a strictly FIFO manner) to prevent > starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-16630) Fragmentation in long running Bucket Cache
[ https://issues.apache.org/jira/browse/HBASE-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-16630. Resolution: Fixed Pushed to all branches including branch-1.2. There was an env issue in my branch-1.2. I was able to correct that and committed this patch. Thanks for all the reviews and for your persistence with this patch [~dvdreddy]. > Fragmentation in long running Bucket Cache > -- > > Key: HBASE-16630 > URL: https://issues.apache.org/jira/browse/HBASE-16630 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.3 >Reporter: deepankar >Assignee: deepankar >Priority: Critical > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.6 > > Attachments: 16630-v2-suggest.patch, 16630-v3-suggest.patch, > HBASE-16630.patch, HBASE-16630-v2.patch, HBASE-16630-v3-branch-1.patch, > HBASE-16630-v3-branch-1.X.patch, HBASE-16630-v3.patch, > HBASE-16630-v4-branch-1.X.patch > > > As we are running bucket cache for a long time in our system, we are > observing cases where some nodes after some time does not fully utilize the > bucket cache, in some cases it is even worse in the sense they get stuck at a > value < 0.25 % of the bucket cache (DEFAULT_MEMORY_FACTOR as all our tables > are configured in-memory for simplicity sake). > We took a heap dump and analyzed what is happening and saw that is classic > case of fragmentation, current implementation of BucketCache (mainly > BucketAllocator) relies on the logic that fullyFreeBuckets are available for > switching/adjusting cache usage between different bucketSizes . But once a > compaction / bulkload happens and the blocks are evicted from a bucket size , > these are usually evicted from random places of the buckets of a bucketSize > and thus locking the number of buckets associated with a bucketSize and in > the worst case of the fragmentation we have seen some bucketSizes with > occupancy ratio of < 10 % But they dont have any completelyFreeBuckets to > share with the other bucketSize. > Currently the existing eviction logic helps in the cases where cache used is > more the MEMORY_FACTOR or MULTI_FACTOR and once those evictions are also > done, the eviction (freeSpace function) will not evict anything and the cache > utilization will be stuck at that value without any allocations for other > required sizes. > The fix for this we came up with is simple that we do deFragmentation ( > compaction) of the bucketSize and thus increasing the occupancy ratio and > also freeing up the buckets to be fullyFree, this logic itself is not > complicated as the bucketAllocator takes care of packing the blocks in the > buckets, we need evict and re-allocate the blocks for all the BucketSizes > that dont fit the criteria. > I am attaching an initial patch just to give an idea of what we are thinking > and I'll improve it based on the comments from the community. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16630) Fragmentation in long running Bucket Cache
[ https://issues.apache.org/jira/browse/HBASE-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16630: --- Fix Version/s: 1.4.0 > Fragmentation in long running Bucket Cache > -- > > Key: HBASE-16630 > URL: https://issues.apache.org/jira/browse/HBASE-16630 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.3 >Reporter: deepankar >Assignee: deepankar >Priority: Critical > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.6 > > Attachments: 16630-v2-suggest.patch, 16630-v3-suggest.patch, > HBASE-16630.patch, HBASE-16630-v2.patch, HBASE-16630-v3-branch-1.patch, > HBASE-16630-v3-branch-1.X.patch, HBASE-16630-v3.patch, > HBASE-16630-v4-branch-1.X.patch > > > As we are running bucket cache for a long time in our system, we are > observing cases where some nodes after some time does not fully utilize the > bucket cache, in some cases it is even worse in the sense they get stuck at a > value < 0.25 % of the bucket cache (DEFAULT_MEMORY_FACTOR as all our tables > are configured in-memory for simplicity sake). > We took a heap dump and analyzed what is happening and saw that is classic > case of fragmentation, current implementation of BucketCache (mainly > BucketAllocator) relies on the logic that fullyFreeBuckets are available for > switching/adjusting cache usage between different bucketSizes . But once a > compaction / bulkload happens and the blocks are evicted from a bucket size , > these are usually evicted from random places of the buckets of a bucketSize > and thus locking the number of buckets associated with a bucketSize and in > the worst case of the fragmentation we have seen some bucketSizes with > occupancy ratio of < 10 % But they dont have any completelyFreeBuckets to > share with the other bucketSize. > Currently the existing eviction logic helps in the cases where cache used is > more the MEMORY_FACTOR or MULTI_FACTOR and once those evictions are also > done, the eviction (freeSpace function) will not evict anything and the cache > utilization will be stuck at that value without any allocations for other > required sizes. > The fix for this we came up with is simple that we do deFragmentation ( > compaction) of the bucketSize and thus increasing the occupancy ratio and > also freeing up the buckets to be fullyFree, this logic itself is not > complicated as the bucketAllocator takes care of packing the blocks in the > buckets, we need evict and re-allocate the blocks for all the BucketSizes > that dont fit the criteria. > I am attaching an initial patch just to give an idea of what we are thinking > and I'll improve it based on the comments from the community. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
[ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895487#comment-15895487 ] James Taylor commented on HBASE-12790: -- How about a new JIRA, [~lhofhansl]? > Support fairness across parallelized scans > -- > > Key: HBASE-12790 > URL: https://issues.apache.org/jira/browse/HBASE-12790 > Project: HBase > Issue Type: New Feature >Reporter: James Taylor >Assignee: ramkrishna.s.vasudevan > Labels: Phoenix > Attachments: AbstractRoundRobinQueue.java, HBASE-12790_1.patch, > HBASE-12790_5.patch, HBASE-12790_callwrapper.patch, HBASE-12790.patch, > HBASE-12790_trunk_1.patch, PHOENIX_4.5.3-HBase-0.98-2317-SNAPSHOT.zip > > > Some HBase clients parallelize the execution of a scan to reduce latency in > getting back results. This can lead to starvation with a loaded cluster and > interleaved scans, since the RPC queue will be ordered and processed on a > FIFO basis. For example, if there are two clients, A & B that submit largish > scans at the same time. Say each scan is broken down into 100 scans by the > client (broken down into equal depth chunks along the row key), and the 100 > scans of client A are queued first, followed immediately by the 100 scans of > client B. In this case, client B will be starved out of getting any results > back until the scans for client A complete. > One solution to this is to use the attached AbstractRoundRobinQueue instead > of the standard FIFO queue. The queue to be used could be (maybe it already > is) configurable based on a new config parameter. Using this queue would > require the client to have the same identifier for all of the 100 parallel > scans that represent a single logical scan from the clients point of view. > With this information, the round robin queue would pick off a task from the > queue in a round robin fashion (instead of a strictly FIFO manner) to prevent > starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17680) Run mini cluster through JNI in tests
[ https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17680: --- Attachment: 17680.v30.txt Rebased patch. The following passed: buck test //core:retry-test > Run mini cluster through JNI in tests > - > > Key: HBASE-17680 > URL: https://issues.apache.org/jira/browse/HBASE-17680 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17680.v14.txt, 17680.v17.txt, 17680.v18.txt, > 17680.v1.txt, 17680.v20.txt, 17680.v22.txt, 17680.v23.txt, 17680.v26.txt, > 17680.v27.txt, 17680.v28.txt, 17680.v29.txt, 17680.v30.txt, 17680.v3.txt, > 17680.v8.txt > > > Currently tests start local hbase cluster through hbase shell. > There is less control over the configuration of the local cluster this way. > This issue would replace hbase shell with JNI interface to mini cluster. > We would have full control over the cluster behavior. > Thanks to [~devaraj] who started this initiative. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17732) Add CpType template to CoprocessorEnvironment
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895485#comment-15895485
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 7s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 43s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 9s
{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 23s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 47m 14s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 103m 11s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Uncallable method
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost$RegionEnvironment)
defined in anonymous class At RegionCoprocessorHost.java:anonymous class At
RegionCoprocessorHost.java:[lines 493-494] |
| Failed junit tests | hadoop.hbase.filter.TestDependentColumnFilter |
| | hadoop.hbase.coprocessor.TestCoprocessorInterface |
| | hadoop.hbase.regionserver.TestWALMonotonicallyIncreasingSeqId |
| | hadoop.hbase.regionserver.TestHRegionFileSystem |
| | hadoop.hbase.master.locking.TestLockManager |
| | hadoop.hbase.master.locking.TestLockProcedure |
| | hadoop.hbase.coprocessor.TestRegionObserverStacking |
| | hadoop.hbase.filter.TestFilter |
| | hadoop.hbase.filter.TestInvocationRecordFilter |
| | hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan |
| | hadoop.hbase.regionserver.TestWideScanner |
| | hadoop.hbase.procedure.TestProcedureManager |
| | hadoop.hbase.filter.TestColumnPrefixFilter |
|
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-17465: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed this to the branch. Very nice work [~xiaobingo]. > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch, > HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch, > HBASE-17465-HBASE-14850.011.patch, hbase-17465_v12.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[
https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-17465:
--
Attachment: hbase-17465_v12.patch
I was able to run {{make lint}} and addressed most of the reported issues as
well as a couple of minor fixes. v12 is the one that is committed.
> [C++] implement request retry mechanism over RPC
>
>
> Key: HBASE-17465
> URL: https://issues.apache.org/jira/browse/HBASE-17465
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HBASE-17465-HBASE-14850.000.patch,
> HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch,
> HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch,
> HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch,
> HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch,
> HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch,
> HBASE-17465-HBASE-14850.011.patch, hbase-17465_v12.patch
>
>
> HBASE-17051 implemented RPC layer. Requests retries will make system
> reliable. This JIRA proposes adding it, which corresponds to similar
> implementation in SingleRequestCallerBuilder (or BatchCallerBuilder,
> ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and
> AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more
> generic, decoupled with HRegionLocation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17731) Fractional latency reporting in MultiThreadedAction
[
https://issues.apache.org/jira/browse/HBASE-17731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895473#comment-15895473
]
Hadoop QA commented on HBASE-17731:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 38s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 19s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 130m 15s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12856017/HBASE-17731.patch |
| JIRA Issue | HBASE-17731 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 4e34fbe52cae 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 404a288 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5946/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5946/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5946/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5946/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Fractional latency reporting in MultiThreadedAction
> ---
>
>
[jira] [Commented] (HBASE-17568) Expire region reports in the HMaster
[
https://issues.apache.org/jira/browse/HBASE-17568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895463#comment-15895463
]
Hadoop QA commented on HBASE-17568:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 19s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 29s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 39s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 50s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 58s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.5.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m 45s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 144m 35s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12856011/HBASE-17568.002.HBASE-16961.patch
|
| JIRA Issue | HBASE-17568 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 38720e5a429a 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-16961 / cc14ccd |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/5945/artifact/patchprocess/whitespace-tabs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5945/testReport/ |
| modules | C:
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895448#comment-15895448
]
Hudson commented on HBASE-17722:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #130 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/130/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev 51dfc1463336841095b83ab96c8903e3ac0bbbe2)
* (edit)
hbase-archetypes/hbase-client-project/src/main/resources/log4j.properties
* (edit) hbase-server/src/test/resources/log4j.properties
* (edit) hbase-common/src/test/resources/log4j.properties
* (edit) conf/log4j.properties
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) hbase-rest/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit)
hbase-archetypes/hbase-shaded-client-project/src/main/resources/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17465) [C++] implement request retry mechanism over RPC
[
https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895432#comment-15895432
]
Hadoop QA commented on HBASE-17465:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m 8s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 9s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 2s
{color} | {color:green} HBASE-14850 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} HBASE-14850 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m 4s
{color} | {color:red} The patch generated 1 new + 436 unchanged - 1 fixed = 437
total (was 437) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 41s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
2s {color} | {color:green} the patch passed {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 15s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:date2017-03-04 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12856021/HBASE-17465-HBASE-14850.011.patch
|
| JIRA Issue | HBASE-17465 |
| Optional Tests | shellcheck shelldocs cc compile |
| uname | Linux fd35663f0326 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | nobuild |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-14850 / 7d284f6 |
| shellcheck | v0.4.5 |
| shellcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/5947/artifact/patchprocess/diff-patch-shellcheck.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5947/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> [C++] implement request retry mechanism over RPC
>
>
> Key: HBASE-17465
> URL: https://issues.apache.org/jira/browse/HBASE-17465
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HBASE-17465-HBASE-14850.000.patch,
> HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch,
> HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch,
> HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch,
> HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch,
> HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch,
> HBASE-17465-HBASE-14850.011.patch
>
>
> HBASE-17051 implemented RPC layer. Requests retries will make system
> reliable. This JIRA proposes adding it, which corresponds to similar
> implementation in SingleRequestCallerBuilder (or BatchCallerBuilder,
> ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and
> AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more
> generic, decoupled with HRegionLocation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17716) Formalize Scan Metric names
[ https://issues.apache.org/jira/browse/HBASE-17716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895423#comment-15895423 ] Samarth Jain commented on HBASE-17716: -- +1 for @InterfaceAudience.Public(HBaseInterfaceAudience.METRIC) > Formalize Scan Metric names > --- > > Key: HBASE-17716 > URL: https://issues.apache.org/jira/browse/HBASE-17716 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Karan Mehta >Assignee: Karan Mehta >Priority: Minor > Attachments: HBASE-17716.patch > > > HBase provides various metrics through the API's exposed by ScanMetrics > class. > The JIRA PHOENIX-3248 requires them to be surfaced through the Phoenix > Metrics API. Currently these metrics are referred via hard-coded strings, > which are not formal and can break the Phoenix API. Hence we need to refactor > the code to assign enums for these metrics. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17602) Tweak chore delay/period defaults
[
https://issues.apache.org/jira/browse/HBASE-17602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895419#comment-15895419
]
Hadoop QA commented on HBASE-17602:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
0s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 4s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 4s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 4s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 2s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 1s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.5.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 82m 43s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 124m 46s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12856004/HBASE-17602.001.HBASE-16961.patch
|
| JIRA Issue | HBASE-17602 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux eb8af53d5ed4 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-16961 / cc14ccd |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5943/testReport/ |
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895418#comment-15895418
]
Hudson commented on HBASE-17722:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #110 (See
[https://builds.apache.org/job/HBase-1.2-JDK7/110/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev 5ab33b3cd77d6e1946fea00c9e8e75a5836e8184)
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) hbase-common/src/test/resources/log4j.properties
* (edit) hbase-rest/src/test/resources/log4j.properties
* (edit) conf/log4j.properties
* (edit) hbase-server/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895412#comment-15895412
]
Hudson commented on HBASE-17722:
FAILURE: Integrated in Jenkins build HBase-1.4 #656 (See
[https://builds.apache.org/job/HBase-1.4/656/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev ccf1f17d78f8fb4fd5dbcd5bd6369bee84032914)
* (edit)
hbase-archetypes/hbase-client-project/src/main/resources/log4j.properties
* (edit) conf/log4j.properties
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) hbase-server/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit) hbase-rest/src/test/resources/log4j.properties
* (edit) hbase-common/src/test/resources/log4j.properties
* (edit)
hbase-archetypes/hbase-shaded-client-project/src/main/resources/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17732) Add CpType template to CoprocessorEnvironment
[ https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17732: - Status: Patch Available (was: Open) > Add CpType template to CoprocessorEnvironment > - > > Key: HBASE-17732 > URL: https://issues.apache.org/jira/browse/HBASE-17732 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-17732.master.001.patch > > > Adding template for coprocessor type to CoprocessorEnvironment will: > - get rid of 'instanceOf' checks we do in various coprocessor hosts > - allow us to load only relevant coprocessors in hosts (right now i think > each host loads all coprocs and it's only during execOperation that it checks > if the coproc is of correct type ie XXCoprocessorHost will load XXObserver, > YYObserver, and all others, and in will check in execOperation if coproc is > of type XXObserver and ignore the rest). > - allow sharing of a bunch functions/classes which are currently 99% > duplicate of each other. For eg. CoprocessorOperations, > CoprocessorOperationWithResult, execOperations(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895396#comment-15895396 ] Enis Soztutar commented on HBASE-17465: --- bq. BTW, I have problems of running make lint, e.g. Hmm, maybe the docker container cannot do DNS resolution? It fetches the script from the github location. In any case, let me take another look at the patch. > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch, > HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch, > HBASE-17465-HBASE-14850.011.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895386#comment-15895386
]
Ted Yu commented on HBASE-17623:
+1
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: CHIA-PING TSAI
>Assignee: CHIA-PING TSAI
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.branch-1.v1.patch, HBASE-17623.branch-1.v2.patch,
> HBASE-17623.branch-1.v2.patch, HBASE-17623.v0.patch, HBASE-17623.v1.patch,
> HBASE-17623.v1.patch, HBASE-17623.v2.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The onDiskBlockBytesWithHeader should maintain a bytes array which can be
> reused when building the hfile.
> # The onDiskBlockBytesWithHeader is copied to an new bytes array only when we
> need to cache the block.
> # If no block need to be cached, the uncompressedBlockBytesWithHeader will
> never be created.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895383#comment-15895383
]
Hadoop QA commented on HBASE-17623:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
5s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s
{color} | {color:green} branch-1 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 26s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s
{color} | {color:green} branch-1 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 43s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 55s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m 11s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 136m 21s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:e01ee2f |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855806/HBASE-17623.branch-1.v2.patch
|
| JIRA Issue | HBASE-17623 |
| Optional Tests |
[jira] [Commented] (HBASE-17465) [C++] implement request retry mechanism over RPC
[
https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895380#comment-15895380
]
Xiaobing Zhou commented on HBASE-17465:
---
v11 posted. It addressed all your latest comments except I keep
RpcControllerFactory, RpcController to make easy to follow Java pattern.
is also kept to have generic specification, but
with default value, e.g.
{code}
{code}
BTW, I have problems of running make lint, e.g.
{noformat}
bin/cpplint.sh
--2017-03-04 01:29:15--
https://raw.githubusercontent.com/google/styleguide/gh-pages/cpplint/cpplint.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed:
Temporary failure in name resolution.
wget: unable to resolve host address 'raw.githubusercontent.com'
{noformat}
> [C++] implement request retry mechanism over RPC
>
>
> Key: HBASE-17465
> URL: https://issues.apache.org/jira/browse/HBASE-17465
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HBASE-17465-HBASE-14850.000.patch,
> HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch,
> HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch,
> HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch,
> HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch,
> HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch,
> HBASE-17465-HBASE-14850.011.patch
>
>
> HBASE-17051 implemented RPC layer. Requests retries will make system
> reliable. This JIRA proposes adding it, which corresponds to similar
> implementation in SingleRequestCallerBuilder (or BatchCallerBuilder,
> ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and
> AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more
> generic, decoupled with HRegionLocation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Attachment: HBASE-17465-HBASE-14850.011.patch > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch, > HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch, > HBASE-17465-HBASE-14850.011.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895373#comment-15895373
]
Hudson commented on HBASE-17722:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #105 (See
[https://builds.apache.org/job/HBase-1.2-JDK8/105/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev 5ab33b3cd77d6e1946fea00c9e8e75a5836e8184)
* (edit) hbase-server/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
* (edit) hbase-rest/src/test/resources/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) conf/log4j.properties
* (edit) hbase-common/src/test/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17680) Run mini cluster through JNI in tests
[ https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17680: --- Attachment: 17680.v29.txt Patch v29 removes commented out code. > Run mini cluster through JNI in tests > - > > Key: HBASE-17680 > URL: https://issues.apache.org/jira/browse/HBASE-17680 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17680.v14.txt, 17680.v17.txt, 17680.v18.txt, > 17680.v1.txt, 17680.v20.txt, 17680.v22.txt, 17680.v23.txt, 17680.v26.txt, > 17680.v27.txt, 17680.v28.txt, 17680.v29.txt, 17680.v3.txt, 17680.v8.txt > > > Currently tests start local hbase cluster through hbase shell. > There is less control over the configuration of the local cluster this way. > This issue would replace hbase shell with JNI interface to mini cluster. > We would have full control over the cluster behavior. > Thanks to [~devaraj] who started this initiative. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895361#comment-15895361
]
Hadoop QA commented on HBASE-17706:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 45s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 106m 15s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 154m 58s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855986/HBASE-17706-05.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 5a2f8033f5bf 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 404a288 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5939/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5939/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>
[jira] [Commented] (HBASE-17680) Run mini cluster through JNI in tests
[
https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895347#comment-15895347
]
Enis Soztutar commented on HBASE-17680:
---
Can you please remove all commented out code. There are still references.
Also, in multiple places, you should remove these comments as well:
{code}
// The connection stays open and we don't want that.
// So we are stopping the connection.
// We can remove this once we have fixed the folly part
- delete test_util;
{code}
> Run mini cluster through JNI in tests
> -
>
> Key: HBASE-17680
> URL: https://issues.apache.org/jira/browse/HBASE-17680
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17680.v14.txt, 17680.v17.txt, 17680.v18.txt,
> 17680.v1.txt, 17680.v20.txt, 17680.v22.txt, 17680.v23.txt, 17680.v26.txt,
> 17680.v27.txt, 17680.v28.txt, 17680.v3.txt, 17680.v8.txt
>
>
> Currently tests start local hbase cluster through hbase shell.
> There is less control over the configuration of the local cluster this way.
> This issue would replace hbase shell with JNI interface to mini cluster.
> We would have full control over the cluster behavior.
> Thanks to [~devaraj] who started this initiative.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17732) Add CpType template to CoprocessorEnvironment
[ https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17732: - Attachment: HBASE-17732.master.001.patch > Add CpType template to CoprocessorEnvironment > - > > Key: HBASE-17732 > URL: https://issues.apache.org/jira/browse/HBASE-17732 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-17732.master.001.patch > > > Adding template for coprocessor type to CoprocessorEnvironment will: > - get rid of 'instanceOf' checks we do in various coprocessor hosts > - allow us to load only relevant coprocessors in hosts (right now i think > each host loads all coprocs and it's only during execOperation that it checks > if the coproc is of correct type ie XXCoprocessorHost will load XXObserver, > YYObserver, and all others, and in will check in execOperation if coproc is > of type XXObserver and ignore the rest). > - allow sharing of a bunch functions/classes which are currently 99% > duplicate of each other. For eg. CoprocessorOperations, > CoprocessorOperationWithResult, execOperations(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17731) Fractional latency reporting in MultiThreadedAction
[ https://issues.apache.org/jira/browse/HBASE-17731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-17731: --- Attachment: HBASE-17731.patch Trivial patch > Fractional latency reporting in MultiThreadedAction > --- > > Key: HBASE-17731 > URL: https://issues.apache.org/jira/browse/HBASE-17731 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: HBASE-17731.patch > > > When average latency is less than one millisecond the LoadTestTool tool > reports a latency of 0. Better to report a fraction out to a couple of > decimal points. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17731) Fractional latency reporting in MultiThreadedAction
[ https://issues.apache.org/jira/browse/HBASE-17731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-17731: --- Status: Patch Available (was: Open) > Fractional latency reporting in MultiThreadedAction > --- > > Key: HBASE-17731 > URL: https://issues.apache.org/jira/browse/HBASE-17731 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: HBASE-17731.patch > > > When average latency is less than one millisecond the LoadTestTool tool > reports a latency of 0. Better to report a fraction out to a couple of > decimal points. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17732) Add CpType template to CoprocessorEnvironment
Appy created HBASE-17732: Summary: Add CpType template to CoprocessorEnvironment Key: HBASE-17732 URL: https://issues.apache.org/jira/browse/HBASE-17732 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy Adding template for coprocessor type to CoprocessorEnvironment will: - get rid of 'instanceOf' checks we do in various coprocessor hosts - allow us to load only relevant coprocessors in hosts (right now i think each host loads all coprocs and it's only during execOperation that it checks if the coproc is of correct type ie XXCoprocessorHost will load XXObserver, YYObserver, and all others, and in will check in execOperation if coproc is of type XXObserver and ignore the rest). - allow sharing of a bunch functions/classes which are currently 99% duplicate of each other. For eg. CoprocessorOperations, CoprocessorOperationWithResult, execOperations(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17731) Fractional latency reporting in MultiThreadedAction
Andrew Purtell created HBASE-17731: -- Summary: Fractional latency reporting in MultiThreadedAction Key: HBASE-17731 URL: https://issues.apache.org/jira/browse/HBASE-17731 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Trivial Fix For: 2.0.0, 1.4.0, 1.3.1 Attachments: HBASE-17731.patch When average latency is less than one millisecond the LoadTestTool tool reports a latency of 0. Better to report a fraction out to a couple of decimal points. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17730: - Description: Jiras breaking coprocessor compatibility should be marked with component ' Coprocessor', and label 'incompatible'. Close to releasing 2.0, we should go through all such jiras and write down steps for migrating coprocessor easily. The idea is, it might be very hard to fix coprocessor breakages by reverse engineering errors, but will be easier we suggest easiest way to fix breakages resulting from each individual incompatible change. For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors because BaseXXXObserver classes are gone and will probably result in a lot of confusion, but if we explicitly mention the fix which is just one line change - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - it makes it very easy. was: Jiras breaking coprocessor compatibility should be marked with component ' Coprocessor', and label 'incompatible'. Close to releasing 2.0, we should go through all such jiras and write down steps for migrating coprocessor easily. The idea is, it might be very hard to fix coprocessor breakages by reverse engineering errors, but will be easier we suggest easiest way to fix breakages resulting from each individual incompatible change. For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors because BaseXXXObserver classes are gone, but the fix is actually just one line : simply change "Foo extends BaseXXXObserver" to "Foo implements XXXObserver". > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: migration >Reporter: Appy > Fix For: 2.0.0 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17730) Migration to 2.0 for coprocessors
Appy created HBASE-17730: Summary: Migration to 2.0 for coprocessors Key: HBASE-17730 URL: https://issues.apache.org/jira/browse/HBASE-17730 Project: HBase Issue Type: Sub-task Reporter: Appy Jiras breaking coprocessor compatibility should be marked with component ' Coprocessor', and label 'incompatible'. Close to releasing 2.0, we should go through all such jiras and write down steps for migrating coprocessor easily. The idea is, it might be very hard to fix coprocessor breakages by reverse engineering errors, but will be easier we suggest easiest way to fix breakages resulting from each individual incompatible change. For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors because BaseXXXObserver classes are gone, but the fix is actually just one line : simply change "Foo extends BaseXXXObserver" to "Foo implements XXXObserver". -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17729) Missing shortcuts for some useful HCD options
[
https://issues.apache.org/jira/browse/HBASE-17729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895320#comment-15895320
]
Hadoop QA commented on HBASE-17729:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 0s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 0s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 8s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 10s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 38s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 48s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12856003/HBASE-17729.patch |
| JIRA Issue | HBASE-17729 |
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux effcbe7ab320 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 404a288 |
| Default Java | 1.8.0_121 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5942/testReport/ |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5942/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Missing shortcuts for some useful HCD options
> -
>
> Key: HBASE-17729
> URL: https://issues.apache.org/jira/browse/HBASE-17729
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.3.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: HBASE-17729.patch
>
>
> Missing:
> - CACHE_INDEX_ON_WRITE
> - CACHE_BLOOMS_ON_WRITE
> - EVICT_BLOCKS_ON_CLOSE
> - CACHE_DATA_IN_L1
> It's possible to set them with syntax like
> {noformat}
> { CONFIGURATION = { = } }
> {noformat}
> but let's save some keystrokes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895314#comment-15895314
]
Hudson commented on HBASE-17722:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #119 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/119/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev 51dfc1463336841095b83ab96c8903e3ac0bbbe2)
* (edit)
hbase-archetypes/hbase-client-project/src/main/resources/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
* (edit) conf/log4j.properties
* (edit) hbase-common/src/test/resources/log4j.properties
* (edit) hbase-server/src/test/resources/log4j.properties
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
* (edit) hbase-rest/src/test/resources/log4j.properties
* (edit)
hbase-archetypes/hbase-shaded-client-project/src/main/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17568) Expire region reports in the HMaster
[ https://issues.apache.org/jira/browse/HBASE-17568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17568: --- Attachment: HBASE-17568.002.HBASE-16961.patch .002 Rebase'd patch on top of the recent changes. > Expire region reports in the HMaster > > > Key: HBASE-17568 > URL: https://issues.apache.org/jira/browse/HBASE-17568 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17568.001.patch, HBASE-17568.002.HBASE-16961.patch > > > (From a TODO) > The RegionServers send reports of sizes from to the Master so the Master can > compute the "size" of each Table. > Presently, the Master has no way to expire these reports. Thus, reports for > tables that are deleted would be retained in memory. Retain the time at which > the report was received by the master, and then use that to determine when to > age it off. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895308#comment-15895308
]
Hadoop QA commented on HBASE-17706:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 53s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 102m 56s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 146m 5s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855979/HBASE-17706-04.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 248115c301c8 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 404a288 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5938/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5938/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>
[jira] [Updated] (HBASE-17568) Expire region reports in the HMaster
[ https://issues.apache.org/jira/browse/HBASE-17568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17568: --- Fix Version/s: HBASE-16961 Status: Patch Available (was: Open) > Expire region reports in the HMaster > > > Key: HBASE-17568 > URL: https://issues.apache.org/jira/browse/HBASE-17568 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17568.001.patch, HBASE-17568.002.HBASE-16961.patch > > > (From a TODO) > The RegionServers send reports of sizes from to the Master so the Master can > compute the "size" of each Table. > Presently, the Master has no way to expire these reports. Thus, reports for > tables that are deleted would be retained in memory. Retain the time at which > the report was received by the master, and then use that to determine when to > age it off. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
[ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895299#comment-15895299 ] Lars Hofhansl commented on HBASE-12790: --- Can we restart this discussion? [~jamestaylor] suggests to simply make the queue class pluggable. Phoenix can then inject its own implementation. > Support fairness across parallelized scans > -- > > Key: HBASE-12790 > URL: https://issues.apache.org/jira/browse/HBASE-12790 > Project: HBase > Issue Type: New Feature >Reporter: James Taylor >Assignee: ramkrishna.s.vasudevan > Labels: Phoenix > Attachments: AbstractRoundRobinQueue.java, HBASE-12790_1.patch, > HBASE-12790_5.patch, HBASE-12790_callwrapper.patch, HBASE-12790.patch, > HBASE-12790_trunk_1.patch, PHOENIX_4.5.3-HBase-0.98-2317-SNAPSHOT.zip > > > Some HBase clients parallelize the execution of a scan to reduce latency in > getting back results. This can lead to starvation with a loaded cluster and > interleaved scans, since the RPC queue will be ordered and processed on a > FIFO basis. For example, if there are two clients, A & B that submit largish > scans at the same time. Say each scan is broken down into 100 scans by the > client (broken down into equal depth chunks along the row key), and the 100 > scans of client A are queued first, followed immediately by the 100 scans of > client B. In this case, client B will be starved out of getting any results > back until the scans for client A complete. > One solution to this is to use the attached AbstractRoundRobinQueue instead > of the standard FIFO queue. The queue to be used could be (maybe it already > is) configurable based on a new config parameter. Using this queue would > require the client to have the same identifier for all of the 100 parallel > scans that represent a single logical scan from the clients point of view. > With this information, the round robin queue would pick off a task from the > queue in a round robin fashion (instead of a strictly FIFO manner) to prevent > starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17602) Tweak chore delay/period defaults
[ https://issues.apache.org/jira/browse/HBASE-17602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17602: --- Attachment: HBASE-17602.001.HBASE-16961.patch .001 Switch from 5min period to 1min and 1min initial delay to 15s. Avoids changing the Chore in each RS which scans HDFS to prevent namenode bashing. > Tweak chore delay/period defaults > - > > Key: HBASE-17602 > URL: https://issues.apache.org/jira/browse/HBASE-17602 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Trivial > Fix For: HBASE-16961 > > Attachments: HBASE-17602.001.HBASE-16961.patch > > > In testing, found that the default chore periods/delays were a little too > lax. This resulted in quotas not being updated as soon as they really should > have been. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17703) TestThriftServerCmdLine is flaky in master branch
[
https://issues.apache.org/jira/browse/HBASE-17703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895290#comment-15895290
]
Hudson commented on HBASE-17703:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2607 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2607/])
HBASE-17703 Added random infoport to TestThriftServerCmdLine (tedyu: rev
d3bf246421407dd811ac1c96bc5aa7b7a32dc94a)
* (edit)
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
> TestThriftServerCmdLine is flaky in master branch
> -
>
> Key: HBASE-17703
> URL: https://issues.apache.org/jira/browse/HBASE-17703
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Jan Hentschel
> Fix For: 2.0
>
> Attachments: HBASE-17703.master.001.patch
>
>
> According to
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
> , TestThriftServerCmdLine is the top flaky test.
> It fails in QA run sometimes.
> https://builds.apache.org/job/PreCommit-HBASE-Build/5848/artifact/patchprocess/patch-unit-hbase-thrift.txt
> {code}
> testRunThriftServer[0](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine)
> Time elapsed: 10.057 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.testRunThriftServer(TestThriftServerCmdLine.java:187)
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 60.212 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
> testRunThriftServer[0](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine)
> Time elapsed: 0.166 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServer.checkTableList(TestThriftServer.java:254)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.talkToThriftServer(TestThriftServerCmdLine.java:227)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.testRunThriftServer(TestThriftServerCmdLine.java:190)
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 64.428 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
> testRunThriftServer[0](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine)
> Time elapsed: 0.144 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServer.checkTableList(TestThriftServer.java:254)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.talkToThriftServer(TestThriftServerCmdLine.java:227)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.testRunThriftServer(TestThriftServerCmdLine.java:190)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895291#comment-15895291
]
Hudson commented on HBASE-17722:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2607 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2607/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev 404a2883f2f68f0c2762c0df62a29aebeb7da151)
* (edit) hbase-rest/src/test/resources/log4j.properties
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) hbase-spark/src/test/resources/log4j.properties
* (edit) hbase-common/src/test/resources/log4j.properties
* (edit) conf/log4j.properties
* (edit)
hbase-archetypes/hbase-client-project/src/main/resources/log4j.properties
* (edit) hbase-server/src/test/resources/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
* (edit)
hbase-archetypes/hbase-shaded-client-project/src/main/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17602) Tweak chore delay/period defaults
[ https://issues.apache.org/jira/browse/HBASE-17602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17602: --- Status: Patch Available (was: Open) > Tweak chore delay/period defaults > - > > Key: HBASE-17602 > URL: https://issues.apache.org/jira/browse/HBASE-17602 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Trivial > Fix For: HBASE-16961 > > Attachments: HBASE-17602.001.HBASE-16961.patch > > > In testing, found that the default chore periods/delays were a little too > lax. This resulted in quotas not being updated as soon as they really should > have been. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17602) Tweak chore delay/period defaults
[ https://issues.apache.org/jira/browse/HBASE-17602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17602: --- Fix Version/s: HBASE-16961 > Tweak chore delay/period defaults > - > > Key: HBASE-17602 > URL: https://issues.apache.org/jira/browse/HBASE-17602 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Trivial > Fix For: HBASE-16961 > > > In testing, found that the default chore periods/delays were a little too > lax. This resulted in quotas not being updated as soon as they really should > have been. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17465) [C++] implement request retry mechanism over RPC
[
https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895277#comment-15895277
]
Enis Soztutar commented on HBASE-17465:
---
Also, can you address this warning:
{code}
utils/connection-util.h:23:19: warning: extra tokens at end of #include
directive [-Wextra-tokens]
#include ;
{code}
> [C++] implement request retry mechanism over RPC
>
>
> Key: HBASE-17465
> URL: https://issues.apache.org/jira/browse/HBASE-17465
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HBASE-17465-HBASE-14850.000.patch,
> HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch,
> HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch,
> HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch,
> HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch,
> HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch
>
>
> HBASE-17051 implemented RPC layer. Requests retries will make system
> reliable. This JIRA proposes adding it, which corresponds to similar
> implementation in SingleRequestCallerBuilder (or BatchCallerBuilder,
> ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and
> AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more
> generic, decoupled with HRegionLocation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895275#comment-15895275
]
Hudson commented on HBASE-17722:
FAILURE: Integrated in Jenkins build HBase-1.1-JDK7 #1853 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1853/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev ab9071fa62d61364dd42ed244c3ae5092d5fa8a3)
* (edit) hbase-common/src/test/resources/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) hbase-server/src/test/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
* (edit) conf/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
* (edit) hbase-rest/src/test/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17729) Missing shortcuts for some useful HCD options
[
https://issues.apache.org/jira/browse/HBASE-17729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-17729:
---
Attachment: HBASE-17729.patch
Trivial patch, tested manually
> Missing shortcuts for some useful HCD options
> -
>
> Key: HBASE-17729
> URL: https://issues.apache.org/jira/browse/HBASE-17729
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.3.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: HBASE-17729.patch
>
>
> Missing:
> - CACHE_INDEX_ON_WRITE
> - CACHE_BLOOMS_ON_WRITE
> - EVICT_BLOCKS_ON_CLOSE
> - CACHE_DATA_IN_L1
> It's possible to set them with syntax like
> {noformat}
> { CONFIGURATION = { = } }
> {noformat}
> but let's save some keystrokes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17729) Missing shortcuts for some useful HCD options
[
https://issues.apache.org/jira/browse/HBASE-17729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-17729:
---
Status: Patch Available (was: Open)
> Missing shortcuts for some useful HCD options
> -
>
> Key: HBASE-17729
> URL: https://issues.apache.org/jira/browse/HBASE-17729
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.3.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: HBASE-17729.patch
>
>
> Missing:
> - CACHE_INDEX_ON_WRITE
> - CACHE_BLOOMS_ON_WRITE
> - EVICT_BLOCKS_ON_CLOSE
> - CACHE_DATA_IN_L1
> It's possible to set them with syntax like
> {noformat}
> { CONFIGURATION = { = } }
> {noformat}
> but let's save some keystrokes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17516) Table quota not taking precedence over namespace quota
[ https://issues.apache.org/jira/browse/HBASE-17516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17516: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for your review, Ted! > Table quota not taking precedence over namespace quota > -- > > Key: HBASE-17516 > URL: https://issues.apache.org/jira/browse/HBASE-17516 > Project: HBase > Issue Type: Sub-task >Reporter: Romil Choksi >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17516.001.patch, HBASE-17516.002.HBASE-16961.patch > > > [~romil.choksi] found a bug in the current patch-set where a more restrictive > table quota did not take priority over a less-restrictive namespace quota. > Turns out some of the logic to handle this case was incorrect. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17465) [C++] implement request retry mechanism over RPC
[
https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895264#comment-15895264
]
Enis Soztutar commented on HBASE-17465:
---
bq. I've filed two tickets as follow up work.
Thanks, makes sense.
- Can you run bin/format-code.sh
- And {{make lint}} to address the major lint comments in the new code.
- Remove {{hconstants.h}} and {{.cc}}. We can put the definitions in
{{connection-util.h}} or something.
- Maybe move this logging to be a VLOG?
{code}
LOG(INFO) << "host=" + sn.host_name() + ", port=" +
folly::to(sn.port());
{code}
- You can remove this line:
{code}
// return location_cache_->LocateRegion(*table_name_,
row).get(milliseconds(1000));
{code}
Maybe use one of the operation timeouts or rpc timeouts if available.
- We are not using PB's Service stubs anymore, right? So this may not be
needed:
{code}
hbase-native-client/if/Client.proto
{code}
Also maybe we should remove the RpcChannelImpl. Can be done as a follow up.
- Remove commented out code:
{code}
// folly::Future Get(const hbase::Get&);
{code}
- We do not really need {{RpcControllerFactory}}. You are mocking it, or
getting it via the Connection, but we might as well, just hard code
HBaseRpcController.
- Are we using the RpcController now with PB stubs gone? Do you think that it
is useful for later? We can purge it otherwise.
- Do we need {{template typename RPC_CLIENT}}. I think it is always RpcClient
even in the mocks. Maybe you added that because we may want to be able to mock
the whole RPC layer later? Not a big issue if we leave it there.
> [C++] implement request retry mechanism over RPC
>
>
> Key: HBASE-17465
> URL: https://issues.apache.org/jira/browse/HBASE-17465
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HBASE-17465-HBASE-14850.000.patch,
> HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch,
> HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch,
> HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch,
> HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch,
> HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch
>
>
> HBASE-17051 implemented RPC layer. Requests retries will make system
> reliable. This JIRA proposes adding it, which corresponds to similar
> implementation in SingleRequestCallerBuilder (or BatchCallerBuilder,
> ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and
> AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more
> generic, decoupled with HRegionLocation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17701) Add HadoopAuthFilterInitializer to use hadoop-auth AuthenticationFilter for hbase web ui
[ https://issues.apache.org/jira/browse/HBASE-17701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895259#comment-15895259 ] Josh Elser commented on HBASE-17701: bq. If HBase uses AuthenticationFilter, HBase will load the hadoop.http.authentication.* configurations from core-site.xml, in that way, the HBase ui can be authenticated by the way hadoop does. I don't think we want to tie HBase to the same configuration properties that Hadoop uses. That seems very brittle as we require Hadoop's configuration files on the classpath. I would think that we would want our own configuration properties. Anyways, if you can be more specific about the kind of authentication you think is missing (I don't know what token authentication means at a glance), and write some tests to show it working, that would be a good addition. > Add HadoopAuthFilterInitializer to use hadoop-auth AuthenticationFilter for > hbase web ui > > > Key: HBASE-17701 > URL: https://issues.apache.org/jira/browse/HBASE-17701 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 1.2.4 >Reporter: Pan Yuxuan > Attachments: HBASE-17701.v1.patch > > > The HBase web UI is none secure by default, there is only one > StaticUserWebFilter for a fake user. > For Hadoop, we already have AuthenticationFilter for web authentication based > on token or kerberos. So I think hbase can reuse the hadoop-auth > AuthenticationFilter by adding a HadoopAuthFilterInitializer. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17680) Run mini cluster through JNI in tests
[ https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17680: --- Attachment: 17680.v28.txt Patch v28 removes lint warnings introduced by the previous patch. > Run mini cluster through JNI in tests > - > > Key: HBASE-17680 > URL: https://issues.apache.org/jira/browse/HBASE-17680 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17680.v14.txt, 17680.v17.txt, 17680.v18.txt, > 17680.v1.txt, 17680.v20.txt, 17680.v22.txt, 17680.v23.txt, 17680.v26.txt, > 17680.v27.txt, 17680.v28.txt, 17680.v3.txt, 17680.v8.txt > > > Currently tests start local hbase cluster through hbase shell. > There is less control over the configuration of the local cluster this way. > This issue would replace hbase shell with JNI interface to mini cluster. > We would have full control over the cluster behavior. > Thanks to [~devaraj] who started this initiative. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895244#comment-15895244
]
Hudson commented on HBASE-17722:
SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #1937 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1937/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev ab9071fa62d61364dd42ed244c3ae5092d5fa8a3)
* (edit) conf/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
* (edit) hbase-server/src/test/resources/log4j.properties
* (edit) hbase-common/src/test/resources/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) hbase-rest/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17729) Missing shortcuts for some useful HCD options
Andrew Purtell created HBASE-17729:
--
Summary: Missing shortcuts for some useful HCD options
Key: HBASE-17729
URL: https://issues.apache.org/jira/browse/HBASE-17729
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 1.3.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Trivial
Fix For: 2.0.0, 1.4.0, 1.3.1
Missing:
- CACHE_INDEX_ON_WRITE
- CACHE_BLOOMS_ON_WRITE
- EVICT_BLOCKS_ON_CLOSE
- CACHE_DATA_IN_L1
It's possible to set them with syntax like
{noformat}
{ CONFIGURATION = { = } }
{noformat}
but let's save some keystrokes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17516) Table quota not taking precedence over namespace quota
[ https://issues.apache.org/jira/browse/HBASE-17516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895238#comment-15895238 ] Ted Yu commented on HBASE-17516: Assuming quotas test suite are green, I am +1 > Table quota not taking precedence over namespace quota > -- > > Key: HBASE-17516 > URL: https://issues.apache.org/jira/browse/HBASE-17516 > Project: HBase > Issue Type: Sub-task >Reporter: Romil Choksi >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17516.001.patch, HBASE-17516.002.HBASE-16961.patch > > > [~romil.choksi] found a bug in the current patch-set where a more restrictive > table quota did not take priority over a less-restrictive namespace quota. > Turns out some of the logic to handle this case was incorrect. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17516) Table quota not taking precedence over namespace quota
[ https://issues.apache.org/jira/browse/HBASE-17516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895234#comment-15895234 ] Josh Elser commented on HBASE-17516: You OK with this one, [~tedyu]? > Table quota not taking precedence over namespace quota > -- > > Key: HBASE-17516 > URL: https://issues.apache.org/jira/browse/HBASE-17516 > Project: HBase > Issue Type: Sub-task >Reporter: Romil Choksi >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17516.001.patch, HBASE-17516.002.HBASE-16961.patch > > > [~romil.choksi] found a bug in the current patch-set where a more restrictive > table quota did not take priority over a less-restrictive namespace quota. > Turns out some of the logic to handle this case was incorrect. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17002) [Master] Report quotas and active violations on Master UI and JMX metrics
[ https://issues.apache.org/jira/browse/HBASE-17002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895232#comment-15895232 ] Josh Elser commented on HBASE-17002: Also forgot to mention, this is now blocked on HBASE-17516. Need that fix for the new test to pass. > [Master] Report quotas and active violations on Master UI and JMX metrics > - > > Key: HBASE-17002 > URL: https://issues.apache.org/jira/browse/HBASE-17002 > Project: HBase > Issue Type: Sub-task > Components: master, UI >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17002.001.HBASE-16961.patch, > HBASE-17002.002.HBASE-16961.patch, TableQuotaDefined2.png, > TableQuotaDefined.png > > > We should be able to view the Master UI and determine what quotas exist and > the quotas that are in violation to easily confirm/deny the state of > tables/namespaces. > The JMX metrics from the Master should also include the list of > tables/namespaces which have quotas whose violation policies are presently > being enforced. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17002) [Master] Report quotas and active violations on Master UI and JMX metrics
[ https://issues.apache.org/jira/browse/HBASE-17002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17002: --- Attachment: TableQuotaDefined2.png HBASE-17002.002.HBASE-16961.patch .002 Adds a bit more nice info to the table.jsp page (size from quota calculations and violation/observance of quota). !TableQuotaDefined2.png! > [Master] Report quotas and active violations on Master UI and JMX metrics > - > > Key: HBASE-17002 > URL: https://issues.apache.org/jira/browse/HBASE-17002 > Project: HBase > Issue Type: Sub-task > Components: master, UI >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: HBASE-16961 > > Attachments: HBASE-17002.001.HBASE-16961.patch, > HBASE-17002.002.HBASE-16961.patch, TableQuotaDefined2.png, > TableQuotaDefined.png > > > We should be able to view the Master UI and determine what quotas exist and > the quotas that are in violation to easily confirm/deny the state of > tables/namespaces. > The JMX metrics from the Master should also include the list of > tables/namespaces which have quotas whose violation policies are presently > being enforced. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17723) TestServerSideScanMetricsFromClientSide and TestRecoveredEdits fail
[ https://issues.apache.org/jira/browse/HBASE-17723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895224#comment-15895224 ] CHIA-PING TSAI commented on HBASE-17723: HBASE-17662 had fixed the TestRecoveredEdits > TestServerSideScanMetricsFromClientSide and TestRecoveredEdits fail > --- > > Key: HBASE-17723 > URL: https://issues.apache.org/jira/browse/HBASE-17723 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 2.0.0 > Environment: Ubuntu 14.04 LTS / x86_64 / 16GB RAM >Reporter: Anup Halarnkar > > Command to build and test: > mvn clean install -X > JAVA_TOOL_OPTIONS: -Xmx4G -XX:MaxPermSize=4G -Xms1G > Failure in tests: > Results : > Failed tests: > > TestServerSideScanMetricsFromClientSide.testRowsSeenMetricWithAsync:161->testRowsSeenMetric:174->testRowsSeenMetric:204->testMetric:330 > Metric: ROWS_SCANNED Expected: 9 Actual: 8 expected:<9> but was:<8> > > TestRecoveredEdits.testReplayWorksThoughLotsOfFlushing:78->testReplayWorksWithMemoryCompactionPolicy:149->verifyAllEditsMadeItIn:195 > Failed to find > W\xE1X\xDBv\xD8}\x07\xBF\xC1I\xF1\xF5\xB9\x84\xD8/meta:prev/1422491946444/Put/vlen=16/seqid=0 > Tests run: 1761, Failures: 2, Errors: 0, Skipped: 6 > > This is the output for 1st test failure: > --- > Test set: org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide > --- > Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.971 sec <<< > FAILURE! - in org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide > testRowsSeenMetricWithAsync(org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide) > Time elapsed: 1.134 sec <<< FAILURE! > java.lang.AssertionError: Metric: ROWS_SCANNED Expected: 9 Actual: 8 > expected:<9> but was:<8> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at > org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide.testMetric(TestServerSideScanMetricsFromClientSide.java:330) > at > org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide.testRowsSeenMetric(TestServerSideScanMetricsFromClientSide.java:204) > at > org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide.testRowsSeenMetric(TestServerSideScanMetricsFromClientSide.java:174) > at > org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide.testRowsSeenMetricWithAsync(TestServerSideScanMetricsFromClientSide.java:161) > > This is output of 2nd test failure: > --- > Test set: org.apache.hadoop.hbase.regionserver.TestRecoveredEdits > --- > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.911 sec > <<< FAILURE! - in org.apache.hadoop.hbase.regionserver.TestRecoveredEdits > testReplayWorksThoughLotsOfFlushing(org.apache.hadoop.hbase.regionserver.TestRecoveredEdits) > Time elapsed: 11.871 sec <<< FAILURE! > java.lang.AssertionError: Failed to find > W\xE1X\xDBv\xD8}\x07\xBF\xC1I\xF1\xF5\xB9\x84\xD8/meta:prev/1422491946444/Put/vlen=16/seqid=0 > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.assertTrue(Assert.java:41) > at > org.apache.hadoop.hbase.regionserver.TestRecoveredEdits.verifyAllEditsMadeItIn(TestRecoveredEdits.java:195) > at > org.apache.hadoop.hbase.regionserver.TestRecoveredEdits.testReplayWorksWithMemoryCompactionPolicy(TestRecoveredEdits.java:149) > at > org.apache.hadoop.hbase.regionserver.TestRecoveredEdits.testReplayWorksThoughLotsOfFlushing(TestRecoveredEdits.java:78) > Please let me know if you need more information > Thanks, > Anup -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17623) Reuse the bytes array when building the hfile block
[
https://issues.apache.org/jira/browse/HBASE-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
CHIA-PING TSAI updated HBASE-17623:
---
Status: Patch Available (was: Open)
> Reuse the bytes array when building the hfile block
> ---
>
> Key: HBASE-17623
> URL: https://issues.apache.org/jira/browse/HBASE-17623
> Project: HBase
> Issue Type: Improvement
>Reporter: CHIA-PING TSAI
>Assignee: CHIA-PING TSAI
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after(snappy_hfilesize=5.04GB).png,
> after(snappy_hfilesize=755MB).png, before(snappy_hfilesize=5.04GB).png,
> before(snappy_hfilesize=755MB).png, HBASE-17623.branch-1.v0.patch,
> HBASE-17623.branch-1.v1.patch, HBASE-17623.branch-1.v2.patch,
> HBASE-17623.branch-1.v2.patch, HBASE-17623.v0.patch, HBASE-17623.v1.patch,
> HBASE-17623.v1.patch, HBASE-17623.v2.patch, memory allocation measurement.xlsx
>
>
> There are two improvements.
> # The onDiskBlockBytesWithHeader should maintain a bytes array which can be
> reused when building the hfile.
> # The onDiskBlockBytesWithHeader is copied to an new bytes array only when we
> need to cache the block.
> # If no block need to be cached, the uncompressedBlockBytesWithHeader will
> never be created.
> {code:title=HFileBlock.java|borderStyle=solid}
> private void finishBlock() throws IOException {
> if (blockType == BlockType.DATA) {
> this.dataBlockEncoder.endBlockEncoding(dataBlockEncodingCtx,
> userDataStream,
> baosInMemory.getBuffer(), blockType);
> blockType = dataBlockEncodingCtx.getBlockType();
> }
> userDataStream.flush();
> // This does an array copy, so it is safe to cache this byte array when
> cache-on-write.
> // Header is still the empty, 'dummy' header that is yet to be filled
> out.
> uncompressedBlockBytesWithHeader = baosInMemory.toByteArray();
> prevOffset = prevOffsetByType[blockType.getId()];
> // We need to set state before we can package the block up for
> cache-on-write. In a way, the
> // block is ready, but not yet encoded or compressed.
> state = State.BLOCK_READY;
> if (blockType == BlockType.DATA || blockType == BlockType.ENCODED_DATA)
> {
> onDiskBlockBytesWithHeader = dataBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> } else {
> onDiskBlockBytesWithHeader = defaultBlockEncodingCtx.
> compressAndEncrypt(uncompressedBlockBytesWithHeader);
> }
> // Calculate how many bytes we need for checksum on the tail of the
> block.
> int numBytes = (int) ChecksumUtil.numBytes(
> onDiskBlockBytesWithHeader.length,
> fileContext.getBytesPerChecksum());
> // Put the header for the on disk bytes; header currently is
> unfilled-out
> putHeader(onDiskBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> // Set the header for the uncompressed bytes (for cache-on-write) --
> IFF different from
> // onDiskBlockBytesWithHeader array.
> if (onDiskBlockBytesWithHeader != uncompressedBlockBytesWithHeader) {
> putHeader(uncompressedBlockBytesWithHeader, 0,
> onDiskBlockBytesWithHeader.length + numBytes,
> uncompressedBlockBytesWithHeader.length,
> onDiskBlockBytesWithHeader.length);
> }
> if (onDiskChecksum.length != numBytes) {
> onDiskChecksum = new byte[numBytes];
> }
> ChecksumUtil.generateChecksums(
> onDiskBlockBytesWithHeader, 0, onDiskBlockBytesWithHeader.length,
> onDiskChecksum, 0, fileContext.getChecksumType(),
> fileContext.getBytesPerChecksum());
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895198#comment-15895198
]
Hudson commented on HBASE-17722:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #2 (See
[https://builds.apache.org/job/HBase-1.3-IT/2/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev 51dfc1463336841095b83ab96c8903e3ac0bbbe2)
* (edit) hbase-rest/src/test/resources/log4j.properties
* (edit)
hbase-archetypes/hbase-shaded-client-project/src/main/resources/log4j.properties
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
* (edit) hbase-server/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit)
hbase-archetypes/hbase-client-project/src/main/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
* (edit) conf/log4j.properties
* (edit) hbase-common/src/test/resources/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895188#comment-15895188
]
Hudson commented on HBASE-17722:
SUCCESS: Integrated in Jenkins build HBase-1.2-IT #843 (See
[https://builds.apache.org/job/HBase-1.2-IT/843/])
HBASE-17722 Metrics subsystem stop/start messages add a lot of useless
(apurtell: rev 5ab33b3cd77d6e1946fea00c9e8e75a5836e8184)
* (edit) conf/log4j.properties
* (edit) hbase-examples/src/test/resources/log4j.properties
* (edit) hbase-common/src/test/resources/log4j.properties
* (edit) hbase-shell/src/test/resources/log4j.properties
* (edit) hbase-prefix-tree/src/test/resources/log4j.properties
* (edit) hbase-rest/src/test/resources/log4j.properties
* (edit) hbase-procedure/src/test/resources/log4j.properties
* (edit) hbase-client/src/test/resources/log4j.properties
* (edit) hbase-server/src/test/resources/log4j.properties
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kahlil Oppenheimer updated HBASE-17706: --- Status: Patch Available (was: Open) > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895168#comment-15895168 ] Kahlil Oppenheimer commented on HBASE-17706: Just fixed line lengths! > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kahlil Oppenheimer updated HBASE-17706: --- Status: Open (was: Patch Available) > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kahlil Oppenheimer updated HBASE-17706: --- Attachment: HBASE-17706-05.patch > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895146#comment-15895146 ] Xiaobing Zhou commented on HBASE-17465: --- I've filed two tickets as follow up work. HBASE-17726 and HBASE-17727 > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch, > HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895099#comment-15895099
]
Ted Yu commented on HBASE-17706:
Almost there.
{code}
+ // if old server had max num regions, assume (for now) max num regions
went down since we moved the region
{code}
Wrap long lines (there is one in test).
Limit to 100 characters per line.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>Reporter: Kahlil Oppenheimer
>Assignee: Kahlil Oppenheimer
>Priority: Minor
> Labels: patch
> Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch,
> HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706.patch
>
>
> We noticed while running unit tests that the TableSkewCostFunction computed
> cost did not change as the balancer ran and simulated moves across the
> cluster. After investigating, we found that this happened in particular when
> the cluster started out with at least one table very strongly skewed.
> We noticed that the TableSkewCostFunction depends on a field of the
> BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field
> is not properly maintained as regionMoves are simulated for the cluster. The
> field only ever increases as the maximum number of regions per table
> increases, but it does not decrease as the maximum number per table goes down.
> This patch corrects that behavior so that the field is accurately maintained,
> and thus the TableSkewCostFunction produces a more correct value as the
> balancer runs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kahlil Oppenheimer updated HBASE-17706: --- Status: Patch Available (was: Open) > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kahlil Oppenheimer updated HBASE-17706: --- Attachment: HBASE-17706-04.patch > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kahlil Oppenheimer updated HBASE-17706: --- Status: Open (was: Patch Available) > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17709) New MoveCostFunction that respects MaxMovePercent
[
https://issues.apache.org/jira/browse/HBASE-17709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895082#comment-15895082
]
Hadoop QA commented on HBASE-17709:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 29s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 33s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 150m 32s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855946/HBASE-17709-02.patch |
| JIRA Issue | HBASE-17709 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux dc279da1eadc 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 678ad0e |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5936/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5936/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5936/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5936/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> New MoveCostFunction that respects MaxMovePercent
> -
>
> Key:
[jira] [Updated] (HBASE-17728) Separate Configuration into common module
[ https://issues.apache.org/jira/browse/HBASE-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17728: --- Attachment: 17728.v1.txt Ran 'buck test --all' Tests passed. [~enis] [~xiaobingo]: Can you take a look ? > Separate Configuration into common module > - > > Key: HBASE-17728 > URL: https://issues.apache.org/jira/browse/HBASE-17728 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17728.v1.txt > > > User is in security module. > When User::isSecurityEnabled() is added, we need to query Configuration for > security setting. > However, this introduces a circular build dependency: > BUILD FAILED: Cycle found: //connection:connection -> //security:security -> > //core:core -> //connection:connection > This issue is to separate Configuration into common module which is depended > upon by both core and connection modules. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17465) [C++] implement request retry mechanism over RPC
[
https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895057#comment-15895057
]
Hadoop QA commented on HBASE-17465:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m 56s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 9s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 2s
{color} | {color:green} HBASE-14850 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} HBASE-14850 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 2s
{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m 2s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m 4s
{color} | {color:red} The patch generated 1 new + 436 unchanged - 1 fixed = 437
total (was 437) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 56s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 14s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2017-03-03 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855963/HBASE-17465-HBASE-14850.010.patch
|
| JIRA Issue | HBASE-17465 |
| Optional Tests | shellcheck shelldocs cc compile hbaseprotoc |
| uname | Linux 0ffc29fab1d5 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | nobuild |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-14850 / 7d284f6 |
| shellcheck | v0.4.5 |
| shellcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/5937/artifact/patchprocess/diff-patch-shellcheck.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/5937/artifact/patchprocess/whitespace-eol.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5937/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> [C++] implement request retry mechanism over RPC
>
>
> Key: HBASE-17465
> URL: https://issues.apache.org/jira/browse/HBASE-17465
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HBASE-17465-HBASE-14850.000.patch,
> HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch,
> HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch,
> HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch,
> HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch,
> HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch
>
>
> HBASE-17051 implemented RPC layer. Requests retries will make system
> reliable. This JIRA proposes adding it, which corresponds to similar
> implementation in SingleRequestCallerBuilder (or BatchCallerBuilder,
> ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and
> AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more
> generic, decoupled with HRegionLocation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17728) Separate Configuration into common module
Ted Yu created HBASE-17728: -- Summary: Separate Configuration into common module Key: HBASE-17728 URL: https://issues.apache.org/jira/browse/HBASE-17728 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu User is in security module. When User::isSecurityEnabled() is added, we need to query Configuration for security setting. However, this introduces a circular build dependency: BUILD FAILED: Cycle found: //connection:connection -> //security:security -> //core:core -> //connection:connection This issue is to separate Configuration into common module which is depended upon by both core and connection modules. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17727) [C++] Make RespConverter work with MockRawAsyncTableImpl
[
https://issues.apache.org/jira/browse/HBASE-17727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HBASE-17727:
--
Description:
This is a follow up work of HBASE-17465.
There's a problem to dereference instance of RpcCallback when it's passed as
function argument.
{code}
template
using RespConverter = std::function;
{code}
{code}
template
folly::Future Call(
std::shared_ptr rpc_client,
std::shared_ptr controller,
std::shared_ptr loc,
const REQ& req,
const ReqConverter&
req_converter,
const hbase::RpcCall& rpc_call,
const RespConverter& resp_converter) {
rpc_call(
rpc_client,
loc,
controller,
std::move(req_converter(req, loc->region_name(
.then([&, this](std::unique_ptr presp) {
// std::unique_ptr result = resp_converter(presp);
std::unique_ptr result =
hbase::ResponseConverter::FromGetResponse(*presp);
promise_->setValue(std::move(*result));
})
.onError([this] (const std::exception& e) {promise_->setException(e);});
return promise_->getFuture();
}
{code}
was:
This is a follow up work of HBASE-17465.
There's a problem to dereference instance of RpcCallback when it's passed as
function argument.
{code}
template
using RespConverter = std::function;
{code}
{code}
template
folly::Future Call(
std::shared_ptr rpc_client,
std::shared_ptr controller,
std::shared_ptr loc,
const REQ& req,
const ReqConverter&
req_converter,
const hbase::RpcCall& rpc_call,
const RespConverter& resp_converter) {
rpc_call(
rpc_client,
loc,
controller,
std::move(req_converter(req, loc->region_name(
.then([&, this](std::unique_ptr presp) {
// std::unique_ptr result = resp_converter(presp);
std::unique_ptr result =
hbase::ResponseConverter::FromGetResponse(*presp);
promise_->setValue(std::move(*result));
})
.onError([this] (const std::exception& e) {promise_->setException(e);});
return promise_->getFuture();
}
{code}
> [C++] Make RespConverter work with MockRawAsyncTableImpl
>
>
> Key: HBASE-17727
> URL: https://issues.apache.org/jira/browse/HBASE-17727
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
>
> This is a follow up work of HBASE-17465.
> There's a problem to dereference instance of RpcCallback when it's passed as
> function argument.
> {code}
> template typename S>
> using RespConverter = std::function;
> {code}
> {code}
> templatetypename PREQ,
>typename PRESP,
>typename RESP>
> folly::Future Call(
> std::shared_ptr rpc_client,
> std::shared_ptr controller,
> std::shared_ptr loc,
> const REQ& req,
> const ReqConverter&
> req_converter,
> const hbase::RpcCall& rpc_call,
> const RespConverter& resp_converter) {
> rpc_call(
> rpc_client,
> loc,
> controller,
> std::move(req_converter(req, loc->region_name(
> .then([&, this](std::unique_ptr presp) {
> // std::unique_ptr result = resp_converter(presp);
> std::unique_ptr result =
> hbase::ResponseConverter::FromGetResponse(*presp);
> promise_->setValue(std::move(*result));
> })
> .onError([this] (const std::exception& e)
> {promise_->setException(e);});
> return promise_->getFuture();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17716) Formalize Scan Metric names
[ https://issues.apache.org/jira/browse/HBASE-17716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895019#comment-15895019 ] stack commented on HBASE-17716: --- bq. Are the scan metrics in MBeans for all the scans or are they per scan level? All scans. Not per scan for sure. You want per scan (I suppose so, yeah, makes sense). What is weird here is that this metrics is NOT available via JMX. bq. We could have metric names defined as public static final String too with their declarations annotated with the @Metric annotation to tell HBase developers not to change these names This seems like a good idea to me. It'd be cleanup on our part recognizing that metric names are not internals-only. On @Metric, chatting w/ coworker, how about we just used our current InterfaceAudience annotations? A metric name is probably inside a class that is stamped as Private. We'd annotate the static final name as InterfaceAudience.Public. I could add a type so we could actually write @InterfaceAudience.Public(HBaseInterfaceAudience.METRIC) so it would be clear why we have this String in the middle of an internal class as public. My coworker ([~jmhsieh]) suggested that not all metrics would get this stamping, just those we know for sure should never change -- the ones folks depend on -- and other metrics might get a LimitedPrivate stamping (or as is currently allowed, no stamping means PRIVATE). Would something like this work [~samarth.j...@gmail.com]? If so, I'll go shop it up on dev list and then we can change this patch to implement? I'll skip the enum thing. There are benefits but minimal and we could attack these in another issue? > Formalize Scan Metric names > --- > > Key: HBASE-17716 > URL: https://issues.apache.org/jira/browse/HBASE-17716 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Karan Mehta >Assignee: Karan Mehta >Priority: Minor > Attachments: HBASE-17716.patch > > > HBase provides various metrics through the API's exposed by ScanMetrics > class. > The JIRA PHOENIX-3248 requires them to be surfaced through the Phoenix > Metrics API. Currently these metrics are referred via hard-coded strings, > which are not formal and can break the Phoenix API. Hence we need to refactor > the code to assign enums for these metrics. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17727) [C++] Make RespConverter work with MockRawAsyncTableImpl
Xiaobing Zhou created HBASE-17727:
-
Summary: [C++] Make RespConverter work with MockRawAsyncTableImpl
Key: HBASE-17727
URL: https://issues.apache.org/jira/browse/HBASE-17727
Project: HBase
Issue Type: Sub-task
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
This is a follow up work of HBASE-17465.
There's a problem to dereference instance of RpcCallback when it's passed as
function argument.
{code}
template
using RespConverter = std::function;
{code}
{code}
template
folly::Future Call(
std::shared_ptr rpc_client,
std::shared_ptr controller,
std::shared_ptr loc,
const REQ& req,
const ReqConverter&
req_converter,
const hbase::RpcCall& rpc_call,
const RespConverter& resp_converter) {
rpc_call(
rpc_client,
loc,
controller,
std::move(req_converter(req, loc->region_name(
.then([&, this](std::unique_ptr presp) {
// std::unique_ptr result = resp_converter(presp);
std::unique_ptr result =
hbase::ResponseConverter::FromGetResponse(*presp);
promise_->setValue(std::move(*result));
})
.onError([this] (const std::exception& e) {promise_->setException(e);});
return promise_->getFuture();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895015#comment-15895015
]
Hadoop QA commented on HBASE-17706:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 20s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 107m 33s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 147m 31s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855936/HBASE-17706-03.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e307a5cbe1e0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 678ad0e |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/5934/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5934/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5934/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
>
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Attachment: HBASE-17465-HBASE-14850.010.patch > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch, > HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Attachment: (was: HBASE-17465-HBASE-14850.011.patch) > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch, > HBASE-17465-HBASE-14850.009.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17726) [C++] Move implementation from header to cc for request retry
Xiaobing Zhou created HBASE-17726: - Summary: [C++] Move implementation from header to cc for request retry Key: HBASE-17726 URL: https://issues.apache.org/jira/browse/HBASE-17726 Project: HBase Issue Type: Sub-task Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou This is a follow up work related to HBASE-17465. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17725) LimitedPrivate COPROC / PHOENIX interface broken on maintenance release
[
https://issues.apache.org/jira/browse/HBASE-17725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895001#comment-15895001
]
Sean Busbey commented on HBASE-17725:
-
[VOTE thread for
1.1.8|https://lists.apache.org/thread.html/18127e20750955d91dae8d6a079c4814a18600dd73a41bbff7d82159@%3Cdev.hbase.apache.org%3E],
both [~ndimiduk] and I said the change looked fine.
I think this means probably things are fine as is. I'll try to ping phoenix for
impact so we have a better idea on how likely this is to break downstream
coprocessor writers.
> LimitedPrivate COPROC / PHOENIX interface broken on maintenance release
> ---
>
> Key: HBASE-17725
> URL: https://issues.apache.org/jira/browse/HBASE-17725
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.2.5, 1.1.8
>Reporter: Sean Busbey
>Priority: Blocker
> Fix For: 1.2.5
>
> Attachments: 1.2.4_1.2.5RC0_compat_report.html
>
>
> The compatibility report for HBase 1.2.5 RC0 flagged one item, the interface
> {{RpcServiceInterface}} added a new method {{setRsRpcServices}}. That
> interface is labeled {{IA.LimitedPrivate(COPROC, PHOENIX)}}.
> AFAICT this change came in on HBASE-16972. There's discussion on that ticket
> about the compatibility impact of the change in logging on earlier releases
> (which I remain fine with). The only discussion about binary/source
> compatibility expressly changes a different part of the patch to avoid
> breaking coprocessors and phoenix.
> As an extra wrinkle, this change has already been published in the 1.1.z
> release line for 1.1.8 and 1.1.9. Apologies, but I haven't gone to verify
> there wasn't a discussion of this break when 1.1.8 came out.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894996#comment-15894996 ] Xiaobing Zhou commented on HBASE-17465: --- v10 did rebase. > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch, > HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Status: Patch Available (was: Open) > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch, > HBASE-17465-HBASE-14850.009.patch, HBASE-17465-HBASE-14850.010.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17725) LimitedPrivate COPROC / PHOENIX interface broken on maintenance release
[
https://issues.apache.org/jira/browse/HBASE-17725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-17725:
Attachment: 1.2.4_1.2.5RC0_compat_report.html
> LimitedPrivate COPROC / PHOENIX interface broken on maintenance release
> ---
>
> Key: HBASE-17725
> URL: https://issues.apache.org/jira/browse/HBASE-17725
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.2.5, 1.1.8
>Reporter: Sean Busbey
>Priority: Blocker
> Fix For: 1.2.5
>
> Attachments: 1.2.4_1.2.5RC0_compat_report.html
>
>
> The compatibility report for HBase 1.2.5 RC0 flagged one item, the interface
> {{RpcServiceInterface}} added a new method {{setRsRpcServices}}. That
> interface is labeled {{IA.LimitedPrivate(COPROC, PHOENIX)}}.
> AFAICT this change came in on HBASE-16972. There's discussion on that ticket
> about the compatibility impact of the change in logging on earlier releases
> (which I remain fine with). The only discussion about binary/source
> compatibility expressly changes a different part of the patch to avoid
> breaking coprocessors and phoenix.
> As an extra wrinkle, this change has already been published in the 1.1.z
> release line for 1.1.8 and 1.1.9. Apologies, but I haven't gone to verify
> there wasn't a discussion of this break when 1.1.8 came out.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Attachment: HBASE-17465-HBASE-14850.011.patch > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch, HBASE-17465-HBASE-14850.008.patch, > HBASE-17465-HBASE-14850.009.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17725) LimitedPrivate COPROC / PHOENIX interface broken on maintenance release
Sean Busbey created HBASE-17725:
---
Summary: LimitedPrivate COPROC / PHOENIX interface broken on
maintenance release
Key: HBASE-17725
URL: https://issues.apache.org/jira/browse/HBASE-17725
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 1.1.8, 1.2.5
Reporter: Sean Busbey
Priority: Blocker
Fix For: 1.2.5
The compatibility report for HBase 1.2.5 RC0 flagged one item, the interface
{{RpcServiceInterface}} added a new method {{setRsRpcServices}}. That interface
is labeled {{IA.LimitedPrivate(COPROC, PHOENIX)}}.
AFAICT this change came in on HBASE-16972. There's discussion on that ticket
about the compatibility impact of the change in logging on earlier releases
(which I remain fine with). The only discussion about binary/source
compatibility expressly changes a different part of the patch to avoid breaking
coprocessors and phoenix.
As an extra wrinkle, this change has already been published in the 1.1.z
release line for 1.1.8 and 1.1.9. Apologies, but I haven't gone to verify there
wasn't a discussion of this break when 1.1.8 came out.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-17722:
---
Attachment: HBASE-17722.patch
Attaching what I committed, with [~jerryhe]'s suggested changes
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-17722:
---
Resolution: Fixed
Fix Version/s: 1.1.10
1.2.5
1.3.1
Status: Resolved (was: Patch Available)
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-17722.patch, HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17703) TestThriftServerCmdLine is flaky in master branch
[
https://issues.apache.org/jira/browse/HBASE-17703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-17703:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Jan.
> TestThriftServerCmdLine is flaky in master branch
> -
>
> Key: HBASE-17703
> URL: https://issues.apache.org/jira/browse/HBASE-17703
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Jan Hentschel
> Fix For: 2.0
>
> Attachments: HBASE-17703.master.001.patch
>
>
> According to
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
> , TestThriftServerCmdLine is the top flaky test.
> It fails in QA run sometimes.
> https://builds.apache.org/job/PreCommit-HBASE-Build/5848/artifact/patchprocess/patch-unit-hbase-thrift.txt
> {code}
> testRunThriftServer[0](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine)
> Time elapsed: 10.057 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.testRunThriftServer(TestThriftServerCmdLine.java:187)
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 60.212 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
> testRunThriftServer[0](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine)
> Time elapsed: 0.166 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServer.checkTableList(TestThriftServer.java:254)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.talkToThriftServer(TestThriftServerCmdLine.java:227)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.testRunThriftServer(TestThriftServerCmdLine.java:190)
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 64.428 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
> testRunThriftServer[0](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine)
> Time elapsed: 0.144 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServer.checkTableList(TestThriftServer.java:254)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.talkToThriftServer(TestThriftServerCmdLine.java:227)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.testRunThriftServer(TestThriftServerCmdLine.java:190)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17724) Include a list of all metrics in the refguide
stack created HBASE-17724: - Summary: Include a list of all metrics in the refguide Key: HBASE-17724 URL: https://issues.apache.org/jira/browse/HBASE-17724 Project: HBase Issue Type: Improvement Components: documentation Reporter: stack It keeps coming up that we are missing a list of all metrics in our documentation. How could we get full list? Metrics are published on JMX. A jsp page scrapes our published mbeans to draw a json dump of metrics when you browse to /jmx. JMX is the source of truth regards the complete set published. How to go from jmx to a page in the refguide? Seems like you have to start up an instance just to query it. Could be done as part of the book build or as preamble to the book build. So, we could scrape jmx mbeans or /jmx json dump. But then we'd have to get it into a form asciidoc could digest. Haven't dug in. Let me make this a beginner issue. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17721) Provide streaming APIs with SSL/TLS
[ https://issues.apache.org/jira/browse/HBASE-17721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-17721: Component/s: security rpc > Provide streaming APIs with SSL/TLS > --- > > Key: HBASE-17721 > URL: https://issues.apache.org/jira/browse/HBASE-17721 > Project: HBase > Issue Type: Umbrella > Components: rpc, security >Reporter: Alex Araujo >Assignee: Alex Araujo > Fix For: 2.0.0 > > > Umbrella to add optional client/server streaming capabilities to HBase. > This would allow bandwidth to be used more efficiently for certain > operations, and allow clients to use SSL/TLS for authentication and > encryption. > Desired client/server scaffolding: > - HTTP/2 support > - Protocol negotiation (blocking vs streaming, auth, encryption, etc.) > - TLS/SSL support > - Streaming RPC support > Possibilities (and their tradeoffs): > - gRPC: Some initial work and discussion on HBASE-13467 (Prototype using GRPC > as IPC mechanism) > -- Has most or all of the desired scaffolding > -- Adds additional g* dependencies. Compat story for g* dependencies not > always ideal > - Custom HTTP/2 based client/server APIs > -- More control over compat story > -- Non-trivial to build scaffolding; might reinvent wheels along the way > - Others? > Related Jiras that might be rolled in as sub-tasks (or closed/replaced with > new ones): > HBASE-17708 (Expose config to set two-way auth over TLS in HttpServer and add > a test) > HBASE-8691 (High-Throughput Streaming Scan API) > HBASE-14899 (Create custom Streaming ReplicationEndpoint) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[
https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894955#comment-15894955
]
Sean Busbey commented on HBASE-17722:
-
+1
bq. We should ideally vet log output changes while reviewing patches, so as to
not even have to do this afterwards, like now?
This is something I look at when doing reviews. Happy to see an increase in
others doing it as well. Maybe worth a push on HBASE-12396 if other folks are
interested?
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging
> --
>
> Key: HBASE-17722
> URL: https://issues.apache.org/jira/browse/HBASE-17722
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.3.0, 1.2.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17722.patch
>
>
> Metrics subsystem stop/start messages add a lot of useless bulk to
> operational logging. Say you are collecting logs from a fleet of thousands of
> servers and want to have them around for ~month or longer. It adds up.
> I think these should at least be at DEBUG level and ideally at TRACE. They
> don't offer much utility. Unfortunately they are Hadoop classes so we can
> tweak log4j.properties defaults instead. We do this in test resources but not
> in what we ship in conf/ .
> {noformat}
> INFO [] impl.MetricsSystemImpl: HBase metrics system started
> INFO [] impl.MetricsSystemImpl: Stopping HBase metrics
> system...
> INFO [] impl.MetricsSystemImpl: HBase metrics system stopped.
> INFO [] impl.MetricsConfig: loaded properties from
> hadoop-metrics2-hbase.properties
> INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at
> 10 second(s).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17703) TestThriftServerCmdLine is flaky in master branch
[
https://issues.apache.org/jira/browse/HBASE-17703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894923#comment-15894923
]
Hadoop QA commented on HBASE-17703:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 47s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 22s
{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 23s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855945/HBASE-17703.master.001.patch
|
| JIRA Issue | HBASE-17703 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 633044f64309 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 678ad0e |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5935/testReport/ |
| modules | C: hbase-thrift U: hbase-thrift |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5935/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TestThriftServerCmdLine is flaky in master branch
> -
>
> Key: HBASE-17703
> URL: https://issues.apache.org/jira/browse/HBASE-17703
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Jan Hentschel
> Attachments: HBASE-17703.master.001.patch
>
>
> According to
>
[jira] [Commented] (HBASE-15143) Procedure v2 - Web UI displaying queues
[
https://issues.apache.org/jira/browse/HBASE-15143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894903#comment-15894903
]
Hadoop QA commented on HBASE-15143:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 1s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 1s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
8s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 6s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 28s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 22m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 36 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s
{color} | {color:red} The patch 13 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 34s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 31s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 1s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 51s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 24s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 105m 35s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color}
